git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
						download: 
* [PATCH 3/5] submodule foreach: clarify the '$toplevel' variable documentation
      [irrelevant] <20180503005358.89082-1-sbeller@google.com>
  2018-05-03  0:53 ` [PATCH 2/5] submodule foreach: document '$sm_path' instead of '$path' Stefan Beller
@ 2018-05-03  0:53 ` Stefan Beller
  2018-05-03 17:51   ` Jonathan Tan
  2018-05-03  0:53 ` [PATCH 4/5] submodule foreach: document variable '$displaypath' Stefan Beller
                   ` (3 subsequent siblings)
  5 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-03  0:53 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, christian.couder, pc44800, Stefan Beller

From: Prathamesh Chavan <pc44800@gmail.com>

It does not contain the topmost superproject as the author assumed,
but the direct superproject, such that $toplevel/$sm_path is the
actual absolute path of the submodule.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 Documentation/git-submodule.txt | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 755ed695f08..408d5a0387f 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -188,7 +188,8 @@ foreach [--recursive] <command>::
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
 	$sha1 is the commit as recorded in the superproject, and
-	$toplevel is the absolute path to the top-level of the superproject.
+	$toplevel is the absolute path to its direct superproject, such that
+	$toplevel/$sm_path is the absolute path of the submodule.
 	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
 	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
-- 
2.17.0.441.gb46fe60e1d-goog


^ permalink raw reply	[relevance 20%]

* [PATCH 2/5] submodule foreach: document '$sm_path' instead of '$path'
      [irrelevant] <20180503005358.89082-1-sbeller@google.com>
@ 2018-05-03  0:53 ` Stefan Beller
  2018-05-03 17:50   ` Jonathan Tan
  2018-05-03  0:53 ` [PATCH 3/5] submodule foreach: clarify the '$toplevel' variable documentation Stefan Beller
                   ` (4 subsequent siblings)
  5 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-03  0:53 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, christian.couder, pc44800, Stefan Beller

From: Prathamesh Chavan <pc44800@gmail.com>

As using a variable '$path' may be harmful to users due to
capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
use $path variable in eval_gettext string, 2012-04-17). Adjust
the documentation to advocate for using $sm_path,  which contains
the same value. We still make the 'path' variable available and
document it as a deprecated synonym of 'sm_path'.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-submodule.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 71c5618e82a..755ed695f08 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,12 +183,14 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $path, $sha1 and
+	The command has access to the variables $name, $sm_path, $sha1 and
 	$toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
-	$path is the name of the submodule directory relative to the
-	superproject, $sha1 is the commit as recorded in the superproject,
-	and $toplevel is the absolute path to the top-level of the superproject.
+	$sm_path is the path of the submodule as recorded in the superproject,
+	$sha1 is the commit as recorded in the superproject, and
+	$toplevel is the absolute path to the top-level of the superproject.
+	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
+	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given `--quiet`, foreach prints the name
 	of each submodule before evaluating the command.
-- 
2.17.0.441.gb46fe60e1d-goog


^ permalink raw reply	[relevance 21%]

* [PATCH 4/5] submodule foreach: document variable '$displaypath'
      [irrelevant] <20180503005358.89082-1-sbeller@google.com>
  2018-05-03  0:53 ` [PATCH 2/5] submodule foreach: document '$sm_path' instead of '$path' Stefan Beller
  2018-05-03  0:53 ` [PATCH 3/5] submodule foreach: clarify the '$toplevel' variable documentation Stefan Beller
@ 2018-05-03  0:53 ` Stefan Beller
  2018-05-03  0:53 ` [PATCH 5/5] submodule: port submodule subcommand 'foreach' from shell to C Stefan Beller
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-03  0:53 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, christian.couder, pc44800, Stefan Beller

From: Prathamesh Chavan <pc44800@gmail.com>

It was observed that the variable '$displaypath' was accessible but
undocumented. Hence, document it.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-submodule.txt |  6 ++++--
 t/t7407-submodule-foreach.sh    | 22 +++++++++++-----------
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 408d5a0387f..4372d00c42e 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,10 +183,12 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $sm_path, $sha1 and
-	$toplevel:
+	The command has access to the variables $name, $sm_path, $displaypath,
+	$sha1 and $toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the superproject,
+	$displaypath contains the relative path from the current working
+	directory to the submodules root directory,
 	$sha1 is the commit as recorded in the superproject, and
 	$toplevel is the absolute path to its direct superproject, such that
 	$toplevel/$sm_path is the absolute path of the submodule.
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 5144cc6926b..77729ac4aa1 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,16 +82,16 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-sub1-$sub1sha1
+$pwd/clone-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-sub3-$sub3sha1
+$pwd/clone-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
 	mkdir clone/sub &&
 	(
 		cd clone/sub &&
-		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
@@ -206,25 +206,25 @@ submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEA
 
 cat >expect <<EOF
 Entering '../nested1'
-toplevel: $pwd/clone2 name: nested1 path: nested1 hash: $nested1sha1
+toplevel: $pwd/clone2 name: nested1 path: nested1 displaypath: ../nested1 hash: $nested1sha1
 Entering '../nested1/nested2'
-toplevel: $pwd/clone2/nested1 name: nested2 path: nested2 hash: $nested2sha1
+toplevel: $pwd/clone2/nested1 name: nested2 path: nested2 displaypath: ../nested1/nested2 hash: $nested2sha1
 Entering '../nested1/nested2/nested3'
-toplevel: $pwd/clone2/nested1/nested2 name: nested3 path: nested3 hash: $nested3sha1
+toplevel: $pwd/clone2/nested1/nested2 name: nested3 path: nested3 displaypath: ../nested1/nested2/nested3 hash: $nested3sha1
 Entering '../nested1/nested2/nested3/submodule'
-toplevel: $pwd/clone2/nested1/nested2/nested3 name: submodule path: submodule hash: $submodulesha1
+toplevel: $pwd/clone2/nested1/nested2/nested3 name: submodule path: submodule displaypath: ../nested1/nested2/nested3/submodule hash: $submodulesha1
 Entering '../sub1'
-toplevel: $pwd/clone2 name: foo1 path: sub1 hash: $sub1sha1
+toplevel: $pwd/clone2 name: foo1 path: sub1 displaypath: ../sub1 hash: $sub1sha1
 Entering '../sub2'
-toplevel: $pwd/clone2 name: foo2 path: sub2 hash: $sub2sha1
+toplevel: $pwd/clone2 name: foo2 path: sub2 displaypath: ../sub2 hash: $sub2sha1
 Entering '../sub3'
-toplevel: $pwd/clone2 name: foo3 path: sub3 hash: $sub3sha1
+toplevel: $pwd/clone2 name: foo3 path: sub3 displaypath: ../sub3 hash: $sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
 	(
 		cd clone2/untracked &&
-		git submodule foreach --recursive "echo toplevel: \$toplevel name: \$name path: \$sm_path hash: \$sha1" >../../actual
+		git submodule foreach --recursive "echo toplevel: \$toplevel name: \$name path: \$sm_path displaypath: \$displaypath hash: \$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
-- 
2.17.0.441.gb46fe60e1d-goog


^ permalink raw reply	[relevance 28%]

* [PATCH 5/5] submodule: port submodule subcommand 'foreach' from shell to C
      [irrelevant] <20180503005358.89082-1-sbeller@google.com>
                   ` (2 preceding siblings ...)
  2018-05-03  0:53 ` [PATCH 4/5] submodule foreach: document variable '$displaypath' Stefan Beller
@ 2018-05-03  0:53 ` Stefan Beller
  2018-05-03  1:06   ` Stefan Beller
  2018-05-03 18:05   ` Jonathan Tan
      [irrelevant] ` <20180503005358.89082-2-sbeller@google.com>
  2018-05-09  0:29 ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Stefan Beller
  5 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2018-05-03  0:53 UTC (permalink / raw)
  To: gitster; +Cc: git, jonathantanmy, christian.couder, pc44800, Stefan Beller

From: Prathamesh Chavan <pc44800@gmail.com>

This aims to make git-submodule foreach a builtin. 'foreach' is ported to
the submodule--helper, and submodule--helper is called from
git-submodule.sh.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 148 ++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  39 +---------
 2 files changed, 149 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index a404df3ea49..bbbea5868de 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -439,6 +439,153 @@ static void for_each_listed_submodule(const struct module_list *list,
 		fn(list->entries[i], cb_data);
 }
 
+struct cb_foreach {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	char *toplevel;
+	int quiet;
+	int recursive;
+};
+#define CB_FOREACH_INIT { 0 }
+
+static void runcommand_in_submodule_cb(const struct cache_entry *list_item,
+				       void *cb_data)
+{
+	struct cb_foreach *info = cb_data;
+	const char *path = list_item->name;
+	const struct object_id *ce_oid = &list_item->oid;
+
+	const struct submodule *sub;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *displaypath;
+
+	displaypath = get_submodule_displaypath(path, info->prefix);
+
+	sub = submodule_from_path(&null_oid, path);
+
+	if (!sub)
+		die(_("No url found for submodule path '%s' in .gitmodules"),
+			displaypath);
+
+	if (!is_submodule_populated_gently(path, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+
+	/*
+	* For the purpose of executing <command> in the submodule,
+	* separate shell is used for the purpose of running the
+	* child process.
+	*/
+	cp.use_shell = 1;
+	cp.dir = path;
+
+	/*
+	* NEEDSWORK: the command currently has access to the variables $name,
+	* $sm_path, $displaypath, $sha1 and $toplevel only when the command
+	* contains a single argument. This is done for maintaining a faithful
+	* translation from shell script.
+	*/
+	if (info->argc == 1) {
+		char *toplevel = xgetcwd();
+
+		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
+		argv_array_pushf(&cp.env_array, "sm_path=%s", path);
+		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
+		argv_array_pushf(&cp.env_array, "sha1=%s",
+				oid_to_hex(ce_oid));
+		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
+
+		/*
+		* Since the path variable was accessible from the script
+		* before porting, it is also made available after porting.
+		* The environment variable "PATH" has a very special purpose
+		* on windows. And since environment variables are
+		* case-insensitive in windows, it interferes with the
+		* existing PATH variable. Hence, to avoid that, we expose
+		* path via the args argv_array and not via env_array.
+		*/
+		argv_array_pushf(&cp.args, "path=%s; %s",
+				path, info->argv[0]);
+
+		free(toplevel);
+	} else {
+		argv_array_pushv(&cp.args, info->argv);
+	}
+
+	if (!info->quiet)
+		printf(_("Entering '%s'\n"), displaypath);
+
+	if (info->argv[0] && run_command(&cp))
+		die(_("run_command returned non-zero status for %s\n."),
+			displaypath);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = path;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", NULL);
+		argv_array_pushf(&cpr.args, "%s/", displaypath);
+		argv_array_pushl(&cpr.args, "submodule--helper", "foreach", "--recursive",
+				NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		if (info->toplevel)
+			argv_array_pushf(&cpr.args, "--toplevel=%s", info->toplevel);
+
+		argv_array_pushv(&cpr.args, info->argv);
+
+		if (run_command(&cpr))
+			die(_("run_command returned non-zero status while"
+				"recursing in the nested submodules of %s\n."),
+				displaypath);
+	}
+
+cleanup:
+	free(displaypath);
+}
+
+static int module_foreach(int argc, const char **argv, const char *prefix)
+{
+	struct cb_foreach info = CB_FOREACH_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+
+	struct option module_foreach_options[] = {
+		OPT__QUIET(&info.quiet, N_("Suppress output of entering each submodule command")),
+		OPT_BOOL(0, "recursive", &info.recursive,
+			 N_("Recurse into nested submodules")),
+		OPT_STRING(0, "toplevel", &info.toplevel, N_("path"),
+			   N_("path from the top level of the invocation")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_foreach_options,
+			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+
+	for_each_listed_submodule(&list, runcommand_in_submodule_cb, &info);
+
+	return 0;
+}
+
 struct init_cb {
 	const char *prefix;
 	unsigned int flags;
@@ -1838,6 +1985,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 331d71c908b..cba585f0754 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -323,44 +323,7 @@ cmd_foreach()
 		shift
 	done
 
-	toplevel=$(pwd)
-
-	# dup stdin so that it can be restored when running the external
-	# command in the subshell (and a recursive call to this function)
-	exec 3<&0
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		if test -e "$sm_path"/.git
-		then
-			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-			say "$(eval_gettext "Entering '\$displaypath'")"
-			name=$(git submodule--helper name "$sm_path")
-			(
-				prefix="$prefix$sm_path/"
-				sanitize_submodule_env
-				cd "$sm_path" &&
-				# we make $path available to scripts ...
-				path=$sm_path &&
-				if test $# -eq 1
-				then
-					eval "$1"
-				else
-					"$@"
-				fi &&
-				if test -n "$recursive"
-				then
-					cmd_foreach "--recursive" "$@"
-				fi
-			) <&3 3<&- ||
-			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 #
-- 
2.17.0.441.gb46fe60e1d-goog


^ permalink raw reply	[relevance 20%]

* Re: [PATCH 5/5] submodule: port submodule subcommand 'foreach' from shell to C
  2018-05-03  0:53 ` [PATCH 5/5] submodule: port submodule subcommand 'foreach' from shell to C Stefan Beller
@ 2018-05-03  1:06   ` Stefan Beller
  2018-05-03 18:05   ` Jonathan Tan
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-03  1:06 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jonathan Tan, Christian Couder, Prathamesh Chavan, Stefan Beller

On Wed, May 2, 2018 at 5:53 PM, Stefan Beller <sbeller@google.com> wrote:

> +struct cb_foreach {
> +       char *toplevel;
...
> +               OPT_STRING(0, "toplevel", &info.toplevel, N_("path"),
> +                          N_("path from the top level of the invocation")),

This is a leftover from my experimentation that I hinted at in the cover letter
that would be used to implement $topmost.  I'll remove this in a reroll.

^ permalink raw reply	[relevance 8%]

* [PATCH v4 0/3] Add --progress and --dissociate to git submodule
@ 2018-05-03 10:53 Casey Fitzpatrick
  0 siblings, 0 replies; 200+ results
From: Casey Fitzpatrick @ 2018-05-03 10:53 UTC (permalink / raw)
  To: git, sbeller, sunshine, gitster; +Cc: Casey Fitzpatrick

These patches add --progress and --dissociate options to git submodule.

The --progress option existed beforehand, but only for the update command and
it was left undocumented.

Both add and update submodule commands supported --reference, but not its pair
option --dissociate which allows for independent clones rather than depending
on the reference.

This is a resubmission with comments from Stefan Beller, Eric Sunshine, and
Junio C Hamano addressed.


Casey Fitzpatrick (3):
  submodule: clean up subsititions in script
  submodule: add --progress option to add command
  submodule: add --dissociate option to add/update commands

 Documentation/git-submodule.txt | 17 ++++++++++++++++-
 builtin/submodule--helper.c     | 16 +++++++++++++---
 git-submodule.sh                | 21 ++++++++++++++++-----
 t/t7400-submodule-basic.sh      | 16 ++++++++++++++++
 t/t7408-submodule-reference.sh  | 17 +++++++++++++++++
 5 files changed, 78 insertions(+), 9 deletions(-)

-- 
2.17.0.1.ge0414f29c.dirty


^ permalink raw reply	[relevance 8%]

* Re: [PATCH 1/5] submodule foreach: correct '$path' in nested submodules from a subdirectory
      [irrelevant] ` <20180503005358.89082-2-sbeller@google.com>
@ 2018-05-03 13:29   ` Ramsay Jones
  2018-05-03 17:47   ` Jonathan Tan
  1 sibling, 0 replies; 200+ results
From: Ramsay Jones @ 2018-05-03 13:29 UTC (permalink / raw)
  To: Stefan Beller, gitster; +Cc: git, jonathantanmy, christian.couder, pc44800



On 03/05/18 01:53, Stefan Beller wrote:
> From: Prathamesh Chavan <pc44800@gmail.com>
> 
> When running 'git submodule foreach --recursive' from a subdirectory of
> your repository, nested submodules get a bogus value for $path:
> For a submodule 'sub' that contains a nested submodule 'nested',
> running 'git -C dir submodule foreach echo $path' from the root of the
> superproject would report path='../nested' for the nested submodule.
> The first part '../' is derived from the logic computing the relative
> path from $pwd to the root of the superproject. The second part is the
> submodule path inside the submodule. This value is of little use and is
> hard to document.
> 
> There are three different possible solutions that have more value:
> (a) The path value is documented as the path from the toplevel of the
>     superproject to the mount point of the submodule. If 'the' refers to
>     the superproject holding this submodule ('sub' holding 'nested'),
>     the path would be expected to be path='nested'.
> (b) In case 'the' superproject is referring to the toplevel, which
>     is the superproject in which the original command was invoked,
>     then path is expected to be path='sub/nested'.
> (c) The documentation explains $path as [...] "relative to the
>     superproject", following 091a6eb0fe (submodule: drop the
>     top-level requirement, 2013-06-16), such that the nested submodule
>     would be expected as path='../sub/nested', when "the" superproject
>     is the superproject, where the command was run from
> (d) or the value of path='nested' is expected if we take the
>     intermediate superproject into account. [This is the same as
>     (a); it highlights that the documentation is not clear, but
>     technically correct if we were to revert 091a6eb0fe.]
> 
> The behavior for (c) was introduced in 091a6eb0fe (submodule: drop the
> top-level requirement, 2013-06-16) the intent for $path seemed to be
> relative to $cwd to the submodule worktree, but that did not work for
> nested submodules, as the intermittent submodules were not included in
----------------------------^^^^^^^^^^^^
intermediate

ATB,
Ramsay Jones

^ permalink raw reply	[relevance 5%]

* Re: git merge banch w/ different submodule revision
      [irrelevant]         ` <1525246025.2176.12.camel@klsmartin.com>
@ 2018-05-03 16:42           ` Heiko Voigt
  2018-05-04  8:29             ` Middelschulte, Leif
  0 siblings, 1 reply; 200+ results
From: Heiko Voigt @ 2018-05-03 16:42 UTC (permalink / raw)
  To: Middelschulte, Leif; +Cc: sbeller, git, jacob.keller

Hi,

On Wed, May 02, 2018 at 07:30:25AM +0000, Middelschulte, Leif wrote:
> Am Montag, den 30.04.2018, 19:02 +0200 schrieb Heiko Voigt:
> > On Thu, Apr 26, 2018 at 03:19:36PM -0700, Stefan Beller wrote:
> > > Stefan wrote:
> > > > See https://github.com/git/git/commit/68d03e4a6e448aa557f52adef92595ac4d6cd4bd
> > > > (68d03e4a6e (Implement automatic fast-forward merge for submodules, 2010-07-07)
> > > > to explain the situation you encounter. (specifically merge_submodule
> > > > at the end of the diff)
> > > 
> > > +cc Heiko, author of that commit.
> > 
> > In that commit we tried to be very careful about. I do not understand
> > the situation in which the current strategy would be wrong by default.
> > 
> > We only merge if the following applies:
> > 
> >  * The changes in the superproject on both sides point forward in the
> >    submodule.
> > 
> >  * One side is contained in the other. Contained from the submodule
> >    perspective. Sides from the superproject merge perspective.
> > 
> > So in case of the mentioned rewind of a submodule: Only one side of the
> > 3-way merge would point forward and the merge would fail.
> > 
> > I can imagine, that in case of a temporary revert of a commit in the
> > submodule that you would not want that merged into some other branch.
> > But that would be the same without submodules. If you merge a temporary
> > revert from another branch you will not get any conflict.
> > 
> > So maybe someone can explain the use case in which one would get the
> > results that seem wrong?
> In an ideal world, where there are no regressions between revisions, a
> fast-forward is appropriate. However, we might have regressions within
> submodules.
> 
> So the usecase is the following:
> 
> Environment:
> - We have a base library L that is developed by some team (Team B).
> - Another team (Team A) developes a product P based on those libraries using git-flow.
> 
> Case:
> The problem occurs, when a developer (D) of Team A tries to have a feature
> that he developed on a branch accepted by a core developer of P:
> If a core developer of P advanced the reference of L within P (linear history), he might
> deem the work D insufficient. Not because of the actual work by D, but regressions
> that snuck into L. The core developer will not be informed about the missmatching
> revisions of L.
> 
> So it would be nice if there was some kind of switch or at least some trigger.

I still do not understand how the current behaviour is mismatching with
users expectations. Let's assume that you directly tracked the files of
L in your product repository P, without any submodule boundary. How
would the behavior be different? Would it be? If D started on an older
revision and gets merged into a newer revision, there can always be
regressions even without submodules.

Why would the core developer need to be informed about mismatching
revisions if he himself advanced the submodule?

It seems to me that you do not want to mix integration testing and
testing of the feature itself. How about just testing/reviewing on the
branch then? You would still get the submodule revision D was working on
and then in a later stage check if integration with everything else
works.

Cheers Heiko

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 1/5] submodule foreach: correct '$path' in nested submodules from a subdirectory
      [irrelevant] ` <20180503005358.89082-2-sbeller@google.com>
  2018-05-03 13:29   ` [PATCH 1/5] submodule foreach: correct '$path' in nested submodules from a subdirectory Ramsay Jones
@ 2018-05-03 17:47   ` Jonathan Tan
  2018-05-03 18:12     ` Stefan Beller
  1 sibling, 1 reply; 200+ results
From: Jonathan Tan @ 2018-05-03 17:47 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, christian.couder, pc44800

On Wed,  2 May 2018 17:53:54 -0700
Stefan Beller <sbeller@google.com> wrote:

> From: Prathamesh Chavan <pc44800@gmail.com>
> 
> When running 'git submodule foreach --recursive' from a subdirectory of
> your repository, nested submodules get a bogus value for $path:

I know I said in a previous e-mail [1] that we should use $path instead
of $sm_path, but now I got confused because the test shows a difference
in $sm_path, not $path. Maybe add "(and $sm_path, which is an alias of
$path)".

[1] https://public-inbox.org/git/20180206145406.b759164cead02cd3bb3fdce0@google.com/

> There are three different possible solutions that have more value:
> (a) The path value is documented as the path from the toplevel of the
>     superproject to the mount point of the submodule. If 'the' refers to
>     the superproject holding this submodule ('sub' holding 'nested'),
>     the path would be expected to be path='nested'.

What is "the", and why is it quoted?

Also, in (c), you use the same indicative present tense as "The path
value is documented" to describe the current situation, whereas this
seems like a situation you're proposing. It would be clearer to use the
imperative here ("Document $path to be the path from the toplevel...").
Do the same for the others.

Also, whenever you mention "superproject", make it clear which
superproject you're referring to ("outermost superproject" and
"immediate superproject" seem like good terms to me).

> (b) In case 'the' superproject is referring to the toplevel, which
>     is the superproject in which the original command was invoked,
>     then path is expected to be path='sub/nested'.

Same comment about "the", and I think s/toplevel, which is the
superproject in which the original command was invoked/outermost
superproject/.

> (c) The documentation explains $path as [...] "relative to the
>     superproject", following 091a6eb0fe (submodule: drop the
>     top-level requirement, 2013-06-16), such that the nested submodule
>     would be expected as path='../sub/nested', when "the" superproject
>     is the superproject, where the command was run from

How does "relative to the superproject" result in "../" appearing in the
path? I would expect "../" to appear only if a path is relative to $pwd.

> (d) or the value of path='nested' is expected if we take the
>     intermediate superproject into account. [This is the same as
>     (a); it highlights that the documentation is not clear, but
>     technically correct if we were to revert 091a6eb0fe.]

How exactly are we taking the intermediate superproject into account?

> The behavior for (c) was introduced in 091a6eb0fe (submodule: drop the
> top-level requirement, 2013-06-16) the intent for $path seemed to be
> relative to $cwd to the submodule worktree, but that did not work for
> nested submodules, as the intermittent submodules were not included in
> the path.

I think "pwd" is more used in the Git project than "cwd", so maybe use
$pwd here.

> If we were to fix the meaning of the $path using (a), (d) such that "path"
> is "the path from the intermediate superproject to the mount point of the
> submodule", we would break any existing submodule user that runs foreach
> from non-root of the superproject as the non-nested submodule
> '../sub' would change its path to 'sub'.
> 
> If we were to fix the meaning of $path using (b) such that "path"
> is "the path from the topmost superproject to the mount point of the
> submodule", then we would break any user that uses nested submodules
> (even from the root directory) as the 'nested' would become 'sub/nested'.
> 
> If we were to fix the meaning of $path using (c) such that "path"
> is "the display path from where the original command was invoked to the
> submodule", then we would break users that rely on the assumption that
> "$toplevel / $path" is the absolute path of the submodule.
> 
> All groups can be found in the wild.  The author has no data if one group
> outweighs the other by large margin, and offending each one seems equally
> bad at first.  However in the authors imagination it is better to go with
> (a) as running from a sub directory sounds like it is carried out by a
> human rather than by some automation task.  With a human on the keyboard
> the feedback loop is short and the changed behavior can be adapted to
> quickly unlike some automation that can break silently.

As I said in my previous e-mail, this is a good analysis.

> Another argument in favor of (a) is the consistency of the variables
> provided, "$toplevel / $path" gives the absolute path of the submodule,
> with 'toplevel' (both the variable as well as the documentation) referring
> to the immediate superproject of the submodule.

It's confusing to me that $toplevel is not the path to the outermost
superproject, but the path to the immediate superproject, so I'm not
sure that the goodness of "$toplevel/$path" exists. I would omit this
paragraph.

> Documentation of the variable is adjusted in a follow-up patch.

By "the variable", I assume you mean $toplevel? If yes, this doesn't
seem relevant to this patch, since this patch does not change the
meaning of $toplevel at all.

^ permalink raw reply	[relevance 4%]

* Re: [PATCH 2/5] submodule foreach: document '$sm_path' instead of '$path'
  2018-05-03  0:53 ` [PATCH 2/5] submodule foreach: document '$sm_path' instead of '$path' Stefan Beller
@ 2018-05-03 17:50   ` Jonathan Tan
  0 siblings, 0 replies; 200+ results
From: Jonathan Tan @ 2018-05-03 17:50 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, christian.couder, pc44800

On Wed,  2 May 2018 17:53:55 -0700
Stefan Beller <sbeller@google.com> wrote:

>  foreach [--recursive] <command>::
>  	Evaluates an arbitrary shell command in each checked out submodule.
> -	The command has access to the variables $name, $path, $sha1 and
> +	The command has access to the variables $name, $sm_path, $sha1 and
>  	$toplevel:
>  	$name is the name of the relevant submodule section in `.gitmodules`,
> -	$path is the name of the submodule directory relative to the
> -	superproject, $sha1 is the commit as recorded in the superproject,
> -	and $toplevel is the absolute path to the top-level of the superproject.
> +	$sm_path is the path of the submodule as recorded in the superproject,
> +	$sha1 is the commit as recorded in the superproject, and
> +	$toplevel is the absolute path to the top-level of the superproject.
> +	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
> +	variable is now a deprecated synonym of '$sm_path' variable.
>  	Any submodules defined in the superproject but not checked out are
>  	ignored by this command. Unless given `--quiet`, foreach prints the name
>  	of each submodule before evaluating the command.

This patch is fine as-is. I would go further and replace all mentions of
"the superproject" to "its immediate superproject", but that is
optional.

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 3/5] submodule foreach: clarify the '$toplevel' variable documentation
  2018-05-03  0:53 ` [PATCH 3/5] submodule foreach: clarify the '$toplevel' variable documentation Stefan Beller
@ 2018-05-03 17:51   ` Jonathan Tan
  0 siblings, 0 replies; 200+ results
From: Jonathan Tan @ 2018-05-03 17:51 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, christian.couder, pc44800

On Wed,  2 May 2018 17:53:56 -0700
Stefan Beller <sbeller@google.com> wrote:

>  	$name is the name of the relevant submodule section in `.gitmodules`,
>  	$sm_path is the path of the submodule as recorded in the superproject,
>  	$sha1 is the commit as recorded in the superproject, and
> -	$toplevel is the absolute path to the top-level of the superproject.
> +	$toplevel is the absolute path to its direct superproject, such that
> +	$toplevel/$sm_path is the absolute path of the submodule.
>  	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
>  	variable is now a deprecated synonym of '$sm_path' variable.
>  	Any submodules defined in the superproject but not checked out are

Ah, I commented on patch 2 before reading patch 3 properly. I think that
if you follow my suggestions on patch 2, it will be obvious that
$toplevel/$sm_path is the absolute path of the submodule, so that this
patch is no longer needed.

^ permalink raw reply	[relevance 7%]

* Re: [PATCH 5/5] submodule: port submodule subcommand 'foreach' from shell to C
  2018-05-03  0:53 ` [PATCH 5/5] submodule: port submodule subcommand 'foreach' from shell to C Stefan Beller
  2018-05-03  1:06   ` Stefan Beller
@ 2018-05-03 18:05   ` Jonathan Tan
  1 sibling, 0 replies; 200+ results
From: Jonathan Tan @ 2018-05-03 18:05 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git, christian.couder, pc44800

On Wed,  2 May 2018 17:53:58 -0700
Stefan Beller <sbeller@google.com> wrote:

> +		argv_array_pushf(&cp.args, "path=%s; %s",
> +				path, info->argv[0]);

Do we need to quote the path here? (For example, what if path has a
quotation mark?)

Also, would it be useful to have a test testing a submodule with a
"weird" character (e.g. greater-than or single quote) in the name and
the path?

^ permalink raw reply	[relevance 8%]

* Re: [PATCH 1/5] submodule foreach: correct '$path' in nested submodules from a subdirectory
  2018-05-03 17:47   ` Jonathan Tan
@ 2018-05-03 18:12     ` Stefan Beller
  2018-05-04 21:03       ` Jonathan Tan
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-03 18:12 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: Junio C Hamano, git, Christian Couder, Prathamesh Chavan

On Thu, May 3, 2018 at 10:47 AM, Jonathan Tan <jonathantanmy@google.com> wrote:
> On Wed,  2 May 2018 17:53:54 -0700
> Stefan Beller <sbeller@google.com> wrote:
>
>> From: Prathamesh Chavan <pc44800@gmail.com>
>>
>> When running 'git submodule foreach --recursive' from a subdirectory of
>> your repository, nested submodules get a bogus value for $path:
>
> I know I said in a previous e-mail [1] that we should use $path instead
> of $sm_path, but now I got confused because the test shows a difference
> in $sm_path, not $path. Maybe add "(and $sm_path, which is an alias of
> $path)".
>
> [1] https://public-inbox.org/git/20180206145406.b759164cead02cd3bb3fdce0@google.com/
>
>> There are three different possible solutions that have more value:
>> (a) The path value is documented as the path from the toplevel of the
>>     superproject to the mount point of the submodule. If 'the' refers to
>>     the superproject holding this submodule ('sub' holding 'nested'),
>>     the path would be expected to be path='nested'.
>
> What is "the", and why is it quoted?

because it is unclear what is emphasizes.
It could be the intermediate (direct) superproject, or it
could be the topmost superproject (where the command was run from).

Just having "the", makes it unclear which of both it refers to.

> Also, in (c), you use the same indicative present tense as "The path
> value is documented" to describe the current situation, whereas this
> seems like a situation you're proposing. It would be clearer to use the
> imperative here ("Document $path to be the path from the toplevel...").
> Do the same for the others.
>
> Also, whenever you mention "superproject", make it clear which
> superproject you're referring to ("outermost superproject" and
> "immediate superproject" seem like good terms to me).

ok, I'll rewrite the commit message with clearer superproject
annotations.

>> (b) In case 'the' superproject is referring to the toplevel, which
>>     is the superproject in which the original command was invoked,
>>     then path is expected to be path='sub/nested'.
>
> Same comment about "the", and I think s/toplevel, which is the
> superproject in which the original command was invoked/outermost
> superproject/.

The outermost superproject may not be the one you invoke the
command in.

We have
* the direct superproject. You can access it currently via $toplevel,
  which is misleading
* the superproject, where the command was invoked from,
  Currently only the undocumented $displaypath gives hints
  at this
* the outermost superproject, which may be even further
  out than the previous superproject in this list; Consider
  a layout with 4 git repositories nested as follows:

    outmost/invoked/direct/submodule

Submodule is part of the superproject "direct", but the command
could have been invoked from outmost/invoked/dir which has "direct"
as a submodule at '../direct' and itself is a submodule of outmost.

IMHO 'outmost' is of no relevance to the discussion. If you care about
it, discover it yourself via repeated calls to
'git rev-parse --show-superproject-working-tree'

'invoked' is only interesting for $displaypath, but apart from that
there is no benefit in knowing that at the time of processing
'submodule'. (It doesn't contain information for submodule, as
all those configs are in 'direct')

'direct' is a better name than 'toplevel', which is confusing as it
could be understood as any of 'direct', 'invoked' or 'outmost'.

>> (c) The documentation explains $path as [...] "relative to the
>>     superproject", following 091a6eb0fe (submodule: drop the
>>     top-level requirement, 2013-06-16), such that the nested submodule
>>     would be expected as path='../sub/nested', when "the" superproject
>>     is the superproject, where the command was run from
>
> How does "relative to the superproject" result in "../" appearing in the
> path? I would expect "../" to appear only if a path is relative to $pwd.

Gah. I messed that up. I wanted to emphasize *relative* and not the
superproject that is merely mentioned to form a sentence there.

>
>> (d) or the value of path='nested' is expected if we take the
>>     intermediate superproject into account. [This is the same as
>>     (a); it highlights that the documentation is not clear, but
>>     technically correct if we were to revert 091a6eb0fe.]
>
> How exactly are we taking the intermediate superproject into account?

'nested' is the full in-tree path from the intermediate (direct) superproject
to that submodule.

>> The behavior for (c) was introduced in 091a6eb0fe (submodule: drop the
>> top-level requirement, 2013-06-16) the intent for $path seemed to be
>> relative to $cwd to the submodule worktree, but that did not work for
>> nested submodules, as the intermittent submodules were not included in
>> the path.
>
> I think "pwd" is more used in the Git project than "cwd", so maybe use
> $pwd here.

ok.

>
>> If we were to fix the meaning of the $path using (a), (d) such that "path"
>> is "the path from the intermediate superproject to the mount point of the
>> submodule", we would break any existing submodule user that runs foreach
>> from non-root of the superproject as the non-nested submodule
>> '../sub' would change its path to 'sub'.
>>
>> If we were to fix the meaning of $path using (b) such that "path"
>> is "the path from the topmost superproject to the mount point of the
>> submodule", then we would break any user that uses nested submodules
>> (even from the root directory) as the 'nested' would become 'sub/nested'.
>>
>> If we were to fix the meaning of $path using (c) such that "path"
>> is "the display path from where the original command was invoked to the
>> submodule", then we would break users that rely on the assumption that
>> "$toplevel / $path" is the absolute path of the submodule.
>>
>> All groups can be found in the wild.  The author has no data if one group
>> outweighs the other by large margin, and offending each one seems equally
>> bad at first.  However in the authors imagination it is better to go with
>> (a) as running from a sub directory sounds like it is carried out by a
>> human rather than by some automation task.  With a human on the keyboard
>> the feedback loop is short and the changed behavior can be adapted to
>> quickly unlike some automation that can break silently.
>
> As I said in my previous e-mail, this is a good analysis.
>
>> Another argument in favor of (a) is the consistency of the variables
>> provided, "$toplevel / $path" gives the absolute path of the submodule,
>> with 'toplevel' (both the variable as well as the documentation) referring
>> to the immediate superproject of the submodule.
>
> It's confusing to me that $toplevel is not the path to the outermost
> superproject,

yes. That confused me for a while, too. Maybe deprecate that (just like we
deprecate $path) and introduce $superproject to mean the direct
superproject that holds the submodule currently being processed?


> but the path to the immediate superproject, so I'm not
> sure that the goodness of "$toplevel/$path" exists. I would omit this
> paragraph.

This exists for all nested submodules currently as these are run
from its root tree. For non-nested submodules, you still have the
part that is relative from pwd to the submodule.


>> Documentation of the variable is adjusted in a follow-up patch.
>
> By "the variable", I assume you mean $toplevel? If yes, this doesn't
> seem relevant to this patch, since this patch does not change the
> meaning of $toplevel at all.

ok.

^ permalink raw reply	[relevance 8%]

* Re: git merge banch w/ different submodule revision
  2018-05-03 16:42           ` git merge banch w/ different submodule revision Heiko Voigt
@ 2018-05-04  8:29             ` Middelschulte, Leif
  0 siblings, 0 replies; 200+ results
From: Middelschulte, Leif @ 2018-05-04  8:29 UTC (permalink / raw)
  To: hvoigt; +Cc: git, sbeller, jacob.keller

Hi,
Am Donnerstag, den 03.05.2018, 18:42 +0200 schrieb Heiko Voigt:
> Hi,
> 
> On Wed, May 02, 2018 at 07:30:25AM +0000, Middelschulte, Leif wrote:
> > Am Montag, den 30.04.2018, 19:02 +0200 schrieb Heiko Voigt:
> > > On Thu, Apr 26, 2018 at 03:19:36PM -0700, Stefan Beller wrote:
> > > > Stefan wrote:
> > > > > See https://github.com/git/git/commit/68d03e4a6e448aa557f52adef92595ac4d6cd4bd
> > > > > (68d03e4a6e (Implement automatic fast-forward merge for submodules, 2010-07-07)
> > > > > to explain the situation you encounter. (specifically merge_submodule
> > > > > at the end of the diff)
> > > > 
> > > > +cc Heiko, author of that commit.
> > > 
> > > In that commit we tried to be very careful about. I do not understand
> > > the situation in which the current strategy would be wrong by default.
> > > 
> > > We only merge if the following applies:
> > > 
> > >  * The changes in the superproject on both sides point forward in the
> > >    submodule.
> > > 
> > >  * One side is contained in the other. Contained from the submodule
> > >    perspective. Sides from the superproject merge perspective.
> > > 
> > > So in case of the mentioned rewind of a submodule: Only one side of the
> > > 3-way merge would point forward and the merge would fail.
> > > 
> > > I can imagine, that in case of a temporary revert of a commit in the
> > > submodule that you would not want that merged into some other branch.
> > > But that would be the same without submodules. If you merge a temporary
> > > revert from another branch you will not get any conflict.
> > > 
> > > So maybe someone can explain the use case in which one would get the
> > > results that seem wrong?
> > 
> > In an ideal world, where there are no regressions between revisions, a
> > fast-forward is appropriate. However, we might have regressions within
> > submodules.
> > 
> > So the usecase is the following:
> > 
> > Environment:
> > - We have a base library L that is developed by some team (Team B).
> > - Another team (Team A) developes a product P based on those libraries using git-flow.
> > 
> > Case:
> > The problem occurs, when a developer (D) of Team A tries to have a feature
> > that he developed on a branch accepted by a core developer of P:
> > If a core developer of P advanced the reference of L within P (linear history), he might
> > deem the work D insufficient. Not because of the actual work by D, but regressions
> > that snuck into L. The core developer will not be informed about the missmatching
> > revisions of L.
> > 
> > So it would be nice if there was some kind of switch or at least some trigger.
> 
> I still do not understand how the current behaviour is mismatching with
> users expectations. Let's assume that you directly tracked the files of
> L in your product repository P, without any submodule boundary. How
> would the behavior be different? Would it be? If D started on an older
> revision and gets merged into a newer revision, there can always be
> regressions even without submodules.
> 
> Why would the core developer need to be informed about mismatching
> revisions if he himself advanced the submodule?
In that case you'd be right. I should have picked my example more wisely.
Assume right here that not a core developer, but another developer advanced
the submodule (also via feature branch + merge).
> 
> It seems to me that you do not want to mix integration testing and
> testing of the feature itself. 
That's on point. That's why it would be nice if git *at least* warned about the different revisions wrt submodules.

But, I guess, I learned something about submodules:
I used to think of submodules as means to pin down a specific revision like: `ver == x`.
Now I'm learning that submodules are treated as `ver >= x` during a merge.

> How about just testing/reviewing on the
> branch then? You would still get the submodule revision D was working on
> and then in a later stage check if integration with everything else
> works.
Sure. But if the behavior deviates after a merge the merging developer is currently not
aware that it *might* have to do with different submodule revisions used, not the "actual" code merged.

Like not even "beware: the (feature) branch you've merged used an 'older' revision of X"

> 
> Cheers Heiko

Cheers,

Leif

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 1/5] submodule foreach: correct '$path' in nested submodules from a subdirectory
  2018-05-03 18:12     ` Stefan Beller
@ 2018-05-04 21:03       ` Jonathan Tan
  0 siblings, 0 replies; 200+ results
From: Jonathan Tan @ 2018-05-04 21:03 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Junio C Hamano, git, Christian Couder, Prathamesh Chavan

On Thu, 3 May 2018 11:12:27 -0700
Stefan Beller <sbeller@google.com> wrote:

> >> There are three different possible solutions that have more value:
> >> (a) The path value is documented as the path from the toplevel of the
> >>     superproject to the mount point of the submodule. If 'the' refers to
> >>     the superproject holding this submodule ('sub' holding 'nested'),
> >>     the path would be expected to be path='nested'.
> >
> > What is "the", and why is it quoted?
> 
> because it is unclear what is emphasizes.
> It could be the intermediate (direct) superproject, or it
> could be the topmost superproject (where the command was run from).
> 
> Just having "the", makes it unclear which of both it refers to.

Ah, I see - so s/'the'/'the superproject'/

> >> (b) In case 'the' superproject is referring to the toplevel, which
> >>     is the superproject in which the original command was invoked,
> >>     then path is expected to be path='sub/nested'.

And here, s/'the' superproject/'the superproject'/

> > Same comment about "the", and I think s/toplevel, which is the
> > superproject in which the original command was invoked/outermost
> > superproject/.
> 
> The outermost superproject may not be the one you invoke the
> command in.

Good point. Maybe "the superproject the original command was run from",
but I'm open to a better name. So I would write the beginning as
follows:

  <your first paragraph starting with "When running">

  Also, in git-submodule.txt, $path is documented to be the "name of the
  submodule directory relative to the superproject", but "the
  superproject" is ambiguous.

  To resolve both these issues, we could:
  (a) Change "the superproject" to "its immediate superproject", so
      $path would be "nested" instead of "../nested".
  (b) Change "the superproject" to "the superproject the original
      command was run from", so $path would be "sub/nested" instead of
      "../nested".
  (c) Change "the superproject" to "the directory the original command
      was run from", so $path would be "../sub/nested" instead of
      "../nested".

Going back to the original patch:

> The behavior for (c) was introduced in 091a6eb0fe (submodule: drop the
> top-level requirement, 2013-06-16) the intent for $path seemed to be
> relative to $cwd to the submodule worktree, but that did not work for
> nested submodules, as the intermittent submodules were not included in
> the path.

The (c) behavior was never really introduced, right? 091a6eb0fe
attempted to introduce (c), but it didn't work when --recursive was
specified.

^ permalink raw reply	[relevance 8%]

* Re: Weak option parsing in `git submodule`
      [irrelevant] <1525630243.15782.4.camel@gmail.com>
@ 2018-05-07 19:05 ` Stefan Beller
  2018-05-08  7:56   ` Kaartic Sivaraam
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-07 19:05 UTC (permalink / raw)
  To: Kaartic Sivaraam; +Cc: Git Mailing list, Jens Lehmann

On Sun, May 6, 2018 at 11:10 AM, Kaartic Sivaraam
<kaartic.sivaraam@gmail.com> wrote:
> I recently faced the consequence of the weak option parsing done by in
> `git submodule`. Initially tried to add a new submodule using the `git
> submodule add` sub-command in the following way,
>
>     $ git submodule add ./foo/bar
>
> This cloned the submodule into a new directory named 'bar' in the
> present directory. This was initially confusing as I expected `git
> submodule` to use the actual directory itself as it resides inside a
> sub-directory the main project and thus avoiding the creation of a new
> clone. Then I came to know that `git submodule` wasn't smart enough yet
> to identify this, by reading the documentation. Then, after realizing
> that I would have to specify the path in the command to avoid the
> redundant clone, I corrected the command without consulting the
> documentation (thoroughly) to,
>
>     $ git submodule add ./foo/bar --path ./foo/bar
>
> expecting that the path needs to be specified after an argument. This
> is what triggered the weird consequence of weak option parsing. The
> output I got for the above command was:
>
>     The following path is ignored by one of your .gitignore files:
>     --path
>     Use -f if you really want to add it.

Yuck, that is bad UX.

> That really confused me pretty much (being a person who doesn't know
> much about how `git submodule` works). Instead of complaining about an
> inexistent option '--path', it considers '--path' to be the <path>
> argument which seems to be bad. It doesn't even complain about the
> extra argument. I also dug to find the reason behind this totally weird
> message. It seems to be a consequence of the following lousy check
> ($sm_path is the normalized <path> argument):
>
>     if test -z "$force" &&
>             ! git add --dry-run --ignore-missing --no-warn-embedded-repo "$sm_path" > /dev/null 2>&1
>     then
>             eval_gettextln "The following path is ignored by one of your .gitignore files:
>     \$sm_path
>     Use -f if you really want to add it." >&2
>             exit 1
>     fi
>
>     The lack of checking for the reason behind why `git add` fails seems to
>     be the reason behind that weird message.

(from the man page)
git submodule [--quiet] add [<options>] [--] <repository> [<path>]

When options are given after <repository> or <path> we can count
the arguments and know something is up. (The number of arguments
must be 1 or 2. If it is 3 or above, something fishy is going on), which
I would suggest as a first step.

>     Ways to fix this:
>
>     1. Fix this particular issue by adding a '--' after the '--no-warn-
>     embedded-repo' option in the above check. But that would also
>     require that we allow other parts of the script to accept weird
>     paths such as '--path'. Not so useful/helpful.
>
>     2. Check for the actual return value of `git add` in the check and
>     act accordingly. Also, check if there are unnecessary arguments for
>     `submodule add`.

The second part of this suggestion seems to me as the way to go.
Do you want to write a patch?

>     3. Tighten option parsing in the `git-submodule` script to ensure
>     this doesn't happen in the long term and helps users to get more
>     relevant error messages.
>
>     I find 3 to be a long term solution but not sure if it's worth trying
>     as there are efforts towards porting `git submodule` to C. So, I guess
>     we should at least do 2 as a short-term solution.

Yeah, bringing it to C, would be nice as a long term solution instead
of piling up more and more shell features. :)

Thanks for such a well written bug report with suggested bug fixes. :)
Stefan

^ permalink raw reply	[relevance 7%]

* [PATCH v2 01/13] repository: introduce parsed objects field
      [irrelevant] ` <20180507225916.155236-1-sbeller@google.com>
@ 2018-05-07 22:59   ` Stefan Beller
      [irrelevant]   ` <20180508193736.14883-1-sbeller@google.com>
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-07 22:59 UTC (permalink / raw)
  To: git; +Cc: pclouds, jonathantanmy, gitster, jamill, Stefan Beller

Convert the existing global cache for parsed objects (obj_hash) into
repository-specific parsed object caches. Existing code that uses
obj_hash are modified to use the parsed object cache of
the_repository; future patches will use the parsed object caches of
other repositories.

Another future use case for a pool of objects is ease of memory management
in revision walking: If we can free the rev-list related memory early in
pack-objects (e.g. part of repack operation) then it could lower memory
pressure significantly when running on large repos. While this has been
discussed on the mailing list lately, this series doesn't implement this.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c     | 63 +++++++++++++++++++++++++++++++++-------------------
 object.h     |  8 +++++++
 repository.c |  7 ++++++
 repository.h | 13 ++++++++++-
 4 files changed, 67 insertions(+), 24 deletions(-)

diff --git a/object.c b/object.c
index 5044d08e96c..f7c624a7ba6 100644
--- a/object.c
+++ b/object.c
@@ -8,17 +8,14 @@
 #include "object-store.h"
 #include "packfile.h"
 
-static struct object **obj_hash;
-static int nr_objs, obj_hash_size;
-
 unsigned int get_max_object_index(void)
 {
-	return obj_hash_size;
+	return the_repository->parsed_objects->obj_hash_size;
 }
 
 struct object *get_indexed_object(unsigned int idx)
 {
-	return obj_hash[idx];
+	return the_repository->parsed_objects->obj_hash[idx];
 }
 
 static const char *object_type_strings[] = {
@@ -90,15 +87,16 @@ struct object *lookup_object(const unsigned char *sha1)
 	unsigned int i, first;
 	struct object *obj;
 
-	if (!obj_hash)
+	if (!the_repository->parsed_objects->obj_hash)
 		return NULL;
 
-	first = i = hash_obj(sha1, obj_hash_size);
-	while ((obj = obj_hash[i]) != NULL) {
+	first = i = hash_obj(sha1,
+			     the_repository->parsed_objects->obj_hash_size);
+	while ((obj = the_repository->parsed_objects->obj_hash[i]) != NULL) {
 		if (!hashcmp(sha1, obj->oid.hash))
 			break;
 		i++;
-		if (i == obj_hash_size)
+		if (i == the_repository->parsed_objects->obj_hash_size)
 			i = 0;
 	}
 	if (obj && i != first) {
@@ -107,7 +105,8 @@ struct object *lookup_object(const unsigned char *sha1)
 		 * that we do not need to walk the hash table the next
 		 * time we look for it.
 		 */
-		SWAP(obj_hash[i], obj_hash[first]);
+		SWAP(the_repository->parsed_objects->obj_hash[i],
+		     the_repository->parsed_objects->obj_hash[first]);
 	}
 	return obj;
 }
@@ -124,19 +123,19 @@ static void grow_object_hash(void)
 	 * Note that this size must always be power-of-2 to match hash_obj
 	 * above.
 	 */
-	int new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;
+	int new_hash_size = the_repository->parsed_objects->obj_hash_size < 32 ? 32 : 2 * the_repository->parsed_objects->obj_hash_size;
 	struct object **new_hash;
 
 	new_hash = xcalloc(new_hash_size, sizeof(struct object *));
-	for (i = 0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i = 0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (!obj)
 			continue;
 		insert_obj_hash(obj, new_hash, new_hash_size);
 	}
-	free(obj_hash);
-	obj_hash = new_hash;
-	obj_hash_size = new_hash_size;
+	free(the_repository->parsed_objects->obj_hash);
+	the_repository->parsed_objects->obj_hash = new_hash;
+	the_repository->parsed_objects->obj_hash_size = new_hash_size;
 }
 
 void *create_object(const unsigned char *sha1, void *o)
@@ -147,11 +146,12 @@ void *create_object(const unsigned char *sha1, void *o)
 	obj->flags = 0;
 	hashcpy(obj->oid.hash, sha1);
 
-	if (obj_hash_size - 1 <= nr_objs * 2)
+	if (the_repository->parsed_objects->obj_hash_size - 1 <= the_repository->parsed_objects->nr_objs * 2)
 		grow_object_hash();
 
-	insert_obj_hash(obj, obj_hash, obj_hash_size);
-	nr_objs++;
+	insert_obj_hash(obj, the_repository->parsed_objects->obj_hash,
+			the_repository->parsed_objects->obj_hash_size);
+	the_repository->parsed_objects->nr_objs++;
 	return obj;
 }
 
@@ -431,8 +431,8 @@ void clear_object_flags(unsigned flags)
 {
 	int i;
 
-	for (i=0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i=0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (obj)
 			obj->flags &= ~flags;
 	}
@@ -442,13 +442,20 @@ void clear_commit_marks_all(unsigned int flags)
 {
 	int i;
 
-	for (i = 0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i = 0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (obj && obj->type == OBJ_COMMIT)
 			obj->flags &= ~flags;
 	}
 }
 
+struct parsed_object_pool *parsed_object_pool_new(void)
+{
+	struct parsed_object_pool *o = xmalloc(sizeof(*o));
+	memset(o, 0, sizeof(*o));
+	return o;
+}
+
 struct raw_object_store *raw_object_store_new(void)
 {
 	struct raw_object_store *o = xmalloc(sizeof(*o));
@@ -488,3 +495,13 @@ void raw_object_store_clear(struct raw_object_store *o)
 	close_all_packs(o);
 	o->packed_git = NULL;
 }
+
+void parsed_object_pool_clear(struct parsed_object_pool *o)
+{
+	/*
+	 * TOOD free objects in o->obj_hash.
+	 *
+	 * As objects are allocated in slabs (see alloc.c), we do
+	 * not need to free each object, but each slab instead.
+	 */
+}
diff --git a/object.h b/object.h
index f13f85b2a94..cecda7da370 100644
--- a/object.h
+++ b/object.h
@@ -1,6 +1,14 @@
 #ifndef OBJECT_H
 #define OBJECT_H
 
+struct parsed_object_pool {
+	struct object **obj_hash;
+	int nr_objs, obj_hash_size;
+};
+
+struct parsed_object_pool *parsed_object_pool_new(void);
+void parsed_object_pool_clear(struct parsed_object_pool *o);
+
 struct object_list {
 	struct object *item;
 	struct object_list *next;
diff --git a/repository.c b/repository.c
index a4848c1bd05..c23404677eb 100644
--- a/repository.c
+++ b/repository.c
@@ -2,6 +2,7 @@
 #include "repository.h"
 #include "object-store.h"
 #include "config.h"
+#include "object.h"
 #include "submodule-config.h"
 
 /* The main repository */
@@ -14,6 +15,8 @@ void initialize_the_repository(void)
 
 	the_repo.index = &the_index;
 	the_repo.objects = raw_object_store_new();
+	the_repo.parsed_objects = parsed_object_pool_new();
+
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
@@ -143,6 +146,7 @@ static int repo_init(struct repository *repo,
 	memset(repo, 0, sizeof(*repo));
 
 	repo->objects = raw_object_store_new();
+	repo->parsed_objects = parsed_object_pool_new();
 
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
@@ -226,6 +230,9 @@ void repo_clear(struct repository *repo)
 	raw_object_store_clear(repo->objects);
 	FREE_AND_NULL(repo->objects);
 
+	parsed_object_pool_clear(repo->parsed_objects);
+	FREE_AND_NULL(repo->parsed_objects);
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
diff --git a/repository.h b/repository.h
index e6e00f541bd..73389e81afd 100644
--- a/repository.h
+++ b/repository.h
@@ -22,10 +22,21 @@ struct repository {
 	char *commondir;
 
 	/*
-	 * Holds any information related to accessing the raw object content.
+	 * Holds any information needed to retrieve the raw content
+	 * of objects. The object_parser uses this to get object
+	 * content which it then parses.
 	 */
 	struct raw_object_store *objects;
 
+	/*
+	 * All objects in this repository that have been parsed. This structure
+	 * owns all objects it references, so users of "struct object *"
+	 * generally do not need to free them; instead, when a repository is no
+	 * longer used, call parsed_object_pool_clear() on this structure, which
+	 * is called by the repositories repo_clear on its desconstruction.
+	 */
+	struct parsed_object_pool *parsed_objects;
+
 	/* The store in which the refs are held. */
 	struct ref_store *refs;
 
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 4%]

* Re: Weak option parsing in `git submodule`
  2018-05-07 19:05 ` Weak option parsing in `git submodule` Stefan Beller
@ 2018-05-08  7:56   ` Kaartic Sivaraam
  0 siblings, 0 replies; 200+ results
From: Kaartic Sivaraam @ 2018-05-08  7:56 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Git Mailing list, Jens Lehmann

[-- Attachment #1.1: Type: text/plain, Size: 3996 bytes --]

On Tuesday 08 May 2018 12:35 AM, Stefan Beller wrote:
>>     The lack of checking for the reason behind why `git add` fails seems to
>>     be the reason behind that weird message.
> 
> (from the man page)
> git submodule [--quiet] add [<options>] [--] <repository> [<path>]
> 
> When options are given after <repository> or <path> we can count
> the arguments and know something is up. (The number of arguments
> must be 1 or 2. If it is 3 or above, something fishy is going on), which
> I would suggest as a first step.
> 
>>     Ways to fix this:
>>
>>     1. Fix this particular issue by adding a '--' after the '--no-warn-
>>     embedded-repo' option in the above check. But that would also
>>     require that we allow other parts of the script to accept weird
>>     paths such as '--path'. Not so useful/helpful.
>>
>>     2. Check for the actual return value of `git add` in the check and
>>     act accordingly. Also, check if there are unnecessary arguments for
>>     `submodule add`.
> 
> The second part of this suggestion seems to me as the way to go.
> Do you want to write a patch?
> 

Incidentally, I was writing a patch to check for the return value of
`git add` to fix the particular issue I noted in my initial message.
Then I was in a dilemma as to whether this was the right way to do it.
So, I thought it would be better to ask before continuing with the patch
and hence started this thread. I wasn't counting the arguments to `git
submodule add` at that time.

Now that I'm ensured that my initial approach is not the worst way to do
things and as I'm about to write a patch for this, I'll sum up what I'm
about to achieve in the short-term fix patch, for the sake of clarity.

	1. Check the return value of `git add ...` and throw an error
	   appropriately.

	2. Check the no. of arguments to `submodule add` and throw an
	   error if there are more arguments than there should be.

	   I require a little clarification with this. How should this
	   be done. Does checking whether the number of arguments after
	   <repository> is not more than one do the job? Or am I missing
	   something?


>>     3. Tighten option parsing in the `git-submodule` script to ensure
>>     this doesn't happen in the long term and helps users to get more
>>     relevant error messages.
>>
>>     I find 3 to be a long term solution but not sure if it's worth trying
>>     as there are efforts towards porting `git submodule` to C. So, I guess
>>     we should at least do 2 as a short-term solution.
> 
> Yeah, bringing it to C, would be nice as a long term solution instead
> of piling up more and more shell features. :)
> 

Hope the day it is brought into C comes soon.


> Thanks for such a well written bug report with suggested bug fixes. :)

You're welcome :-)


-- 
Sivaraam

QUOTE:

“The most valuable person on any team is the person who makes everyone
else on the team more valuable, not the person who knows the most.”

      - Joel Spolsky


Sivaraam?

You possibly might have noticed that my signature recently changed from
'Kaartic' to 'Sivaraam' both of which are parts of my name. I find the
new signature to be better for several reasons one of which is that the
former signature has a lot of ambiguities in the place I live as it is a
common name (NOTE: it's not a common spelling, just a common name). So,
I switched signatures before it's too late.

That said, I won't mind you calling me 'Kaartic' if you like it [of
course ;-)]. You can always call me using either of the names.


KIND NOTE TO THE NATIVE ENGLISH SPEAKER:

As I'm not a native English speaker myself, there might be mistaeks in
my usage of English. I apologise for any mistakes that I make.

It would be "helpful" if you take the time to point out the mistakes.

It would be "super helpful" if you could provide suggestions about how
to correct those mistakes.

Thanks in advance!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[relevance 6%]

* Re: [PATCH 1/2] gitk: show part of submodule log instead of empty pane when listing trees
      [irrelevant] ` <20180508122229.k3n3ccpa5g3g4dxa@pflmari>
@ 2018-05-08 17:07   ` Stefan Beller
  2018-05-09  9:01     ` Alex Riesen
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-08 17:07 UTC (permalink / raw)
  To: Alex Riesen; +Cc: Paul Mackerras, Git Mailing List, Junio C Hamano

On Tue, May 8, 2018 at 5:22 AM, Alex Riesen
<alexander.riesen@cetitec.com> wrote:
> From: Alex Riesen <raa.lkml@gmail.com>
>
> Currently, the submodules either are not shown at all (if listing a
> committed tree) or a Tcl error appears (when clicking on a submodule
> from the index list).

I do not understand where this appears, yet.
Where do I have to click to see the effects of this patch?

>
> This will make it show first arbitrarily chosen number of commits,
> which might be only marginally better.
>
> Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
> ---
>  gitk | 42 ++++++++++++++++++++++++++++++++----------
>  1 file changed, 32 insertions(+), 10 deletions(-)
>
> diff --git a/gitk b/gitk
> index a14d7a1..d34833f 100755
> --- a/gitk
> +++ b/gitk
> @@ -7627,9 +7627,10 @@ proc gettreeline {gtf id} {
>             if {$i < 0} continue
>             set fname [string range $line [expr {$i+1}] end]
>             set line [string range $line 0 [expr {$i-1}]]
> -           if {$diffids ne $nullid2 && [lindex $line 1] ne "blob"} continue
> +           set objtype [lindex $line 1]
> +           if {$diffids ne $nullid2 && $objtype ne "blob" && $objtype ne "commit" } { continue }
>             set sha1 [lindex $line 2]
> -           lappend treeidlist($id) $sha1
> +           lappend treeidlist($id) "$sha1 $objtype"
>         }
>         if {[string index $fname 0] eq "\""} {
>             set fname [lindex $fname 0]
> @@ -7659,21 +7660,42 @@ proc showfile {f} {
>      global ctext_file_names ctext_file_lines
>      global ctext commentend
>
> +    set submodlog "git\\ log\\ --format='%h\\ %aN:\\ %s'\\ -100"

Do we want to respect the config option diff.submodule here?
The -100 is chosen rather arbitrarily. Ideally we'd only walk to the
previous entry?

> +    set fcmt ""
>      set i [lsearch -exact $treefilelist($diffids) $f]
>      if {$i < 0} {
>         puts "oops, $f not in list for id $diffids"
>         return
>      }
>      if {$diffids eq $nullid} {
> -       if {[catch {set bf [open $f r]} err]} {
> -           puts "oops, can't read $f: $err"
> -           return
> +       if {[file isdirectory $f]} {
> +           # a submodule
> +           if {[catch {set bf [open "| sh -c cd\\ \"$f\"&&$submodlog" r]} err]} {

Can we have $submodlog use the "git -C <path> command"
option, then we could save the "cd &&" part, which might even
save us from spawning a shell?

Thanks,
Stefan

^ permalink raw reply	[relevance 7%]

* [PATCH v3 01/13] repository: introduce parsed objects field
      [irrelevant]   ` <20180508193736.14883-1-sbeller@google.com>
@ 2018-05-08 19:37     ` " Stefan Beller
      [irrelevant]     ` <20180510004024.93974-1-sbeller@google.com>
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-08 19:37 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jamill, jonathantanmy, pclouds

Convert the existing global cache for parsed objects (obj_hash) into
repository-specific parsed object caches. Existing code that uses
obj_hash are modified to use the parsed object cache of
the_repository; future patches will use the parsed object caches of
other repositories.

Another future use case for a pool of objects is ease of memory management
in revision walking: If we can free the rev-list related memory early in
pack-objects (e.g. part of repack operation) then it could lower memory
pressure significantly when running on large repos. While this has been
discussed on the mailing list lately, this series doesn't implement this.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c     | 63 +++++++++++++++++++++++++++++++++-------------------
 object.h     |  8 +++++++
 repository.c |  7 ++++++
 repository.h |  9 ++++++++
 4 files changed, 64 insertions(+), 23 deletions(-)

diff --git a/object.c b/object.c
index 5044d08e96c..f7c624a7ba6 100644
--- a/object.c
+++ b/object.c
@@ -8,17 +8,14 @@
 #include "object-store.h"
 #include "packfile.h"
 
-static struct object **obj_hash;
-static int nr_objs, obj_hash_size;
-
 unsigned int get_max_object_index(void)
 {
-	return obj_hash_size;
+	return the_repository->parsed_objects->obj_hash_size;
 }
 
 struct object *get_indexed_object(unsigned int idx)
 {
-	return obj_hash[idx];
+	return the_repository->parsed_objects->obj_hash[idx];
 }
 
 static const char *object_type_strings[] = {
@@ -90,15 +87,16 @@ struct object *lookup_object(const unsigned char *sha1)
 	unsigned int i, first;
 	struct object *obj;
 
-	if (!obj_hash)
+	if (!the_repository->parsed_objects->obj_hash)
 		return NULL;
 
-	first = i = hash_obj(sha1, obj_hash_size);
-	while ((obj = obj_hash[i]) != NULL) {
+	first = i = hash_obj(sha1,
+			     the_repository->parsed_objects->obj_hash_size);
+	while ((obj = the_repository->parsed_objects->obj_hash[i]) != NULL) {
 		if (!hashcmp(sha1, obj->oid.hash))
 			break;
 		i++;
-		if (i == obj_hash_size)
+		if (i == the_repository->parsed_objects->obj_hash_size)
 			i = 0;
 	}
 	if (obj && i != first) {
@@ -107,7 +105,8 @@ struct object *lookup_object(const unsigned char *sha1)
 		 * that we do not need to walk the hash table the next
 		 * time we look for it.
 		 */
-		SWAP(obj_hash[i], obj_hash[first]);
+		SWAP(the_repository->parsed_objects->obj_hash[i],
+		     the_repository->parsed_objects->obj_hash[first]);
 	}
 	return obj;
 }
@@ -124,19 +123,19 @@ static void grow_object_hash(void)
 	 * Note that this size must always be power-of-2 to match hash_obj
 	 * above.
 	 */
-	int new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;
+	int new_hash_size = the_repository->parsed_objects->obj_hash_size < 32 ? 32 : 2 * the_repository->parsed_objects->obj_hash_size;
 	struct object **new_hash;
 
 	new_hash = xcalloc(new_hash_size, sizeof(struct object *));
-	for (i = 0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i = 0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (!obj)
 			continue;
 		insert_obj_hash(obj, new_hash, new_hash_size);
 	}
-	free(obj_hash);
-	obj_hash = new_hash;
-	obj_hash_size = new_hash_size;
+	free(the_repository->parsed_objects->obj_hash);
+	the_repository->parsed_objects->obj_hash = new_hash;
+	the_repository->parsed_objects->obj_hash_size = new_hash_size;
 }
 
 void *create_object(const unsigned char *sha1, void *o)
@@ -147,11 +146,12 @@ void *create_object(const unsigned char *sha1, void *o)
 	obj->flags = 0;
 	hashcpy(obj->oid.hash, sha1);
 
-	if (obj_hash_size - 1 <= nr_objs * 2)
+	if (the_repository->parsed_objects->obj_hash_size - 1 <= the_repository->parsed_objects->nr_objs * 2)
 		grow_object_hash();
 
-	insert_obj_hash(obj, obj_hash, obj_hash_size);
-	nr_objs++;
+	insert_obj_hash(obj, the_repository->parsed_objects->obj_hash,
+			the_repository->parsed_objects->obj_hash_size);
+	the_repository->parsed_objects->nr_objs++;
 	return obj;
 }
 
@@ -431,8 +431,8 @@ void clear_object_flags(unsigned flags)
 {
 	int i;
 
-	for (i=0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i=0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (obj)
 			obj->flags &= ~flags;
 	}
@@ -442,13 +442,20 @@ void clear_commit_marks_all(unsigned int flags)
 {
 	int i;
 
-	for (i = 0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i = 0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (obj && obj->type == OBJ_COMMIT)
 			obj->flags &= ~flags;
 	}
 }
 
+struct parsed_object_pool *parsed_object_pool_new(void)
+{
+	struct parsed_object_pool *o = xmalloc(sizeof(*o));
+	memset(o, 0, sizeof(*o));
+	return o;
+}
+
 struct raw_object_store *raw_object_store_new(void)
 {
 	struct raw_object_store *o = xmalloc(sizeof(*o));
@@ -488,3 +495,13 @@ void raw_object_store_clear(struct raw_object_store *o)
 	close_all_packs(o);
 	o->packed_git = NULL;
 }
+
+void parsed_object_pool_clear(struct parsed_object_pool *o)
+{
+	/*
+	 * TOOD free objects in o->obj_hash.
+	 *
+	 * As objects are allocated in slabs (see alloc.c), we do
+	 * not need to free each object, but each slab instead.
+	 */
+}
diff --git a/object.h b/object.h
index f13f85b2a94..cecda7da370 100644
--- a/object.h
+++ b/object.h
@@ -1,6 +1,14 @@
 #ifndef OBJECT_H
 #define OBJECT_H
 
+struct parsed_object_pool {
+	struct object **obj_hash;
+	int nr_objs, obj_hash_size;
+};
+
+struct parsed_object_pool *parsed_object_pool_new(void);
+void parsed_object_pool_clear(struct parsed_object_pool *o);
+
 struct object_list {
 	struct object *item;
 	struct object_list *next;
diff --git a/repository.c b/repository.c
index a4848c1bd05..c23404677eb 100644
--- a/repository.c
+++ b/repository.c
@@ -2,6 +2,7 @@
 #include "repository.h"
 #include "object-store.h"
 #include "config.h"
+#include "object.h"
 #include "submodule-config.h"
 
 /* The main repository */
@@ -14,6 +15,8 @@ void initialize_the_repository(void)
 
 	the_repo.index = &the_index;
 	the_repo.objects = raw_object_store_new();
+	the_repo.parsed_objects = parsed_object_pool_new();
+
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
@@ -143,6 +146,7 @@ static int repo_init(struct repository *repo,
 	memset(repo, 0, sizeof(*repo));
 
 	repo->objects = raw_object_store_new();
+	repo->parsed_objects = parsed_object_pool_new();
 
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
@@ -226,6 +230,9 @@ void repo_clear(struct repository *repo)
 	raw_object_store_clear(repo->objects);
 	FREE_AND_NULL(repo->objects);
 
+	parsed_object_pool_clear(repo->parsed_objects);
+	FREE_AND_NULL(repo->parsed_objects);
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
diff --git a/repository.h b/repository.h
index e6e00f541bd..6d199819905 100644
--- a/repository.h
+++ b/repository.h
@@ -26,6 +26,15 @@ struct repository {
 	 */
 	struct raw_object_store *objects;
 
+	/*
+	 * All objects in this repository that have been parsed. This structure
+	 * owns all objects it references, so users of "struct object *"
+	 * generally do not need to free them; instead, when a repository is no
+	 * longer used, call parsed_object_pool_clear() on this structure, which
+	 * is called by the repositories repo_clear on its desconstruction.
+	 */
+	struct parsed_object_pool *parsed_objects;
+
 	/* The store in which the refs are held. */
 	struct ref_store *refs;
 
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 4%]

* [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach
      [irrelevant] <20180503005358.89082-1-sbeller@google.com>
                   ` (4 preceding siblings ...)
      [irrelevant] ` <20180503005358.89082-2-sbeller@google.com>
@ 2018-05-09  0:29 ` Stefan Beller
  2018-05-09  0:29   ` [PATCH 1/4] submodule foreach: correct '$path' in nested submodules from a subdirectory Stefan Beller
                     ` (4 more replies)
  5 siblings, 5 replies; 200+ results
From: Stefan Beller @ 2018-05-09  0:29 UTC (permalink / raw)
  To: sbeller; +Cc: christian.couder, git, gitster, jonathantanmy, pc44800

v2:
* rebased onto origin/master
* dropped leftover "toplevel" variable from experimentation
* reworded the commit message for the first patch extensively
* dropped the third patch
* see "branch-diff" below.

v1:
The "What's cooking" email carried this series for some time now:
> * pc/submodule-helper-foreach (2018-02-02) 5 commits
>  - submodule: port submodule subcommand 'foreach' from shell to C
> - submodule foreach: document variable '$displaypath'
>  - submodule foreach: clarify the '$toplevel' variable documentation
>  - submodule foreach: document '$sm_path' instead of '$path'
>  - submodule foreach: correct '$path' in nested submodules from a subdirectory
> 
>  Expecting a response to review comments
>  e.g. cf. <20180206150044.1bffbb573c088d38c8e44bf5@google.com>

I reworded the commit message of the first patch and nearly confused
myself again, as "toplevel" doesn't refer to the "topmost" superproject,
just the direct superproject of the submodule.

However I think the code of the first patch is correct as originally presented.
Just the wording of the commit message was changed to explain the reasoning
more extensively.

With this series, we get
* keep the invariant of $toplevel + $path to be an absolute path that is
  correctly pointing at the submodule. "git -C $toplevel config" + $name
  allowing to ask configuration of the submodule.  
* $displaypath will have the relative path form $pwd to the submodule root.
* better documentation.

In later patches we could add $topmost, that points at the superproject
in which the command was started from, and $path_from_topmost, that would
be the relative path from $topmost to the submodule, potentially skipping
intermediate superprojects.

Thanks,
Stefan

Prathamesh Chavan (4):
  submodule foreach: correct '$path' in nested submodules from a
    subdirectory
  submodule foreach: document '$sm_path' instead of '$path'
  submodule foreach: document variable '$displaypath'
  submodule: port submodule subcommand 'foreach' from shell to C

 Documentation/git-submodule.txt |  15 ++--
 builtin/submodule--helper.c     | 144 ++++++++++++++++++++++++++++++++
 git-submodule.sh                |  40 +--------
 t/t7407-submodule-foreach.sh    |  38 ++++++++-
 4 files changed, 190 insertions(+), 47 deletions(-)

-- 
2.17.0.255.g8bfb7c0704

1:  0c5f405db24 ! 1:  85f91b48379 submodule foreach: correct '$path' in nested submodules from a subdirectory
    @@ -12,45 +12,38 @@
         submodule path inside the submodule. This value is of little use and is
         hard to document.
     
    -    There are three different possible solutions that have more value:
    -    (a) The path value is documented as the path from the toplevel of the
    -        superproject to the mount point of the submodule. If 'the' refers to
    -        the superproject holding this submodule ('sub' holding 'nested'),
    -        the path would be expected to be path='nested'.
    -    (b) In case 'the' superproject is referring to the toplevel, which
    -        is the superproject in which the original command was invoked,
    -        then path is expected to be path='sub/nested'.
    -    (c) The documentation explains $path as [...] "relative to the
    -        superproject", following 091a6eb0fe (submodule: drop the
    -        top-level requirement, 2013-06-16), such that the nested submodule
    -        would be expected as path='../sub/nested', when "the" superproject
    -        is the superproject, where the command was run from
    -    (d) or the value of path='nested' is expected if we take the
    -        intermediate superproject into account. [This is the same as
    -        (a); it highlights that the documentation is not clear, but
    -        technically correct if we were to revert 091a6eb0fe.]
    +    Also, in git-submodule.txt, $path is documented to be the "name of the
    +    submodule directory relative to the superproject", but "the
    +    superproject" is ambiguous.
     
    -    The behavior for (c) was introduced in 091a6eb0fe (submodule: drop the
    -    top-level requirement, 2013-06-16) the intent for $path seemed to be
    -    relative to $cwd to the submodule worktree, but that did not work for
    -    nested submodules, as the intermittent submodules were not included in
    -    the path.
    +    To resolve both these issues, we could:
    +    (a) Change "the superproject" to "its immediate superproject", so
    +        $path would be "nested" instead of "../nested".
    +    (b) Change "the superproject" to "the superproject the original
    +        command was run from", so $path would be "sub/nested" instead of
    +        "../nested".
    +    (c) Change "the superproject" to "the directory the original command
    +        was run from", so $path would be "../sub/nested" instead of
    +        "../nested".
     
    -    If we were to fix the meaning of the $path using (a), (d) such that "path"
    -    is "the path from the intermediate superproject to the mount point of the
    -    submodule", we would break any existing submodule user that runs foreach
    -    from non-root of the superproject as the non-nested submodule
    -    '../sub' would change its path to 'sub'.
    +    The behavior for (c) was attempted to be introduced in 091a6eb0fe
    +    (submodule: drop the top-level requirement, 2013-06-16) with the intent
    +    for $path to be relative from $pwd to the submodule worktree, but that
    +    did not work for nested submodules, as the intermittent submodules
    +    were not included in the path.
     
    -    If we were to fix the meaning of $path using (b) such that "path"
    -    is "the path from the topmost superproject to the mount point of the
    -    submodule", then we would break any user that uses nested submodules
    -    (even from the root directory) as the 'nested' would become 'sub/nested'.
    +    If we were to fix the meaning of the $path using (a), we would break
    +    any existing submodule user that runs foreach from non-root of the
    +    superproject as the non-nested submodule '../sub' would change its
    +    path to 'sub'.
     
    -    If we were to fix the meaning of $path using (c) such that "path"
    -    is "the display path from where the original command was invoked to the
    -    submodule", then we would break users that rely on the assumption that
    -    "$toplevel / $path" is the absolute path of the submodule.
    +    If we were to fix the meaning of $path using (b), then we would break
    +    any user that uses nested submodules (even from the root directory)
    +    as the 'nested' would become 'sub/nested'.
    +
    +    If we were to fix the meaning of $path using (c), then we would break
    +    the same users as in (b) as 'nested' would become 'sub/nested' from
    +    the root directory of the superproject.
     
         All groups can be found in the wild.  The author has no data if one group
         outweighs the other by large margin, and offending each one seems equally
    @@ -60,13 +53,6 @@
         the feedback loop is short and the changed behavior can be adapted to
         quickly unlike some automation that can break silently.
     
    -    Another argument in favor of (a) is the consistency of the variables
    -    provided, "$toplevel / $path" gives the absolute path of the submodule,
    -    with 'toplevel' (both the variable as well as the documentation) referring
    -    to the immediate superproject of the submodule.
    -
    -    Documentation of the variable is adjusted in a follow-up patch.
    -
         Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
         Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
         Signed-off-by: Stefan Beller <sbeller@google.com>
2:  ae5e280a8d3 ! 2:  3a2cb4e0b01 submodule foreach: document '$sm_path' instead of '$path'
    @@ -28,9 +28,10 @@
     -	$path is the name of the submodule directory relative to the
     -	superproject, $sha1 is the commit as recorded in the superproject,
     -	and $toplevel is the absolute path to the top-level of the superproject.
    -+	$sm_path is the path of the submodule as recorded in the superproject,
    -+	$sha1 is the commit as recorded in the superproject, and
    -+	$toplevel is the absolute path to the top-level of the superproject.
    ++	$sm_path is the path of the submodule as recorded in the immediate
    ++	superproject, $sha1 is the commit as recorded in the immediate
    ++	superproject, and $toplevel is the absolute path to the top-level
    ++	of the immediate superproject.
     +	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
     +	variable is now a deprecated synonym of '$sm_path' variable.
      	Any submodules defined in the superproject but not checked out are
3:  23a5f04f846 < -:  ----------- submodule foreach: clarify the '$toplevel' variable documentation
4:  3efed1cdb80 ! 3:  3f4686e8395 submodule foreach: document variable '$displaypath'
    @@ -22,12 +22,14 @@
     +	The command has access to the variables $name, $sm_path, $displaypath,
     +	$sha1 and $toplevel:
      	$name is the name of the relevant submodule section in `.gitmodules`,
    - 	$sm_path is the path of the submodule as recorded in the superproject,
    -+	$displaypath contains the relative path from the current working
    -+	directory to the submodules root directory,
    - 	$sha1 is the commit as recorded in the superproject, and
    - 	$toplevel is the absolute path to its direct superproject, such that
    - 	$toplevel/$sm_path is the absolute path of the submodule.
    + 	$sm_path is the path of the submodule as recorded in the immediate
    +-	superproject, $sha1 is the commit as recorded in the immediate
    ++	superproject, $displaypath contains the relative path from the
    ++	current working directory to the submodules root directory,
    ++	$sha1 is the commit as recorded in the immediate
    + 	superproject, and $toplevel is the absolute path to the top-level
    + 	of the immediate superproject.
    + 	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
     
     diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
     --- a/t/t7407-submodule-foreach.sh
5:  2b17d6891b3 ! 4:  0c677680928 submodule: port submodule subcommand 'foreach' from shell to C
    @@ -23,7 +23,6 @@
     +	int argc;
     +	const char **argv;
     +	const char *prefix;
    -+	char *toplevel;
     +	int quiet;
     +	int recursive;
     +};
    @@ -42,7 +41,7 @@
     +
     +	displaypath = get_submodule_displaypath(path, info->prefix);
     +
    -+	sub = submodule_from_path(&null_oid, path);
    ++	sub = submodule_from_path(the_repository, &null_oid, path);
     +
     +	if (!sub)
     +		die(_("No url found for submodule path '%s' in .gitmodules"),
    @@ -69,6 +68,7 @@
     +	*/
     +	if (info->argc == 1) {
     +		char *toplevel = xgetcwd();
    ++		struct strbuf sb = STRBUF_INIT;
     +
     +		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
     +		argv_array_pushf(&cp.env_array, "sm_path=%s", path);
    @@ -86,9 +86,10 @@
     +		* existing PATH variable. Hence, to avoid that, we expose
     +		* path via the args argv_array and not via env_array.
     +		*/
    ++		sq_quote_buf(&sb, path);
     +		argv_array_pushf(&cp.args, "path=%s; %s",
    -+				path, info->argv[0]);
    -+
    ++				 sb.buf, info->argv[0]);
    ++		strbuf_release(&sb);
     +		free(toplevel);
     +	} else {
     +		argv_array_pushv(&cp.args, info->argv);
    @@ -116,9 +117,6 @@
     +		if (info->quiet)
     +			argv_array_push(&cpr.args, "--quiet");
     +
    -+		if (info->toplevel)
    -+			argv_array_pushf(&cpr.args, "--toplevel=%s", info->toplevel);
    -+
     +		argv_array_pushv(&cpr.args, info->argv);
     +
     +		if (run_command(&cpr))
    @@ -141,8 +139,6 @@
     +		OPT__QUIET(&info.quiet, N_("Suppress output of entering each submodule command")),
     +		OPT_BOOL(0, "recursive", &info.recursive,
     +			 N_("Recurse into nested submodules")),
    -+		OPT_STRING(0, "toplevel", &info.toplevel, N_("path"),
    -+			   N_("path from the top level of the invocation")),
     +		OPT_END()
     +	};
     +

^ permalink raw reply	[relevance 6%]

* [PATCH 1/4] submodule foreach: correct '$path' in nested submodules from a subdirectory
  2018-05-09  0:29 ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Stefan Beller
@ 2018-05-09  0:29   ` Stefan Beller
  2018-05-09  0:29   ` [PATCH 2/4] submodule foreach: document '$sm_path' instead of '$path' Stefan Beller
                     ` (3 subsequent siblings)
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-09  0:29 UTC (permalink / raw)
  To: sbeller; +Cc: christian.couder, git, gitster, jonathantanmy, pc44800

From: Prathamesh Chavan <pc44800@gmail.com>

When running 'git submodule foreach --recursive' from a subdirectory of
your repository, nested submodules get a bogus value for $path:
For a submodule 'sub' that contains a nested submodule 'nested',
running 'git -C dir submodule foreach echo $path' from the root of the
superproject would report path='../nested' for the nested submodule.
The first part '../' is derived from the logic computing the relative
path from $pwd to the root of the superproject. The second part is the
submodule path inside the submodule. This value is of little use and is
hard to document.

Also, in git-submodule.txt, $path is documented to be the "name of the
submodule directory relative to the superproject", but "the
superproject" is ambiguous.

To resolve both these issues, we could:
(a) Change "the superproject" to "its immediate superproject", so
    $path would be "nested" instead of "../nested".
(b) Change "the superproject" to "the superproject the original
    command was run from", so $path would be "sub/nested" instead of
    "../nested".
(c) Change "the superproject" to "the directory the original command
    was run from", so $path would be "../sub/nested" instead of
    "../nested".

The behavior for (c) was attempted to be introduced in 091a6eb0fe
(submodule: drop the top-level requirement, 2013-06-16) with the intent
for $path to be relative from $pwd to the submodule worktree, but that
did not work for nested submodules, as the intermittent submodules
were not included in the path.

If we were to fix the meaning of the $path using (a), we would break
any existing submodule user that runs foreach from non-root of the
superproject as the non-nested submodule '../sub' would change its
path to 'sub'.

If we were to fix the meaning of $path using (b), then we would break
any user that uses nested submodules (even from the root directory)
as the 'nested' would become 'sub/nested'.

If we were to fix the meaning of $path using (c), then we would break
the same users as in (b) as 'nested' would become 'sub/nested' from
the root directory of the superproject.

All groups can be found in the wild.  The author has no data if one group
outweighs the other by large margin, and offending each one seems equally
bad at first.  However in the authors imagination it is better to go with
(a) as running from a sub directory sounds like it is carried out by a
human rather than by some automation task.  With a human on the keyboard
the feedback loop is short and the changed behavior can be adapted to
quickly unlike some automation that can break silently.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh             |  1 -
 t/t7407-submodule-foreach.sh | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 24914963ca2..331d71c908b 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -345,7 +345,6 @@ cmd_foreach()
 				prefix="$prefix$sm_path/"
 				sanitize_submodule_env
 				cd "$sm_path" &&
-				sm_path=$(git submodule--helper relative-path "$sm_path" "$wt_prefix") &&
 				# we make $path available to scripts ...
 				path=$sm_path &&
 				if test $# -eq 1
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 6ba5daf42ee..5144cc6926b 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,9 +82,9 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-../sub1-$sub1sha1
+$pwd/clone-foo1-sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-../sub3-$sub3sha1
+$pwd/clone-foo3-sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
@@ -196,6 +196,38 @@ test_expect_success 'test messages from "foreach --recursive" from subdirectory'
 	) &&
 	test_i18ncmp expect actual
 '
+sub1sha1=$(cd clone2/sub1 && git rev-parse HEAD)
+sub2sha1=$(cd clone2/sub2 && git rev-parse HEAD)
+sub3sha1=$(cd clone2/sub3 && git rev-parse HEAD)
+nested1sha1=$(cd clone2/nested1 && git rev-parse HEAD)
+nested2sha1=$(cd clone2/nested1/nested2 && git rev-parse HEAD)
+nested3sha1=$(cd clone2/nested1/nested2/nested3 && git rev-parse HEAD)
+submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEAD)
+
+cat >expect <<EOF
+Entering '../nested1'
+toplevel: $pwd/clone2 name: nested1 path: nested1 hash: $nested1sha1
+Entering '../nested1/nested2'
+toplevel: $pwd/clone2/nested1 name: nested2 path: nested2 hash: $nested2sha1
+Entering '../nested1/nested2/nested3'
+toplevel: $pwd/clone2/nested1/nested2 name: nested3 path: nested3 hash: $nested3sha1
+Entering '../nested1/nested2/nested3/submodule'
+toplevel: $pwd/clone2/nested1/nested2/nested3 name: submodule path: submodule hash: $submodulesha1
+Entering '../sub1'
+toplevel: $pwd/clone2 name: foo1 path: sub1 hash: $sub1sha1
+Entering '../sub2'
+toplevel: $pwd/clone2 name: foo2 path: sub2 hash: $sub2sha1
+Entering '../sub3'
+toplevel: $pwd/clone2 name: foo3 path: sub3 hash: $sub3sha1
+EOF
+
+test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
+	(
+		cd clone2/untracked &&
+		git submodule foreach --recursive "echo toplevel: \$toplevel name: \$name path: \$sm_path hash: \$sha1" >../../actual
+	) &&
+	test_i18ncmp expect actual
+'
 
 cat > expect <<EOF
 nested1-nested1
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 22%]

* [PATCH 2/4] submodule foreach: document '$sm_path' instead of '$path'
  2018-05-09  0:29 ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Stefan Beller
  2018-05-09  0:29   ` [PATCH 1/4] submodule foreach: correct '$path' in nested submodules from a subdirectory Stefan Beller
@ 2018-05-09  0:29   ` Stefan Beller
  2018-05-09  0:29   ` [PATCH 3/4] submodule foreach: document variable '$displaypath' Stefan Beller
                     ` (2 subsequent siblings)
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-09  0:29 UTC (permalink / raw)
  To: sbeller; +Cc: christian.couder, git, gitster, jonathantanmy, pc44800

From: Prathamesh Chavan <pc44800@gmail.com>

As using a variable '$path' may be harmful to users due to
capitalization issues, see 64394e3ae9 (git-submodule.sh: Don't
use $path variable in eval_gettext string, 2012-04-17). Adjust
the documentation to advocate for using $sm_path,  which contains
the same value. We still make the 'path' variable available and
document it as a deprecated synonym of 'sm_path'.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-submodule.txt | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 630999f41a9..066c7b6c18e 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,12 +183,15 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $path, $sha1 and
+	The command has access to the variables $name, $sm_path, $sha1 and
 	$toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
-	$path is the name of the submodule directory relative to the
-	superproject, $sha1 is the commit as recorded in the superproject,
-	and $toplevel is the absolute path to the top-level of the superproject.
+	$sm_path is the path of the submodule as recorded in the immediate
+	superproject, $sha1 is the commit as recorded in the immediate
+	superproject, and $toplevel is the absolute path to the top-level
+	of the immediate superproject.
+	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
+	variable is now a deprecated synonym of '$sm_path' variable.
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given `--quiet`, foreach prints the name
 	of each submodule before evaluating the command.
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 21%]

* [PATCH 3/4] submodule foreach: document variable '$displaypath'
  2018-05-09  0:29 ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Stefan Beller
  2018-05-09  0:29   ` [PATCH 1/4] submodule foreach: correct '$path' in nested submodules from a subdirectory Stefan Beller
  2018-05-09  0:29   ` [PATCH 2/4] submodule foreach: document '$sm_path' instead of '$path' Stefan Beller
@ 2018-05-09  0:29   ` Stefan Beller
  2018-05-09  0:29   ` [PATCH 4/4] submodule: port submodule subcommand 'foreach' from shell to C Stefan Beller
  2018-05-09 17:13   ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Jonathan Tan
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-09  0:29 UTC (permalink / raw)
  To: sbeller; +Cc: christian.couder, git, gitster, jonathantanmy, pc44800

From: Prathamesh Chavan <pc44800@gmail.com>

It was observed that the variable '$displaypath' was accessible but
undocumented. Hence, document it.

Discussed-with: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-submodule.txt |  8 +++++---
 t/t7407-submodule-foreach.sh    | 22 +++++++++++-----------
 2 files changed, 16 insertions(+), 14 deletions(-)

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 066c7b6c18e..500dfdafd16 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -183,11 +183,13 @@ information too.
 
 foreach [--recursive] <command>::
 	Evaluates an arbitrary shell command in each checked out submodule.
-	The command has access to the variables $name, $sm_path, $sha1 and
-	$toplevel:
+	The command has access to the variables $name, $sm_path, $displaypath,
+	$sha1 and $toplevel:
 	$name is the name of the relevant submodule section in `.gitmodules`,
 	$sm_path is the path of the submodule as recorded in the immediate
-	superproject, $sha1 is the commit as recorded in the immediate
+	superproject, $displaypath contains the relative path from the
+	current working directory to the submodules root directory,
+	$sha1 is the commit as recorded in the immediate
 	superproject, and $toplevel is the absolute path to the top-level
 	of the immediate superproject.
 	Note that to avoid conflicts with '$PATH' on Windows, the '$path'
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 5144cc6926b..77729ac4aa1 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -82,16 +82,16 @@ test_expect_success 'test basic "submodule foreach" usage' '
 
 cat >expect <<EOF
 Entering '../sub1'
-$pwd/clone-foo1-sub1-$sub1sha1
+$pwd/clone-foo1-sub1-../sub1-$sub1sha1
 Entering '../sub3'
-$pwd/clone-foo3-sub3-$sub3sha1
+$pwd/clone-foo3-sub3-../sub3-$sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach" from subdirectory' '
 	mkdir clone/sub &&
 	(
 		cd clone/sub &&
-		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$sha1" >../../actual
+		git submodule foreach "echo \$toplevel-\$name-\$sm_path-\$displaypath-\$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
@@ -206,25 +206,25 @@ submodulesha1=$(cd clone2/nested1/nested2/nested3/submodule && git rev-parse HEA
 
 cat >expect <<EOF
 Entering '../nested1'
-toplevel: $pwd/clone2 name: nested1 path: nested1 hash: $nested1sha1
+toplevel: $pwd/clone2 name: nested1 path: nested1 displaypath: ../nested1 hash: $nested1sha1
 Entering '../nested1/nested2'
-toplevel: $pwd/clone2/nested1 name: nested2 path: nested2 hash: $nested2sha1
+toplevel: $pwd/clone2/nested1 name: nested2 path: nested2 displaypath: ../nested1/nested2 hash: $nested2sha1
 Entering '../nested1/nested2/nested3'
-toplevel: $pwd/clone2/nested1/nested2 name: nested3 path: nested3 hash: $nested3sha1
+toplevel: $pwd/clone2/nested1/nested2 name: nested3 path: nested3 displaypath: ../nested1/nested2/nested3 hash: $nested3sha1
 Entering '../nested1/nested2/nested3/submodule'
-toplevel: $pwd/clone2/nested1/nested2/nested3 name: submodule path: submodule hash: $submodulesha1
+toplevel: $pwd/clone2/nested1/nested2/nested3 name: submodule path: submodule displaypath: ../nested1/nested2/nested3/submodule hash: $submodulesha1
 Entering '../sub1'
-toplevel: $pwd/clone2 name: foo1 path: sub1 hash: $sub1sha1
+toplevel: $pwd/clone2 name: foo1 path: sub1 displaypath: ../sub1 hash: $sub1sha1
 Entering '../sub2'
-toplevel: $pwd/clone2 name: foo2 path: sub2 hash: $sub2sha1
+toplevel: $pwd/clone2 name: foo2 path: sub2 displaypath: ../sub2 hash: $sub2sha1
 Entering '../sub3'
-toplevel: $pwd/clone2 name: foo3 path: sub3 hash: $sub3sha1
+toplevel: $pwd/clone2 name: foo3 path: sub3 displaypath: ../sub3 hash: $sub3sha1
 EOF
 
 test_expect_success 'test "submodule foreach --recursive" from subdirectory' '
 	(
 		cd clone2/untracked &&
-		git submodule foreach --recursive "echo toplevel: \$toplevel name: \$name path: \$sm_path hash: \$sha1" >../../actual
+		git submodule foreach --recursive "echo toplevel: \$toplevel name: \$name path: \$sm_path displaypath: \$displaypath hash: \$sha1" >../../actual
 	) &&
 	test_i18ncmp expect actual
 '
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 28%]

* [PATCH 4/4] submodule: port submodule subcommand 'foreach' from shell to C
  2018-05-09  0:29 ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Stefan Beller
                     ` (2 preceding siblings ...)
  2018-05-09  0:29   ` [PATCH 3/4] submodule foreach: document variable '$displaypath' Stefan Beller
@ 2018-05-09  0:29   ` Stefan Beller
  2018-05-10  6:37     ` Junio C Hamano
  2018-05-09 17:13   ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Jonathan Tan
  4 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-09  0:29 UTC (permalink / raw)
  To: sbeller; +Cc: christian.couder, git, gitster, jonathantanmy, pc44800

From: Prathamesh Chavan <pc44800@gmail.com>

This aims to make git-submodule foreach a builtin. 'foreach' is ported to
the submodule--helper, and submodule--helper is called from
git-submodule.sh.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 144 ++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  39 +---------
 2 files changed, 145 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index c2403a915ff..4b0b773c06b 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -439,6 +439,149 @@ static void for_each_listed_submodule(const struct module_list *list,
 		fn(list->entries[i], cb_data);
 }
 
+struct cb_foreach {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	int quiet;
+	int recursive;
+};
+#define CB_FOREACH_INIT { 0 }
+
+static void runcommand_in_submodule_cb(const struct cache_entry *list_item,
+				       void *cb_data)
+{
+	struct cb_foreach *info = cb_data;
+	const char *path = list_item->name;
+	const struct object_id *ce_oid = &list_item->oid;
+
+	const struct submodule *sub;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *displaypath;
+
+	displaypath = get_submodule_displaypath(path, info->prefix);
+
+	sub = submodule_from_path(the_repository, &null_oid, path);
+
+	if (!sub)
+		die(_("No url found for submodule path '%s' in .gitmodules"),
+			displaypath);
+
+	if (!is_submodule_populated_gently(path, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+
+	/*
+	* For the purpose of executing <command> in the submodule,
+	* separate shell is used for the purpose of running the
+	* child process.
+	*/
+	cp.use_shell = 1;
+	cp.dir = path;
+
+	/*
+	* NEEDSWORK: the command currently has access to the variables $name,
+	* $sm_path, $displaypath, $sha1 and $toplevel only when the command
+	* contains a single argument. This is done for maintaining a faithful
+	* translation from shell script.
+	*/
+	if (info->argc == 1) {
+		char *toplevel = xgetcwd();
+		struct strbuf sb = STRBUF_INIT;
+
+		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
+		argv_array_pushf(&cp.env_array, "sm_path=%s", path);
+		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
+		argv_array_pushf(&cp.env_array, "sha1=%s",
+				oid_to_hex(ce_oid));
+		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
+
+		/*
+		* Since the path variable was accessible from the script
+		* before porting, it is also made available after porting.
+		* The environment variable "PATH" has a very special purpose
+		* on windows. And since environment variables are
+		* case-insensitive in windows, it interferes with the
+		* existing PATH variable. Hence, to avoid that, we expose
+		* path via the args argv_array and not via env_array.
+		*/
+		sq_quote_buf(&sb, path);
+		argv_array_pushf(&cp.args, "path=%s; %s",
+				 sb.buf, info->argv[0]);
+		strbuf_release(&sb);
+		free(toplevel);
+	} else {
+		argv_array_pushv(&cp.args, info->argv);
+	}
+
+	if (!info->quiet)
+		printf(_("Entering '%s'\n"), displaypath);
+
+	if (info->argv[0] && run_command(&cp))
+		die(_("run_command returned non-zero status for %s\n."),
+			displaypath);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = path;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", NULL);
+		argv_array_pushf(&cpr.args, "%s/", displaypath);
+		argv_array_pushl(&cpr.args, "submodule--helper", "foreach", "--recursive",
+				NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		argv_array_pushv(&cpr.args, info->argv);
+
+		if (run_command(&cpr))
+			die(_("run_command returned non-zero status while"
+				"recursing in the nested submodules of %s\n."),
+				displaypath);
+	}
+
+cleanup:
+	free(displaypath);
+}
+
+static int module_foreach(int argc, const char **argv, const char *prefix)
+{
+	struct cb_foreach info = CB_FOREACH_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+
+	struct option module_foreach_options[] = {
+		OPT__QUIET(&info.quiet, N_("Suppress output of entering each submodule command")),
+		OPT_BOOL(0, "recursive", &info.recursive,
+			 N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_foreach_options,
+			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+
+	for_each_listed_submodule(&list, runcommand_in_submodule_cb, &info);
+
+	return 0;
+}
+
 struct init_cb {
 	const char *prefix;
 	unsigned int flags;
@@ -1841,6 +1984,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 331d71c908b..cba585f0754 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -323,44 +323,7 @@ cmd_foreach()
 		shift
 	done
 
-	toplevel=$(pwd)
-
-	# dup stdin so that it can be restored when running the external
-	# command in the subshell (and a recursive call to this function)
-	exec 3<&0
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		if test -e "$sm_path"/.git
-		then
-			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-			say "$(eval_gettext "Entering '\$displaypath'")"
-			name=$(git submodule--helper name "$sm_path")
-			(
-				prefix="$prefix$sm_path/"
-				sanitize_submodule_env
-				cd "$sm_path" &&
-				# we make $path available to scripts ...
-				path=$sm_path &&
-				if test $# -eq 1
-				then
-					eval "$1"
-				else
-					"$@"
-				fi &&
-				if test -n "$recursive"
-				then
-					cmd_foreach "--recursive" "$@"
-				fi
-			) <&3 3<&- ||
-			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 #
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 20%]

* Re: [PATCH 1/2] gitk: show part of submodule log instead of empty pane when listing trees
  2018-05-08 17:07   ` [PATCH 1/2] gitk: show part of submodule log instead of empty pane when listing trees Stefan Beller
@ 2018-05-09  9:01     ` Alex Riesen
  0 siblings, 0 replies; 200+ results
From: Alex Riesen @ 2018-05-09  9:01 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Paul Mackerras, Git Mailing List, Junio C Hamano

Stefan Beller, Tue, May 08, 2018 19:07:29 +0200:
> On Tue, May 8, 2018 at 5:22 AM, Alex Riesen
> <alexander.riesen@cetitec.com> wrote:
> > Currently, the submodules either are not shown at all (if listing a
> > committed tree) or a Tcl error appears (when clicking on a submodule
> > from the index list).
> 
> I do not understand where this appears, yet.
> Where do I have to click to see the effects of this patch?

Er. I meant to say the file list panel (bottom right panel). Sorry,
didn't come out clear. I'll reword the commit message next time.

> > @@ -7659,21 +7660,42 @@ proc showfile {f} {
> >      global ctext_file_names ctext_file_lines
> >      global ctext commentend
> >
> > +    set submodlog "git\\ log\\ --format='%h\\ %aN:\\ %s'\\ -100"
> 
> Do we want to respect the config option diff.submodule here?

Probably not. It is already done when the file list panel is in "Patch" mode.
The "Tree" mode of the panel shows the files in full, so the submodules should
be shown similarly: in a format resembling their full (referenced) contents.

> The -100 is chosen rather arbitrarily. Ideally we'd only walk to the
> previous entry?

Yes, the limit is indeed arbitrary. I'm reluctant of listing full history,
though: it might take too long a while (and does, in my line of work). Maybe
an option in the settings? Or some kind of a more natural limit (for 1 second?
Until the end of panel?)

> > -       if {[catch {set bf [open $f r]} err]} {
> > -           puts "oops, can't read $f: $err"
> > -           return
> > +       if {[file isdirectory $f]} {
> > +           # a submodule
> > +           if {[catch {set bf [open "| sh -c cd\\ \"$f\"&&$submodlog" r]} err]} {
> 
> Can we have $submodlog use the "git -C <path> command"
> option, then we could save the "cd &&" part, which might even
> save us from spawning a shell?

That's because I forgot about that option. Of course, I'll fix this.
Also need a shellquote for the path.

Thanks!
Alex

---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus


^ permalink raw reply	[relevance 6%]

* Re: [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach
  2018-05-09  0:29 ` [PATCHv2 0/4] Rebooting pc/submodule-helper-foreach Stefan Beller
                     ` (3 preceding siblings ...)
  2018-05-09  0:29   ` [PATCH 4/4] submodule: port submodule subcommand 'foreach' from shell to C Stefan Beller
@ 2018-05-09 17:13   ` Jonathan Tan
  4 siblings, 0 replies; 200+ results
From: Jonathan Tan @ 2018-05-09 17:13 UTC (permalink / raw)
  To: Stefan Beller; +Cc: christian.couder, git, gitster, pc44800

On Tue,  8 May 2018 17:29:48 -0700
Stefan Beller <sbeller@google.com> wrote:

> v2:
> * rebased onto origin/master
> * dropped leftover "toplevel" variable from experimentation
> * reworded the commit message for the first patch extensively
> * dropped the third patch
> * see "branch-diff" below.

Patches 1-3 look good to me.

I also can't see anything wrong with patch 4, but I am not an expert at
shell and how we call it from C, so a review from another reviewer would
be appreciated.

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] repository: fix free problem with repo_clear(the_repository)
      [irrelevant] <20180509170409.13666-1-pclouds@gmail.com>
@ 2018-05-09 17:50 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-09 17:50 UTC (permalink / raw)
  To: Nguyễn Thái Ngọc Duy; +Cc: git, Junio C Hamano

On Wed, May 9, 2018 at 10:04 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> the_repository is special. One of the special things about it is that
> it does not allocate a new index_state object like submodules but
> points to the global the_index variable instead. As a global variable,
> the_index cannot be free()'d.

ok. That is the situation we're in.

>
> Add an exception for this in repo_clear(). In the future perhaps we
> would be able to allocate the_repository's index on heap too. Then we
> can remove revert this.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  I was trying to test the new parsed_object_pool_clear() and found this.

So this would go with the latest sb/object-store-alloc ?

My impression was that we never call repo_clear() on
the_repository, which would allow us to special case
the_repository further just as I did in v2 of that series[1] by
having static allocations for certain objects in case of \
the_repository.

[1] https://public-inbox.org/git/20180501213403.14643-14-sbeller@google.com/

We could just deal with all the exceptions, but that makes repo_clear
ugly IMHO.

I would rather special case the_repository even more instead
of having it allocate all its things on the heap. (However we rather
want to profile it and argue with data....)

>  repository.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/repository.c b/repository.c
> index a4848c1bd0..f44733524a 100644
> --- a/repository.c
> +++ b/repository.c
> @@ -238,7 +238,9 @@ void repo_clear(struct repository *repo)
>
>         if (repo->index) {
>                 discard_index(repo->index);
> -               FREE_AND_NULL(repo->index);
> +               if (repo->index != &the_index)
> +                       free(repo->index);
> +               repo->index = NULL;

So after this we have a "dangling" the_index.
It is not really dangling, but it is not part of the_repository any more
and many places still use the_index, it might make up for interesting
bugs?

What is your use case of repo_clear(the_repository)?

Thanks,
Stefan

^ permalink raw reply	[relevance 2%]

* Re: [PATCH v2 0/2] gitk: improve handling of submodules in the file list panel
      [irrelevant] ` <cover.1525868167.git.raa.lkml@gmail.com>
@ 2018-05-09 18:52   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-09 18:52 UTC (permalink / raw)
  To: Alex Riesen
  Cc: Paul Mackerras, Git Mailing List, Junio C Hamano, Alex Riesen,
	Bert Wesarg

On Wed, May 9, 2018 at 5:35 AM, Alex Riesen
<alexander.riesen@cetitec.com> wrote:
> From: Alex Riesen <raa.lkml@gmail.com>
>
> Currently, the submodule entries in the file list panel are mostly ignored.
> This series attempts to improve the situation by showing part of submodule
> history when focusing it in the file list panel and by adding a menu element
> to start gitk in the submodule (similar to git gui).
>
> This iteration does not address the behaviour of file list panel in tree mode
> when gitk is started from a subdirectory (gitk strictly limits the file
> listing to the files in that repository, without a way out).
> I would like to hear some more opinions regarding changing its behaviour to
> always list the full tree.
>
> Alex Riesen (2):
>   gitk: show part of submodule log instead of empty pane when listing
>     trees
>   gitk: add an option to run gitk on an item in the file list

both patches look ok, to my untrained eye.

^ permalink raw reply	[relevance 5%]

* [PATCH v4 01/13] repository: introduce parsed objects field
      [irrelevant]     ` <20180510004024.93974-1-sbeller@google.com>
@ 2018-05-10  0:40       ` " Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-10  0:40 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jamill, jonathantanmy, pclouds

Convert the existing global cache for parsed objects (obj_hash) into
repository-specific parsed object caches. Existing code that uses
obj_hash are modified to use the parsed object cache of
the_repository; future patches will use the parsed object caches of
other repositories.

Another future use case for a pool of objects is ease of memory management
in revision walking: If we can free the rev-list related memory early in
pack-objects (e.g. part of repack operation) then it could lower memory
pressure significantly when running on large repos. While this has been
discussed on the mailing list lately, this series doesn't implement this.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 object.c     | 63 +++++++++++++++++++++++++++++++++-------------------
 object.h     |  8 +++++++
 repository.c |  7 ++++++
 repository.h |  9 ++++++++
 4 files changed, 64 insertions(+), 23 deletions(-)

diff --git a/object.c b/object.c
index 5044d08e96c..f7c624a7ba6 100644
--- a/object.c
+++ b/object.c
@@ -8,17 +8,14 @@
 #include "object-store.h"
 #include "packfile.h"
 
-static struct object **obj_hash;
-static int nr_objs, obj_hash_size;
-
 unsigned int get_max_object_index(void)
 {
-	return obj_hash_size;
+	return the_repository->parsed_objects->obj_hash_size;
 }
 
 struct object *get_indexed_object(unsigned int idx)
 {
-	return obj_hash[idx];
+	return the_repository->parsed_objects->obj_hash[idx];
 }
 
 static const char *object_type_strings[] = {
@@ -90,15 +87,16 @@ struct object *lookup_object(const unsigned char *sha1)
 	unsigned int i, first;
 	struct object *obj;
 
-	if (!obj_hash)
+	if (!the_repository->parsed_objects->obj_hash)
 		return NULL;
 
-	first = i = hash_obj(sha1, obj_hash_size);
-	while ((obj = obj_hash[i]) != NULL) {
+	first = i = hash_obj(sha1,
+			     the_repository->parsed_objects->obj_hash_size);
+	while ((obj = the_repository->parsed_objects->obj_hash[i]) != NULL) {
 		if (!hashcmp(sha1, obj->oid.hash))
 			break;
 		i++;
-		if (i == obj_hash_size)
+		if (i == the_repository->parsed_objects->obj_hash_size)
 			i = 0;
 	}
 	if (obj && i != first) {
@@ -107,7 +105,8 @@ struct object *lookup_object(const unsigned char *sha1)
 		 * that we do not need to walk the hash table the next
 		 * time we look for it.
 		 */
-		SWAP(obj_hash[i], obj_hash[first]);
+		SWAP(the_repository->parsed_objects->obj_hash[i],
+		     the_repository->parsed_objects->obj_hash[first]);
 	}
 	return obj;
 }
@@ -124,19 +123,19 @@ static void grow_object_hash(void)
 	 * Note that this size must always be power-of-2 to match hash_obj
 	 * above.
 	 */
-	int new_hash_size = obj_hash_size < 32 ? 32 : 2 * obj_hash_size;
+	int new_hash_size = the_repository->parsed_objects->obj_hash_size < 32 ? 32 : 2 * the_repository->parsed_objects->obj_hash_size;
 	struct object **new_hash;
 
 	new_hash = xcalloc(new_hash_size, sizeof(struct object *));
-	for (i = 0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i = 0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (!obj)
 			continue;
 		insert_obj_hash(obj, new_hash, new_hash_size);
 	}
-	free(obj_hash);
-	obj_hash = new_hash;
-	obj_hash_size = new_hash_size;
+	free(the_repository->parsed_objects->obj_hash);
+	the_repository->parsed_objects->obj_hash = new_hash;
+	the_repository->parsed_objects->obj_hash_size = new_hash_size;
 }
 
 void *create_object(const unsigned char *sha1, void *o)
@@ -147,11 +146,12 @@ void *create_object(const unsigned char *sha1, void *o)
 	obj->flags = 0;
 	hashcpy(obj->oid.hash, sha1);
 
-	if (obj_hash_size - 1 <= nr_objs * 2)
+	if (the_repository->parsed_objects->obj_hash_size - 1 <= the_repository->parsed_objects->nr_objs * 2)
 		grow_object_hash();
 
-	insert_obj_hash(obj, obj_hash, obj_hash_size);
-	nr_objs++;
+	insert_obj_hash(obj, the_repository->parsed_objects->obj_hash,
+			the_repository->parsed_objects->obj_hash_size);
+	the_repository->parsed_objects->nr_objs++;
 	return obj;
 }
 
@@ -431,8 +431,8 @@ void clear_object_flags(unsigned flags)
 {
 	int i;
 
-	for (i=0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i=0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (obj)
 			obj->flags &= ~flags;
 	}
@@ -442,13 +442,20 @@ void clear_commit_marks_all(unsigned int flags)
 {
 	int i;
 
-	for (i = 0; i < obj_hash_size; i++) {
-		struct object *obj = obj_hash[i];
+	for (i = 0; i < the_repository->parsed_objects->obj_hash_size; i++) {
+		struct object *obj = the_repository->parsed_objects->obj_hash[i];
 		if (obj && obj->type == OBJ_COMMIT)
 			obj->flags &= ~flags;
 	}
 }
 
+struct parsed_object_pool *parsed_object_pool_new(void)
+{
+	struct parsed_object_pool *o = xmalloc(sizeof(*o));
+	memset(o, 0, sizeof(*o));
+	return o;
+}
+
 struct raw_object_store *raw_object_store_new(void)
 {
 	struct raw_object_store *o = xmalloc(sizeof(*o));
@@ -488,3 +495,13 @@ void raw_object_store_clear(struct raw_object_store *o)
 	close_all_packs(o);
 	o->packed_git = NULL;
 }
+
+void parsed_object_pool_clear(struct parsed_object_pool *o)
+{
+	/*
+	 * TOOD free objects in o->obj_hash.
+	 *
+	 * As objects are allocated in slabs (see alloc.c), we do
+	 * not need to free each object, but each slab instead.
+	 */
+}
diff --git a/object.h b/object.h
index f13f85b2a94..cecda7da370 100644
--- a/object.h
+++ b/object.h
@@ -1,6 +1,14 @@
 #ifndef OBJECT_H
 #define OBJECT_H
 
+struct parsed_object_pool {
+	struct object **obj_hash;
+	int nr_objs, obj_hash_size;
+};
+
+struct parsed_object_pool *parsed_object_pool_new(void);
+void parsed_object_pool_clear(struct parsed_object_pool *o);
+
 struct object_list {
 	struct object *item;
 	struct object_list *next;
diff --git a/repository.c b/repository.c
index a4848c1bd05..c23404677eb 100644
--- a/repository.c
+++ b/repository.c
@@ -2,6 +2,7 @@
 #include "repository.h"
 #include "object-store.h"
 #include "config.h"
+#include "object.h"
 #include "submodule-config.h"
 
 /* The main repository */
@@ -14,6 +15,8 @@ void initialize_the_repository(void)
 
 	the_repo.index = &the_index;
 	the_repo.objects = raw_object_store_new();
+	the_repo.parsed_objects = parsed_object_pool_new();
+
 	repo_set_hash_algo(&the_repo, GIT_HASH_SHA1);
 }
 
@@ -143,6 +146,7 @@ static int repo_init(struct repository *repo,
 	memset(repo, 0, sizeof(*repo));
 
 	repo->objects = raw_object_store_new();
+	repo->parsed_objects = parsed_object_pool_new();
 
 	if (repo_init_gitdir(repo, gitdir))
 		goto error;
@@ -226,6 +230,9 @@ void repo_clear(struct repository *repo)
 	raw_object_store_clear(repo->objects);
 	FREE_AND_NULL(repo->objects);
 
+	parsed_object_pool_clear(repo->parsed_objects);
+	FREE_AND_NULL(repo->parsed_objects);
+
 	if (repo->config) {
 		git_configset_clear(repo->config);
 		FREE_AND_NULL(repo->config);
diff --git a/repository.h b/repository.h
index e6e00f541bd..6d199819905 100644
--- a/repository.h
+++ b/repository.h
@@ -26,6 +26,15 @@ struct repository {
 	 */
 	struct raw_object_store *objects;
 
+	/*
+	 * All objects in this repository that have been parsed. This structure
+	 * owns all objects it references, so users of "struct object *"
+	 * generally do not need to free them; instead, when a repository is no
+	 * longer used, call parsed_object_pool_clear() on this structure, which
+	 * is called by the repositories repo_clear on its desconstruction.
+	 */
+	struct parsed_object_pool *parsed_objects;
+
 	/* The store in which the refs are held. */
 	struct ref_store *refs;
 
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 4%]

* Re: [PATCH 4/4] submodule: port submodule subcommand 'foreach' from shell to C
  2018-05-09  0:29   ` [PATCH 4/4] submodule: port submodule subcommand 'foreach' from shell to C Stefan Beller
@ 2018-05-10  6:37     ` Junio C Hamano
  2018-05-10 21:25       ` [PATCH] " Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2018-05-10  6:37 UTC (permalink / raw)
  To: Stefan Beller; +Cc: christian.couder, git, jonathantanmy, pc44800

Stefan Beller <sbeller@google.com> writes:

> +static void runcommand_in_submodule_cb(const struct cache_entry *list_item,
> +				       void *cb_data)
> +{
> +	struct cb_foreach *info = cb_data;
> +	const char *path = list_item->name;
> +	const struct object_id *ce_oid = &list_item->oid;
> +
> +	const struct submodule *sub;
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +	char *displaypath;
> +
> +	displaypath = get_submodule_displaypath(path, info->prefix);
> +
> +	sub = submodule_from_path(the_repository, &null_oid, path);
> +
> +	if (!sub)
> +		die(_("No url found for submodule path '%s' in .gitmodules"),
> +			displaypath);
> +
> +	if (!is_submodule_populated_gently(path, NULL))
> +		goto cleanup;
> +
> +	prepare_submodule_repo_env(&cp.env_array);
> +
> +	/*
> +	* For the purpose of executing <command> in the submodule,
> +	* separate shell is used for the purpose of running the
> +	* child process.
> +	*/

Micronit: this multi-line comment is indented in a funny way.

> +	cp.use_shell = 1;
> +	cp.dir = path;
> +
> +	/*
> +	* NEEDSWORK: the command currently has access to the variables $name,
> +	* $sm_path, $displaypath, $sha1 and $toplevel only when the command
> +	* contains a single argument. This is done for maintaining a faithful
> +	* translation from shell script.
> +	*/

Same micronit.

The scripted version does 'eval "$1"', so $1 could be something like 

	for-each 'echo "$name:$sm_path:$displaypath:$sha1:$toplevel"'

and it can see any variable, not just these 5 (i.e. we could have
fed e.g. $wt_prefix and $mode to the above 'echo' and with the
scripted version the script would have learned their values; with
this version it no longer does, but only these 5 are part of the
documented API, so we choose not to consider it a regression).

> +	if (info->argc == 1) {
> +		char *toplevel = xgetcwd();
> +		struct strbuf sb = STRBUF_INIT;
> +
> +		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
> +		argv_array_pushf(&cp.env_array, "sm_path=%s", path);
> +		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
> +		argv_array_pushf(&cp.env_array, "sha1=%s",
> +				oid_to_hex(ce_oid));
> +		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
> +
> +		/*
> +		* Since the path variable was accessible from the script
> +		* before porting, it is also made available after porting.
> +		* The environment variable "PATH" has a very special purpose
> +		* on windows. And since environment variables are
> +		* case-insensitive in windows, it interferes with the
> +		* existing PATH variable. Hence, to avoid that, we expose
> +		* path via the args argv_array and not via env_array.
> +		*/
> +		sq_quote_buf(&sb, path);
> +		argv_array_pushf(&cp.args, "path=%s; %s",
> +				 sb.buf, info->argv[0]);

OK, so we do the equivalent of

	name=... sm_path=... displaypath=... sha1=... toplevel=... \
	sh -c 'path=...; echo "$name:$sm_path:..."'

when doing

	for-each 'echo "$name:$sm_path:..."'

with parts denoted with ... correctly quoted as necessary.  I guess
it would be the best we could do.

I myself do not know if it is true that bash ported to Windows won't
get confused with the above "we use path (all lowercase) only as a
pure shell variable without exporting it ourselves"; I'd trust those
who are more familiar with the platform to raise objections and
suggest a better alternative if it is not the case.  

Thanks for the (malformatted;-) leading comment to highlight why the
'path' variable alone is treated differently from the others.

> +		strbuf_release(&sb);
> +		free(toplevel);
> +	} else {
> +		argv_array_pushv(&cp.args, info->argv);
> +	}
> +
> +	if (!info->quiet)
> +		printf(_("Entering '%s'\n"), displaypath);
> +
> +	if (info->argv[0] && run_command(&cp))
> +		die(_("run_command returned non-zero status for %s\n."),
> +			displaypath);
> +
> +	if (info->recursive) {
> +		struct child_process cpr = CHILD_PROCESS_INIT;
> +
> +		cpr.git_cmd = 1;
> +		cpr.dir = path;
> +		prepare_submodule_repo_env(&cpr.env_array);
> +
> +		argv_array_pushl(&cpr.args, "--super-prefix", NULL);
> +		argv_array_pushf(&cpr.args, "%s/", displaypath);
> +		argv_array_pushl(&cpr.args, "submodule--helper", "foreach", "--recursive",
> +				NULL);
> +
> +		if (info->quiet)
> +			argv_array_push(&cpr.args, "--quiet");
> +
> +		argv_array_pushv(&cpr.args, info->argv);
> +
> +		if (run_command(&cpr))
> +			die(_("run_command returned non-zero status while"
> +				"recursing in the nested submodules of %s\n."),
> +				displaypath);
> +	}
> +
> +cleanup:
> +	free(displaypath);
> +}


^ permalink raw reply	[relevance 5%]

* Re: [PATCH 1/1] Warn about fast-forwarding of submodules during merge
      [irrelevant] ` <20180510182657.65095-2-leif.middelschulte@gmail.com>
@ 2018-05-10 18:49   ` Stefan Beller
  2018-05-10 20:30     ` Leif Middelschulte
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-10 18:49 UTC (permalink / raw)
  To: Leif Middelschulte; +Cc: git, Junio C Hamano, sandals

On Thu, May 10, 2018 at 11:26 AM, Leif Middelschulte
<leif.middelschulte@gmail.com> wrote:
> From: Leif Middelschulte <Leif.Middelschulte@gmail.com>

Hi Leif!

thanks for following up with a patch!

> Warn the user about an automatically fast-forwarded submodule. The silent merge
> behavior was introduced by commit 68d03e4a6e44 ("Implement automatic fast-forward
> merge for submodules", 2010-07-07)).
>
> Signed-off-by: Leif Middelschulte <Leif.Middelschulte@gmail.com>
> ---
>  submodule.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/submodule.c b/submodule.c
> index 74d35b257..0198a72e6 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1817,10 +1817,12 @@ int merge_submodule(struct object_id *result, const char *path,
>         /* Case #1: a is contained in b or vice versa */
>         if (in_merge_bases(commit_a, commit_b)) {
>                 oidcpy(result, b);
> +               warning("Fast-forwarding submodule %s", path);
>                 return 1;
>         }
>         if (in_merge_bases(commit_b, commit_a)) {
>                 oidcpy(result, a);
> +               warning("Fast-forwarding submodule %s", path);
>                 return 1;
>         }

The code looks correct, however I think we can improve it.
(Originally I was just wondering if stderr is the right output,
which lead me to the thoughts below:)

Looking through the code of merge-recursive.c,
all the other merge outputs are done via 'output()'
that is able to buffer up the output as well as handles
the output for different verbosity settings.

So I would think we should make the output() function available
outside of merge-recursive.c. (and rename it to a be more concise
and descriptive in the global namespace) and make use of it.

Funnily we already have MERGE_WARNING in submodule.c
which outputs information for all the other cases. I would think
we ought to convert those to the output(), too.

Thanks,
Stefan

^ permalink raw reply	[relevance 6%]

* Re: [PATCH 1/1] Warn about fast-forwarding of submodules during merge
  2018-05-10 18:49   ` [PATCH 1/1] Warn about fast-forwarding of submodules during merge Stefan Beller
@ 2018-05-10 20:30     ` Leif Middelschulte
  2018-05-10 21:19       ` [PATCH 0/2] Submodule merging: i18n, verbosity Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Leif Middelschulte @ 2018-05-10 20:30 UTC (permalink / raw)
  To: Stefan Beller; +Cc: sandals, Junio C Hamano, git

Hi Stefan,


Am 10. Mai 2018 um 20:49:39, Stefan Beller
(sbeller@google.com(mailto:sbeller@google.com)) schrieb:

> On Thu, May 10, 2018 at 11:26 AM, Leif Middelschulte
> wrote:
> > From: Leif Middelschulte
>
> Hi Leif!
>
> thanks for following up with a patch!
sure, thanks for the quick review.
>
> > Warn the user about an automatically fast-forwarded submodule. The silent merge
> > behavior was introduced by commit 68d03e4a6e44 ("Implement automatic fast-forward
> > merge for submodules", 2010-07-07)).
> >
> > Signed-off-by: Leif Middelschulte
> > ---
> > submodule.c | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/submodule.c b/submodule.c
> > index 74d35b257..0198a72e6 100644
> > --- a/submodule.c
> > +++ b/submodule.c
> > @@ -1817,10 +1817,12 @@ int merge_submodule(struct object_id *result, const char *path,
> > /* Case #1: a is contained in b or vice versa */
> > if (in_merge_bases(commit_a, commit_b)) {
> > oidcpy(result, b);
> > + warning("Fast-forwarding submodule %s", path);
> > return 1;
> > }
> > if (in_merge_bases(commit_b, commit_a)) {
> > oidcpy(result, a);
> > + warning("Fast-forwarding submodule %s", path);
> > return 1;
> > }
>
> The code looks correct, however I think we can improve it.
> (Originally I was just wondering if stderr is the right output,
> which lead me to the thoughts below:)
I’ve had the same thoughts about stderr. However, I thought that using a
log function named `warning` to warn the user would be the right choice.
If anything, I thought, the warning function might need refactoring.

> Looking through the code of merge-recursive.c,
> all the other merge outputs are done via 'output()'
> that is able to buffer up the output as well as handles
> the output for different verbosity settings.
>
> So I would think we should make the output() function available
> outside of merge-recursive.c. (and rename it to a be more concise
> and descriptive in the global namespace) and make use of it.
Sure, let me know what to use instead and I’ll update and resubmit the patch.

>
> Funnily we already have MERGE_WARNING in submodule.c
> which outputs information for all the other cases. I would think
> we ought to convert those to the output(), too.
Sure, but `MERGE_WARNING` prefixes all the messages with "Failed to
merge submodule“.
>
> Thanks,
> Stefan

Thank you,
Leif

^ permalink raw reply	[relevance 7%]

* [PATCH 0/2] Submodule merging: i18n, verbosity
  2018-05-10 20:30     ` Leif Middelschulte
@ 2018-05-10 21:19       ` Stefan Beller
  2018-05-10 21:19         ` [PATCH 1/2] submodule.c: move submodule merging to merge-recursive.c Stefan Beller
                           ` (2 more replies)
  0 siblings, 3 replies; 200+ results
From: Stefan Beller @ 2018-05-10 21:19 UTC (permalink / raw)
  To: leif.middelschulte; +Cc: git, gitster, sandals, sbeller

Leif wrote:
> Sure, let me know what to use instead and I’ll update and resubmit the patch.
> Sure, but `MERGE_WARNING` prefixes all the messages with "Failed to
> merge submodule“.

I thought about replying and coming up with good reasons, but I wrote some
patches instead.

They can also be found at https://github.com/stefanbeller/git/tree/submodule_i18n_verbose

I think these would be a good foundation for your patch as well, as you can use the
output() function for the desired cases.

Feel free to take these patches as part of your series or adapt
(or be inspired by) as needed.

Thanks,
Stefan


Stefan Beller (2):
  submodule.c: move submodule merging to merge-recursive.c
  merge-recursive: i18n submodule merge output and respect verbosity

 merge-recursive.c | 169 +++++++++++++++++++++++++++++++++++++++++++++-
 submodule.c       | 168 +--------------------------------------------
 submodule.h       |   6 +-
 3 files changed, 170 insertions(+), 173 deletions(-)

-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 11%]

* [PATCH 1/2] submodule.c: move submodule merging to merge-recursive.c
  2018-05-10 21:19       ` [PATCH 0/2] Submodule merging: i18n, verbosity Stefan Beller
@ 2018-05-10 21:19         ` Stefan Beller
  2018-05-10 21:19         ` [PATCH 2/2] merge-recursive: i18n submodule merge output and respect verbosity Stefan Beller
  2018-05-11  0:04         ` [PATCH 0/2] Submodule merging: i18n, verbosity Elijah Newren
  2 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-10 21:19 UTC (permalink / raw)
  To: leif.middelschulte; +Cc: git, gitster, sandals, sbeller

In a later patch we want to improve submodule merging by using the output()
function in merge-recursive.c for submodule merges to deliver a consistent
UI to users.

To do so we could either make the output() function globally available
so we can use it in submodule.c#merge_submodule(), or we could integrate
the submodule merging into the merging code. Choose the later as we
generally want to move submodules closer into the core.

Therefore we move any function related to merging submodules
(merge_submodule(), find_first_merges() and print_commit) to
merge-recursive.c.  We'll keep add_submodule_odb() in submodule.c as it
is used by other submodule functions. While at it, add a TODO note that
we do not really like the function add_submodule_odb().

This commit is best viewed with --color-moved.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 merge-recursive.c | 166 +++++++++++++++++++++++++++++++++++++++++++++
 submodule.c       | 168 +---------------------------------------------
 submodule.h       |   6 +-
 3 files changed, 170 insertions(+), 170 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0c0d48624da..700ba15bf88 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -23,6 +23,7 @@
 #include "merge-recursive.h"
 #include "dir.h"
 #include "submodule.h"
+#include "revision.h"
 
 struct path_hashmap_entry {
 	struct hashmap_entry e;
@@ -977,6 +978,171 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
+static int find_first_merges(struct object_array *result, const char *path,
+		struct commit *a, struct commit *b)
+{
+	int i, j;
+	struct object_array merges = OBJECT_ARRAY_INIT;
+	struct commit *commit;
+	int contains_another;
+
+	char merged_revision[42];
+	const char *rev_args[] = { "rev-list", "--merges", "--ancestry-path",
+				   "--all", merged_revision, NULL };
+	struct rev_info revs;
+	struct setup_revision_opt rev_opts;
+
+	memset(result, 0, sizeof(struct object_array));
+	memset(&rev_opts, 0, sizeof(rev_opts));
+
+	/* get all revisions that merge commit a */
+	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
+			oid_to_hex(&a->object.oid));
+	init_revisions(&revs, NULL);
+	rev_opts.submodule = path;
+	/* FIXME: can't handle linked worktrees in submodules yet */
+	revs.single_worktree = path != NULL;
+	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
+
+	/* save all revisions from the above list that contain b */
+	if (prepare_revision_walk(&revs))
+		die("revision walk setup failed");
+	while ((commit = get_revision(&revs)) != NULL) {
+		struct object *o = &(commit->object);
+		if (in_merge_bases(b, commit))
+			add_object_array(o, NULL, &merges);
+	}
+	reset_revision_walk();
+
+	/* Now we've got all merges that contain a and b. Prune all
+	 * merges that contain another found merge and save them in
+	 * result.
+	 */
+	for (i = 0; i < merges.nr; i++) {
+		struct commit *m1 = (struct commit *) merges.objects[i].item;
+
+		contains_another = 0;
+		for (j = 0; j < merges.nr; j++) {
+			struct commit *m2 = (struct commit *) merges.objects[j].item;
+			if (i != j && in_merge_bases(m2, m1)) {
+				contains_another = 1;
+				break;
+			}
+		}
+
+		if (!contains_another)
+			add_object_array(merges.objects[i].item, NULL, result);
+	}
+
+	object_array_clear(&merges);
+	return result->nr;
+}
+
+static void print_commit(struct commit *commit)
+{
+	struct strbuf sb = STRBUF_INIT;
+	struct pretty_print_context ctx = {0};
+	ctx.date_mode.type = DATE_NORMAL;
+	format_commit_message(commit, " %h: %m %s", &sb, &ctx);
+	fprintf(stderr, "%s\n", sb.buf);
+	strbuf_release(&sb);
+}
+
+#define MERGE_WARNING(path, msg) \
+	warning("Failed to merge submodule %s (%s)", path, msg);
+
+static int merge_submodule(struct object_id *result, const char *path,
+			   const struct object_id *base, const struct object_id *a,
+			   const struct object_id *b, int search)
+{
+	struct commit *commit_base, *commit_a, *commit_b;
+	int parent_count;
+	struct object_array merges;
+
+	int i;
+
+	/* store a in result in case we fail */
+	oidcpy(result, a);
+
+	/* we can not handle deletion conflicts */
+	if (is_null_oid(base))
+		return 0;
+	if (is_null_oid(a))
+		return 0;
+	if (is_null_oid(b))
+		return 0;
+
+	if (add_submodule_odb(path)) {
+		MERGE_WARNING(path, "not checked out");
+		return 0;
+	}
+
+	if (!(commit_base = lookup_commit_reference(base)) ||
+	    !(commit_a = lookup_commit_reference(a)) ||
+	    !(commit_b = lookup_commit_reference(b))) {
+		MERGE_WARNING(path, "commits not present");
+		return 0;
+	}
+
+	/* check whether both changes are forward */
+	if (!in_merge_bases(commit_base, commit_a) ||
+	    !in_merge_bases(commit_base, commit_b)) {
+		MERGE_WARNING(path, "commits don't follow merge-base");
+		return 0;
+	}
+
+	/* Case #1: a is contained in b or vice versa */
+	if (in_merge_bases(commit_a, commit_b)) {
+		oidcpy(result, b);
+		return 1;
+	}
+	if (in_merge_bases(commit_b, commit_a)) {
+		oidcpy(result, a);
+		return 1;
+	}
+
+	/*
+	 * Case #2: There are one or more merges that contain a and b in
+	 * the submodule. If there is only one, then present it as a
+	 * suggestion to the user, but leave it marked unmerged so the
+	 * user needs to confirm the resolution.
+	 */
+
+	/* Skip the search if makes no sense to the calling context.  */
+	if (!search)
+		return 0;
+
+	/* find commit which merges them */
+	parent_count = find_first_merges(&merges, path, commit_a, commit_b);
+	switch (parent_count) {
+	case 0:
+		MERGE_WARNING(path, "merge following commits not found");
+		break;
+
+	case 1:
+		MERGE_WARNING(path, "not fast-forward");
+		fprintf(stderr, "Found a possible merge resolution "
+				"for the submodule:\n");
+		print_commit((struct commit *) merges.objects[0].item);
+		fprintf(stderr,
+			"If this is correct simply add it to the index "
+			"for example\n"
+			"by using:\n\n"
+			"  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
+			"which will accept this suggestion.\n",
+			oid_to_hex(&merges.objects[0].item->oid), path);
+		break;
+
+	default:
+		MERGE_WARNING(path, "multiple merges found");
+		for (i = 0; i < merges.nr; i++)
+			print_commit((struct commit *) merges.objects[i].item);
+	}
+
+	object_array_clear(&merges);
+	return 0;
+}
+
 static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
diff --git a/submodule.c b/submodule.c
index 74d35b25779..654089b3647 100644
--- a/submodule.c
+++ b/submodule.c
@@ -153,7 +153,8 @@ void stage_updated_gitmodules(struct index_state *istate)
 		die(_("staging updated .gitmodules failed"));
 }
 
-static int add_submodule_odb(const char *path)
+/* TODO: remove this function, use repo_submodule_init instead. */
+int add_submodule_odb(const char *path)
 {
 	struct strbuf objects_directory = STRBUF_INIT;
 	int ret = 0;
@@ -1701,171 +1702,6 @@ int submodule_move_head(const char *path,
 	return ret;
 }
 
-static int find_first_merges(struct object_array *result, const char *path,
-		struct commit *a, struct commit *b)
-{
-	int i, j;
-	struct object_array merges = OBJECT_ARRAY_INIT;
-	struct commit *commit;
-	int contains_another;
-
-	char merged_revision[42];
-	const char *rev_args[] = { "rev-list", "--merges", "--ancestry-path",
-				   "--all", merged_revision, NULL };
-	struct rev_info revs;
-	struct setup_revision_opt rev_opts;
-
-	memset(result, 0, sizeof(struct object_array));
-	memset(&rev_opts, 0, sizeof(rev_opts));
-
-	/* get all revisions that merge commit a */
-	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
-			oid_to_hex(&a->object.oid));
-	init_revisions(&revs, NULL);
-	rev_opts.submodule = path;
-	/* FIXME: can't handle linked worktrees in submodules yet */
-	revs.single_worktree = path != NULL;
-	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
-
-	/* save all revisions from the above list that contain b */
-	if (prepare_revision_walk(&revs))
-		die("revision walk setup failed");
-	while ((commit = get_revision(&revs)) != NULL) {
-		struct object *o = &(commit->object);
-		if (in_merge_bases(b, commit))
-			add_object_array(o, NULL, &merges);
-	}
-	reset_revision_walk();
-
-	/* Now we've got all merges that contain a and b. Prune all
-	 * merges that contain another found merge and save them in
-	 * result.
-	 */
-	for (i = 0; i < merges.nr; i++) {
-		struct commit *m1 = (struct commit *) merges.objects[i].item;
-
-		contains_another = 0;
-		for (j = 0; j < merges.nr; j++) {
-			struct commit *m2 = (struct commit *) merges.objects[j].item;
-			if (i != j && in_merge_bases(m2, m1)) {
-				contains_another = 1;
-				break;
-			}
-		}
-
-		if (!contains_another)
-			add_object_array(merges.objects[i].item, NULL, result);
-	}
-
-	object_array_clear(&merges);
-	return result->nr;
-}
-
-static void print_commit(struct commit *commit)
-{
-	struct strbuf sb = STRBUF_INIT;
-	struct pretty_print_context ctx = {0};
-	ctx.date_mode.type = DATE_NORMAL;
-	format_commit_message(commit, " %h: %m %s", &sb, &ctx);
-	fprintf(stderr, "%s\n", sb.buf);
-	strbuf_release(&sb);
-}
-
-#define MERGE_WARNING(path, msg) \
-	warning("Failed to merge submodule %s (%s)", path, msg);
-
-int merge_submodule(struct object_id *result, const char *path,
-		    const struct object_id *base, const struct object_id *a,
-		    const struct object_id *b, int search)
-{
-	struct commit *commit_base, *commit_a, *commit_b;
-	int parent_count;
-	struct object_array merges;
-
-	int i;
-
-	/* store a in result in case we fail */
-	oidcpy(result, a);
-
-	/* we can not handle deletion conflicts */
-	if (is_null_oid(base))
-		return 0;
-	if (is_null_oid(a))
-		return 0;
-	if (is_null_oid(b))
-		return 0;
-
-	if (add_submodule_odb(path)) {
-		MERGE_WARNING(path, "not checked out");
-		return 0;
-	}
-
-	if (!(commit_base = lookup_commit_reference(base)) ||
-	    !(commit_a = lookup_commit_reference(a)) ||
-	    !(commit_b = lookup_commit_reference(b))) {
-		MERGE_WARNING(path, "commits not present");
-		return 0;
-	}
-
-	/* check whether both changes are forward */
-	if (!in_merge_bases(commit_base, commit_a) ||
-	    !in_merge_bases(commit_base, commit_b)) {
-		MERGE_WARNING(path, "commits don't follow merge-base");
-		return 0;
-	}
-
-	/* Case #1: a is contained in b or vice versa */
-	if (in_merge_bases(commit_a, commit_b)) {
-		oidcpy(result, b);
-		return 1;
-	}
-	if (in_merge_bases(commit_b, commit_a)) {
-		oidcpy(result, a);
-		return 1;
-	}
-
-	/*
-	 * Case #2: There are one or more merges that contain a and b in
-	 * the submodule. If there is only one, then present it as a
-	 * suggestion to the user, but leave it marked unmerged so the
-	 * user needs to confirm the resolution.
-	 */
-
-	/* Skip the search if makes no sense to the calling context.  */
-	if (!search)
-		return 0;
-
-	/* find commit which merges them */
-	parent_count = find_first_merges(&merges, path, commit_a, commit_b);
-	switch (parent_count) {
-	case 0:
-		MERGE_WARNING(path, "merge following commits not found");
-		break;
-
-	case 1:
-		MERGE_WARNING(path, "not fast-forward");
-		fprintf(stderr, "Found a possible merge resolution "
-				"for the submodule:\n");
-		print_commit((struct commit *) merges.objects[0].item);
-		fprintf(stderr,
-			"If this is correct simply add it to the index "
-			"for example\n"
-			"by using:\n\n"
-			"  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
-			"which will accept this suggestion.\n",
-			oid_to_hex(&merges.objects[0].item->oid), path);
-		break;
-
-	default:
-		MERGE_WARNING(path, "multiple merges found");
-		for (i = 0; i < merges.nr; i++)
-			print_commit((struct commit *) merges.objects[i].item);
-	}
-
-	object_array_clear(&merges);
-	return 0;
-}
-
 /*
  * Embeds a single submodules git directory into the superprojects git dir,
  * non recursively.
diff --git a/submodule.h b/submodule.h
index e5526f6aaab..b96689ac0db 100644
--- a/submodule.h
+++ b/submodule.h
@@ -89,10 +89,8 @@ extern int submodule_uses_gitfile(const char *path);
 #define SUBMODULE_REMOVAL_IGNORE_UNTRACKED (1<<1)
 #define SUBMODULE_REMOVAL_IGNORE_IGNORED_UNTRACKED (1<<2)
 extern int bad_to_remove_submodule(const char *path, unsigned flags);
-extern int merge_submodule(struct object_id *result, const char *path,
-			   const struct object_id *base,
-			   const struct object_id *a,
-			   const struct object_id *b, int search);
+
+int add_submodule_odb(const char *path);
 
 /* Checks if there are submodule changes in a..b. */
 extern int submodule_touches_in_range(struct object_id *a,
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 18%]

* [PATCH 2/2] merge-recursive: i18n submodule merge output and respect verbosity
  2018-05-10 21:19       ` [PATCH 0/2] Submodule merging: i18n, verbosity Stefan Beller
  2018-05-10 21:19         ` [PATCH 1/2] submodule.c: move submodule merging to merge-recursive.c Stefan Beller
@ 2018-05-10 21:19         ` Stefan Beller
  2018-05-15  1:25           ` Elijah Newren
  2018-05-11  0:04         ` [PATCH 0/2] Submodule merging: i18n, verbosity Elijah Newren
  2 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-10 21:19 UTC (permalink / raw)
  To: leif.middelschulte; +Cc: git, gitster, sandals, sbeller

The submodule merge code now uses the output() function that is used by
all the rest of the merge-recursive-code. This allows for respecting
internationalisation as well as the verbosity setting.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 merge-recursive.c | 33 +++++++++++++++------------------
 1 file changed, 15 insertions(+), 18 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 700ba15bf88..a4b91d17f87 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1048,18 +1048,17 @@ static void print_commit(struct commit *commit)
 	strbuf_release(&sb);
 }
 
-#define MERGE_WARNING(path, msg) \
-	warning("Failed to merge submodule %s (%s)", path, msg);
-
-static int merge_submodule(struct object_id *result, const char *path,
+static int merge_submodule(struct merge_options *o,
+			   struct object_id *result, const char *path,
 			   const struct object_id *base, const struct object_id *a,
-			   const struct object_id *b, int search)
+			   const struct object_id *b)
 {
 	struct commit *commit_base, *commit_a, *commit_b;
 	int parent_count;
 	struct object_array merges;
 
 	int i;
+	int search = !o->call_depth;
 
 	/* store a in result in case we fail */
 	oidcpy(result, a);
@@ -1073,21 +1072,21 @@ static int merge_submodule(struct object_id *result, const char *path,
 		return 0;
 
 	if (add_submodule_odb(path)) {
-		MERGE_WARNING(path, "not checked out");
+		output(o, 1, _("Failed to merge submodule %s (not checked out)"), path);
 		return 0;
 	}
 
 	if (!(commit_base = lookup_commit_reference(base)) ||
 	    !(commit_a = lookup_commit_reference(a)) ||
 	    !(commit_b = lookup_commit_reference(b))) {
-		MERGE_WARNING(path, "commits not present");
+		output(o, 1, _("Failed to merge submodule %s (commits not present)"), path);
 		return 0;
 	}
 
 	/* check whether both changes are forward */
 	if (!in_merge_bases(commit_base, commit_a) ||
 	    !in_merge_bases(commit_base, commit_b)) {
-		MERGE_WARNING(path, "commits don't follow merge-base");
+		output(o, 1, _("Failed to merge submodule %s (commits don't follow merge-base)"), path);
 		return 0;
 	}
 
@@ -1116,25 +1115,24 @@ static int merge_submodule(struct object_id *result, const char *path,
 	parent_count = find_first_merges(&merges, path, commit_a, commit_b);
 	switch (parent_count) {
 	case 0:
-		MERGE_WARNING(path, "merge following commits not found");
+		output(o, 1, _("Failed to merge submodule %s (merge following commits not found)"), path);
 		break;
 
 	case 1:
-		MERGE_WARNING(path, "not fast-forward");
-		fprintf(stderr, "Found a possible merge resolution "
-				"for the submodule:\n");
+		output(o, 1, _("Failed to merge submodule %s (not fast-forward)"), path);
+		output(o, 1, _("Found a possible merge resolution for the submodule:\n"));
 		print_commit((struct commit *) merges.objects[0].item);
-		fprintf(stderr,
+		output(o, 1, _(
 			"If this is correct simply add it to the index "
 			"for example\n"
 			"by using:\n\n"
 			"  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
-			"which will accept this suggestion.\n",
+			"which will accept this suggestion.\n"),
 			oid_to_hex(&merges.objects[0].item->oid), path);
 		break;
 
 	default:
-		MERGE_WARNING(path, "multiple merges found");
+		output(o, 1, _("Failed to merge submodule %s (multiple merges found)"), path);
 		for (i = 0; i < merges.nr; i++)
 			print_commit((struct commit *) merges.objects[i].item);
 	}
@@ -1205,12 +1203,11 @@ static int merge_file_1(struct merge_options *o,
 				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result->clean = merge_submodule(&result->oid,
+			result->clean = merge_submodule(o, &result->oid,
 						       one->path,
 						       &one->oid,
 						       &a->oid,
-						       &b->oid,
-						       !o->call_depth);
+						       &b->oid);
 		} else if (S_ISLNK(a->mode)) {
 			switch (o->recursive_variant) {
 			case MERGE_RECURSIVE_NORMAL:
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 15%]

* [PATCH] submodule: port submodule subcommand 'foreach' from shell to C
  2018-05-10  6:37     ` Junio C Hamano
@ 2018-05-10 21:25       ` " Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-10 21:25 UTC (permalink / raw)
  To: gitster; +Cc: christian.couder, git, jonathantanmy, pc44800, sbeller

From: Prathamesh Chavan <pc44800@gmail.com>

This aims to make git-submodule foreach a builtin. 'foreach' is ported to
the submodule--helper, and submodule--helper is called from
git-submodule.sh.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---

This is a resend of the last commit in origin/pc/submodule-helper-foreach
It addresses the micro nits of funny comment indentation.

Thanks,
Stefan

 builtin/submodule--helper.c | 144 ++++++++++++++++++++++++++++++++++++
 git-submodule.sh            |  39 +---------
 2 files changed, 145 insertions(+), 38 deletions(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index c2403a915ff..4002026d1ac 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -439,6 +439,149 @@ static void for_each_listed_submodule(const struct module_list *list,
 		fn(list->entries[i], cb_data);
 }
 
+struct cb_foreach {
+	int argc;
+	const char **argv;
+	const char *prefix;
+	int quiet;
+	int recursive;
+};
+#define CB_FOREACH_INIT { 0 }
+
+static void runcommand_in_submodule_cb(const struct cache_entry *list_item,
+				       void *cb_data)
+{
+	struct cb_foreach *info = cb_data;
+	const char *path = list_item->name;
+	const struct object_id *ce_oid = &list_item->oid;
+
+	const struct submodule *sub;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	char *displaypath;
+
+	displaypath = get_submodule_displaypath(path, info->prefix);
+
+	sub = submodule_from_path(the_repository, &null_oid, path);
+
+	if (!sub)
+		die(_("No url found for submodule path '%s' in .gitmodules"),
+			displaypath);
+
+	if (!is_submodule_populated_gently(path, NULL))
+		goto cleanup;
+
+	prepare_submodule_repo_env(&cp.env_array);
+
+	/*
+	 * For the purpose of executing <command> in the submodule,
+	 * separate shell is used for the purpose of running the
+	 * child process.
+	 */
+	cp.use_shell = 1;
+	cp.dir = path;
+
+	/*
+	 * NEEDSWORK: the command currently has access to the variables $name,
+	 * $sm_path, $displaypath, $sha1 and $toplevel only when the command
+	 * contains a single argument. This is done for maintaining a faithful
+	 * translation from shell script.
+	 */
+	if (info->argc == 1) {
+		char *toplevel = xgetcwd();
+		struct strbuf sb = STRBUF_INIT;
+
+		argv_array_pushf(&cp.env_array, "name=%s", sub->name);
+		argv_array_pushf(&cp.env_array, "sm_path=%s", path);
+		argv_array_pushf(&cp.env_array, "displaypath=%s", displaypath);
+		argv_array_pushf(&cp.env_array, "sha1=%s",
+				oid_to_hex(ce_oid));
+		argv_array_pushf(&cp.env_array, "toplevel=%s", toplevel);
+
+		/*
+		 * Since the path variable was accessible from the script
+		 * before porting, it is also made available after porting.
+		 * The environment variable "PATH" has a very special purpose
+		 * on windows. And since environment variables are
+		 * case-insensitive in windows, it interferes with the
+		 * existing PATH variable. Hence, to avoid that, we expose
+		 * path via the args argv_array and not via env_array.
+		 */
+		sq_quote_buf(&sb, path);
+		argv_array_pushf(&cp.args, "path=%s; %s",
+				 sb.buf, info->argv[0]);
+		strbuf_release(&sb);
+		free(toplevel);
+	} else {
+		argv_array_pushv(&cp.args, info->argv);
+	}
+
+	if (!info->quiet)
+		printf(_("Entering '%s'\n"), displaypath);
+
+	if (info->argv[0] && run_command(&cp))
+		die(_("run_command returned non-zero status for %s\n."),
+			displaypath);
+
+	if (info->recursive) {
+		struct child_process cpr = CHILD_PROCESS_INIT;
+
+		cpr.git_cmd = 1;
+		cpr.dir = path;
+		prepare_submodule_repo_env(&cpr.env_array);
+
+		argv_array_pushl(&cpr.args, "--super-prefix", NULL);
+		argv_array_pushf(&cpr.args, "%s/", displaypath);
+		argv_array_pushl(&cpr.args, "submodule--helper", "foreach", "--recursive",
+				NULL);
+
+		if (info->quiet)
+			argv_array_push(&cpr.args, "--quiet");
+
+		argv_array_pushv(&cpr.args, info->argv);
+
+		if (run_command(&cpr))
+			die(_("run_command returned non-zero status while"
+				"recursing in the nested submodules of %s\n."),
+				displaypath);
+	}
+
+cleanup:
+	free(displaypath);
+}
+
+static int module_foreach(int argc, const char **argv, const char *prefix)
+{
+	struct cb_foreach info = CB_FOREACH_INIT;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+
+	struct option module_foreach_options[] = {
+		OPT__QUIET(&info.quiet, N_("Suppress output of entering each submodule command")),
+		OPT_BOOL(0, "recursive", &info.recursive,
+			 N_("Recurse into nested submodules")),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper foreach [--quiet] [--recursive] <command>"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, module_foreach_options,
+			     git_submodule_helper_usage, PARSE_OPT_KEEP_UNKNOWN);
+
+	if (module_list_compute(0, NULL, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	info.argc = argc;
+	info.argv = argv;
+	info.prefix = prefix;
+
+	for_each_listed_submodule(&list, runcommand_in_submodule_cb, &info);
+
+	return 0;
+}
+
 struct init_cb {
 	const char *prefix;
 	unsigned int flags;
@@ -1841,6 +1984,7 @@ static struct cmd_struct commands[] = {
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
 	{"print-default-remote", print_default_remote, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 331d71c908b..cba585f0754 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -323,44 +323,7 @@ cmd_foreach()
 		shift
 	done
 
-	toplevel=$(pwd)
-
-	# dup stdin so that it can be restored when running the external
-	# command in the subshell (and a recursive call to this function)
-	exec 3<&0
-
-	{
-		git submodule--helper list --prefix "$wt_prefix" ||
-		echo "#unmatched" $?
-	} |
-	while read -r mode sha1 stage sm_path
-	do
-		die_if_unmatched "$mode" "$sha1"
-		if test -e "$sm_path"/.git
-		then
-			displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
-			say "$(eval_gettext "Entering '\$displaypath'")"
-			name=$(git submodule--helper name "$sm_path")
-			(
-				prefix="$prefix$sm_path/"
-				sanitize_submodule_env
-				cd "$sm_path" &&
-				# we make $path available to scripts ...
-				path=$sm_path &&
-				if test $# -eq 1
-				then
-					eval "$1"
-				else
-					"$@"
-				fi &&
-				if test -n "$recursive"
-				then
-					cmd_foreach "--recursive" "$@"
-				fi
-			) <&3 3<&- ||
-			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
-		fi
-	done
+	git ${wt_prefix:+-C "$wt_prefix"} ${prefix:+--super-prefix "$prefix"} submodule--helper foreach ${GIT_QUIET:+--quiet} ${recursive:+--recursive} "$@"
 }
 
 #
-- 
2.17.0.255.g8bfb7c0704


^ permalink raw reply	[relevance 20%]

* Re: [PATCH 0/2] Submodule merging: i18n, verbosity
  2018-05-10 21:19       ` [PATCH 0/2] Submodule merging: i18n, verbosity Stefan Beller
  2018-05-10 21:19         ` [PATCH 1/2] submodule.c: move submodule merging to merge-recursive.c Stefan Beller
  2018-05-10 21:19         ` [PATCH 2/2] merge-recursive: i18n submodule merge output and respect verbosity Stefan Beller
@ 2018-05-11  0:04         ` Elijah Newren
  2018-05-11  1:00           ` Stefan Beller
  2 siblings, 1 reply; 200+ results
From: Elijah Newren @ 2018-05-11  0:04 UTC (permalink / raw)
  To: Stefan Beller
  Cc: leif.middelschulte, Git Mailing List, Junio C Hamano, sandals

On Thu, May 10, 2018 at 2:19 PM, Stefan Beller <sbeller@google.com> wrote:
> Leif wrote:
>> Sure, let me know what to use instead and I’ll update and resubmit the patch.
>> Sure, but `MERGE_WARNING` prefixes all the messages with "Failed to
>> merge submodule“.
>
> I thought about replying and coming up with good reasons, but I wrote some
> patches instead.
>
> They can also be found at https://github.com/stefanbeller/git/tree/submodule_i18n_verbose
>
> I think these would be a good foundation for your patch as well, as you can use the
> output() function for the desired cases.
>
> Feel free to take these patches as part of your series or adapt
> (or be inspired by) as needed.

This is awesome.  In addition to the good reasons you gave, switching
merge_submodule() to use output() was one of several things on my todo
list since I think it'd be needed for remerge-diffs
(https://bugs.chromium.org/p/git/issues/detail?id=12) and might be
useful for merges in bare repos; thanks for tackling it.

Patches look good to me.  Having Leif's patch on top of these two
would be great.

Elijah

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 0/2] Submodule merging: i18n, verbosity
  2018-05-11  0:04         ` [PATCH 0/2] Submodule merging: i18n, verbosity Elijah Newren
@ 2018-05-11  1:00           ` Stefan Beller
  2018-05-14 20:57             ` [PATCH 0/1] rebased: inform about auto submodule ff during merge Leif Middelschulte
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-11  1:00 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Leif Middelschulte, Git Mailing List, Junio C Hamano, sandals

Hi Elijah,

On Thu, May 10, 2018 at 5:04 PM, Elijah Newren <newren@gmail.com> wrote:
> On Thu, May 10, 2018 at 2:19 PM, Stefan Beller <sbeller@google.com> wrote:
>> Leif wrote:
>>> Sure, let me know what to use instead and I’ll update and resubmit the patch.
>>> Sure, but `MERGE_WARNING` prefixes all the messages with "Failed to
>>> merge submodule“.
>>
>> I thought about replying and coming up with good reasons, but I wrote some
>> patches instead.
>>
>> They can also be found at https://github.com/stefanbeller/git/tree/submodule_i18n_verbose
>>
>> I think these would be a good foundation for your patch as well, as you can use the
>> output() function for the desired cases.
>>
>> Feel free to take these patches as part of your series or adapt
>> (or be inspired by) as needed.
>
> This is awesome.  In addition to the good reasons you gave, switching
> merge_submodule() to use output() was one of several things on my todo
> list since I think it'd be needed for remerge-diffs
> (https://bugs.chromium.org/p/git/issues/detail?id=12) and might be
> useful for merges in bare repos; thanks for tackling it.

Thanks for the encouraging words!
The one nit I find on that series is that we need to rely on and export
the add_submodule_odb function as I want to get rid of that function
once the object store series has progressed far enough.

>
> Patches look good to me.  Having Leif's patch on top of these two
> would be great.

ok, Let's go with that.

Stefan

^ permalink raw reply	[relevance 5%]

* Re: [PATCH v2 0/4] Fix mem leaks of recent object store conversions.
      [irrelevant] <20180510195849.28023-1-sbeller@google.com>
@ 2018-05-11  8:37 ` Jeff King
  2018-05-11 18:59   ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Jeff King @ 2018-05-11  8:37 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git

On Thu, May 10, 2018 at 12:58:45PM -0700, Stefan Beller wrote:

> This series replaces the two commits that were queued on sb/object-store-replace,
> fixing memory leaks that were recently introduced.
> 
> Compared to v1, I merged the two independent series from yesterday,
> rewrote the commit message to clear up Junios confusion and addresses Peffs
> comments for the packfiles as well.

Mostly. :)

My one remaining complaint is that the bitmap code may hold on to a
dangling pointer to a packed_git after this series.

I think that is part of a larger problem, though, which is that the
bitmap code's globals need to be part of the struct raw_object_store.
I think this can already cause problems before your series if we were to
try to use bitmaps in both a superproject and a submodule in the same
process, though I think we'd at least hit the "ignoring extra bitmap
file" code path in open_pack_bitmap_1(). So right now it's an annoyance,
but after your series it becomes a potential segfault.

-Peff

^ permalink raw reply	[relevance 4%]

* Re: [PATCH v2 0/4] Fix mem leaks of recent object store conversions.
  2018-05-11  8:37 ` [PATCH v2 0/4] Fix mem leaks of recent object store conversions Jeff King
@ 2018-05-11 18:59   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-11 18:59 UTC (permalink / raw)
  To: Jeff King; +Cc: Junio C Hamano, git

On Fri, May 11, 2018 at 1:37 AM, Jeff King <peff@peff.net> wrote:
> On Thu, May 10, 2018 at 12:58:45PM -0700, Stefan Beller wrote:
>
>> This series replaces the two commits that were queued on sb/object-store-replace,
>> fixing memory leaks that were recently introduced.
>>
>> Compared to v1, I merged the two independent series from yesterday,
>> rewrote the commit message to clear up Junios confusion and addresses Peffs
>> comments for the packfiles as well.
>
> Mostly. :)
>
> My one remaining complaint is that the bitmap code may hold on to a
> dangling pointer to a packed_git after this series.

Ok, I'll look into that.

>
> I think that is part of a larger problem, though, which is that the
> bitmap code's globals need to be part of the struct raw_object_store.
> I think this can already cause problems before your series if we were to
> try to use bitmaps in both a superproject and a submodule in the same
> process, though I think we'd at least hit the "ignoring extra bitmap
> file" code path in open_pack_bitmap_1(). So right now it's an annoyance,
> but after your series it becomes a potential segfault.

Ok, maybe we'll need to convert bitmaps into the object store for that.

Thanks for the pointer,
Stefan

^ permalink raw reply	[relevance 2%]

* [PATCH] git-submodule.sh: try harder to fetch a submodule
@ 2018-05-11 23:17 Stefan Beller
  2018-05-11 23:28 ` Jonathan Nieder
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-11 23:17 UTC (permalink / raw)
  To: gitster; +Cc: git, Stefan Beller

This is the logical continuum of fb43e31f2b4 (submodule: try harder to
fetch needed sha1 by direct fetching sha1, 2016-02-23) and fixes it as
some assumptions were not correct.

> If $sha1 was not part of the default fetch ... fail ourselves here
assumes that the fetch_in_submodule only fails when the serverside does
not support fetching by sha1.

There are other failures, why such a fetch may fail, such as
    fatal: Couldn't find remote ref HEAD
which can happen if the remote side doesn't advertise HEAD. Not advertising
HEAD is allowed by the protocol spec and would happen, if HEAD points at a
ref, that this user cannot see (due to ACLs for example).

So do try even harder for a submodule by ignoring the exit code of the
first fetch and rather relying on the following is_tip_reachable to
see if we try fetching again.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh | 1 -
 1 file changed, 1 deletion(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 24914963ca2..13b378a6c8f 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -614,7 +614,6 @@ cmd_update()
 				# is not reachable from a ref.
 				is_tip_reachable "$sm_path" "$sha1" ||
 				fetch_in_submodule "$sm_path" $depth ||
-				die "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
 
 				# Now we tried the usual fetch, but $sha1 may
 				# not be reachable from any of the refs
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 17%]

* Re: [PATCH] git-submodule.sh: try harder to fetch a submodule
  2018-05-11 23:17 [PATCH] git-submodule.sh: try harder to fetch a submodule Stefan Beller
@ 2018-05-11 23:28 ` Jonathan Nieder
  2018-05-11 23:42   ` Stefan Beller
      [irrelevant]   ` <xmqqmux5g2pa.fsf@gitster-ct.c.googlers.com>
  0 siblings, 2 replies; 200+ results
From: Jonathan Nieder @ 2018-05-11 23:28 UTC (permalink / raw)
  To: Stefan Beller; +Cc: gitster, git

Hi,

Stefan Beller wrote:

> This is the logical continuum of fb43e31f2b4 (submodule: try harder to
> fetch needed sha1 by direct fetching sha1, 2016-02-23) and fixes it as
> some assumptions were not correct.

Interesting.

I think what would help most is an example set of commands I can use
to reproduce this (bonus points if in the form of a test).

> > If $sha1 was not part of the default fetch ... fail ourselves here
> assumes that the fetch_in_submodule only fails when the serverside does

I'm having some trouble with the formatting here.  Is the part
preceded by '>' a quote, and if so a quote from what?

> not support fetching by sha1.
>
> There are other failures, why such a fetch may fail, such as
>     fatal: Couldn't find remote ref HEAD
> which can happen if the remote side doesn't advertise HEAD. Not advertising

nit: it can be useful to have a blank line before and after such
example output to help both my eyes and tools like "git log
--format='%w(100)%b'" to understand the formatting.

> HEAD is allowed by the protocol spec and would happen, if HEAD points at a
> ref, that this user cannot see (due to ACLs for example).

A more typical example would be if the ref simply doesn't exist (i.e.,
is a branch yet to be born).

> So do try even harder for a submodule by ignoring the exit code of the
> first fetch and rather relying on the following is_tip_reachable to
> see if we try fetching again.
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  git-submodule.sh | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 24914963ca2..13b378a6c8f 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -614,7 +614,6 @@ cmd_update()
>  				# is not reachable from a ref.
>  				is_tip_reachable "$sm_path" "$sha1" ||
>  				fetch_in_submodule "$sm_path" $depth ||

Is keeping the '||' at the end of this line intended?

> -				die "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
>  
>  				# Now we tried the usual fetch, but $sha1 may
>  				# not be reachable from any of the refs
> 				is_tip_reachable "$sm_path" "$sha1" ||
> 				fetch_in_submodule "$sm_path" $depth "$sha1" ||
> 				die "$(eval_gettext "Fetched in submodule path '\$displaypath', but it did not contain \$sha1. Direct fetching of that commit failed.")"

Should this error message be changed?

Thanks,
Jonathan

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] git-submodule.sh: try harder to fetch a submodule
  2018-05-11 23:28 ` Jonathan Nieder
@ 2018-05-11 23:42   ` Stefan Beller
      [irrelevant]   ` <xmqqmux5g2pa.fsf@gitster-ct.c.googlers.com>
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-11 23:42 UTC (permalink / raw)
  To: Jonathan Nieder; +Cc: Junio C Hamano, git

On Fri, May 11, 2018 at 4:28 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> Hi,
>
> Stefan Beller wrote:
>
>> This is the logical continuum of fb43e31f2b4 (submodule: try harder to
>> fetch needed sha1 by direct fetching sha1, 2016-02-23) and fixes it as
>> some assumptions were not correct.
>
> Interesting.
>
> I think what would help most is an example set of commands I can use
> to reproduce this (bonus points if in the form of a test).

I tried coming up with a test in git-core that produces a remote similar to
Gerrit, and let me tell you, it's not Git that is weird here. ;)

>> > If $sha1 was not part of the default fetch ... fail ourselves here
>> assumes that the fetch_in_submodule only fails when the serverside does
>
> I'm having some trouble with the formatting here.  Is the part
> preceded by '>' a quote, and if so a quote from what?

The quote is from fb43e31f2b4.

>> There are other failures, why such a fetch may fail, such as
>>     fatal: Couldn't find remote ref HEAD
>> which can happen if the remote side doesn't advertise HEAD. Not advertising
>
> nit: it can be useful to have a blank line before and after such
> example output to help both my eyes and tools like "git log
> --format='%w(100)%b'" to understand the formatting.

Why would you use this formatting?

>
>> HEAD is allowed by the protocol spec and would happen, if HEAD points at a
>> ref, that this user cannot see (due to ACLs for example).
>
> A more typical example would be if the ref simply doesn't exist (i.e.,
> is a branch yet to be born).

Oh, I checked that but not for the submodule case, let me check that.

>> diff --git a/git-submodule.sh b/git-submodule.sh
>> index 24914963ca2..13b378a6c8f 100755
>> --- a/git-submodule.sh
>> +++ b/git-submodule.sh
>> @@ -614,7 +614,6 @@ cmd_update()
>>                               # is not reachable from a ref.
>>                               is_tip_reachable "$sm_path" "$sha1" ||
>>                               fetch_in_submodule "$sm_path" $depth ||
>
> Is keeping the '||' at the end of this line intended?

yes, as we only want to run the code below if there was some error.

>> -                             die "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
>>
>>                               # Now we tried the usual fetch, but $sha1 may
>>                               # not be reachable from any of the refs
>>                               is_tip_reachable "$sm_path" "$sha1" ||
>>                               fetch_in_submodule "$sm_path" $depth "$sha1" ||
>>                               die "$(eval_gettext "Fetched in submodule path '\$displaypath', but it did not contain \$sha1. Direct fetching of that commit failed.")"
>
> Should this error message be changed?

I don't think so?

^ permalink raw reply	[relevance 8%]

* [PATCH 0/1] rebased: inform about auto submodule ff during merge
  2018-05-11  1:00           ` Stefan Beller
@ 2018-05-14 20:57             ` Leif Middelschulte
      [irrelevant]               ` <20180514205737.21313-2-leif.middelschulte@gmail.com>
  0 siblings, 1 reply; 200+ results
From: Leif Middelschulte @ 2018-05-14 20:57 UTC (permalink / raw)
  To: git; +Cc: gitster, Leif Middelschulte

From: Leif Middelschulte <Leif.Middelschulte@gmail.com>

This patch is in response to Stefan Beller's Commit 0357af480
("merge-recursive: i18n submodule merge output and respect verbosity",
2018-05-10) and is based on the changes it provided.

Leif Middelschulte (1):
  Inform about fast-forwarding of submodules during merge

 merge-recursive.c | 4 ++++
 1 file changed, 4 insertions(+)

-- 
2.15.1 (Apple Git-101)


^ permalink raw reply	[relevance 8%]

* Re: [PATCH 1/1] Inform about fast-forwarding of submodules during merge
      [irrelevant]               ` <20180514205737.21313-2-leif.middelschulte@gmail.com>
@ 2018-05-15  0:41                 ` " Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15  0:41 UTC (permalink / raw)
  To: Leif Middelschulte; +Cc: git, Junio C Hamano

On Mon, May 14, 2018 at 1:57 PM, Leif Middelschulte
<leif.middelschulte@gmail.com> wrote:
> From: Leif Middelschulte <Leif.Middelschulte@gmail.com>
>
> Inform the user about an automatically fast-forwarded submodule. The silent merge
> behavior was introduced by commit 68d03e4a6e44 ("Implement automatic fast-forward
> merge for submodules", 2010-07-07)).
>
> Signed-off-by: Leif Middelschulte <Leif.Middelschulte@gmail.com>

Thanks for following up with a patch.
This looks good to me!

Thanks,
Stefan

> ---
>  merge-recursive.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index a4b91d17f..4a03044d1 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1093,10 +1093,14 @@ static int merge_submodule(struct merge_options *o,
>         /* Case #1: a is contained in b or vice versa */
>         if (in_merge_bases(commit_a, commit_b)) {
>                 oidcpy(result, b);
> +               output(o, 1, _("Note: Fast-forwarding submodule %s to the following commit"), path);
> +               output_commit_title(o, commit_b);
>                 return 1;
>         }
>         if (in_merge_bases(commit_b, commit_a)) {
>                 oidcpy(result, a);
> +               output(o, 1, _("Note: Fast-forwarding submodule %s to the following commit:"), path);
> +               output_commit_title(o, commit_a);
>                 return 1;
>         }
>
> --
> 2.15.1 (Apple Git-101)
>

^ permalink raw reply	[relevance 5%]

* Re: [RFC PATCH 01/10] config: make config_from_gitmodules generally useful
      [irrelevant] ` <20180514105823.8378-2-ao2@ao2.it>
@ 2018-05-15  1:05   ` Stefan Beller
  2018-06-20 18:06     ` Antonio Ospite
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-15  1:05 UTC (permalink / raw)
  To: Antonio Ospite
  Cc: git, Brandon Williams, Daniel Graña, Jonathan Nieder,
	Richard Hartmann

On Mon, May 14, 2018 at 3:58 AM, Antonio Ospite <ao2@ao2.it> wrote:
> The config_from_gitmodules() function is a good candidate for
> a centralized point where to read the gitmodules configuration file.

It is very tempting to use that function. However it was introduced
specifically to not do that. ;)

See the series that was merged at 5aa0b6c506c (Merge branch
'bw/grep-recurse-submodules', 2017-08-22), specifically
f20e7c1ea24 (submodule: remove submodule.fetchjobs from
submodule-config parsing, 2017-08-02), where both
builtin/fetch as well as the submodule helper use the pattern to
read from the .gitmodules file va this function and then overlay it
with the read from config.

> Add a repo argument to it to make the function more general, and adjust
> the current callers in cmd_fetch and update-clone.

This could be a preparatory patch, but including it here is fine, too.

> As a proof of the utility of the change, start using the function also
> in repo_read_gitmodules which was basically doing the same operations.

I think they were separated for the reason outlined above, or what Brandon
said in his reply.

I think extending 'repo_read_gitmodules' makes sense, as that is
used everywhere for the loading of submodule configuration.

^ permalink raw reply	[relevance 6%]

* Re: [RFC PATCH 00/10] Make submodules work if .gitmodules is not checked out
      [irrelevant] <20180514105823.8378-1-ao2@ao2.it>
      [irrelevant] ` <20180514105823.8378-2-ao2@ao2.it>
@ 2018-05-15  1:14 ` Stefan Beller
      [irrelevant] ` <20180514105823.8378-3-ao2@ao2.it>
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15  1:14 UTC (permalink / raw)
  To: Antonio Ospite
  Cc: git, Brandon Williams, Daniel Graña, Jonathan Nieder,
	Richard Hartmann

Hi Antonio,

thanks for sending this series! Happy to review it!

>   - my git terminology may still be a little off: do "work tree" and
>     "work directory" mean the same thing?

Back in the old days, you had a "worktree" which is a directory where
things are checked out and you modify files. It sort of the opposite of the
"git directory", which git uses to store all its information.

Then quite some time later, the command git-worktree was invented.
Now we had 2 things with the same name, which is unfortunate.
But as the command git-worktree added more of these directories
that you could work in, the name collision was not apparent.

Later when people noticed the subtle difference between the
command and the thing on the file system, consensus seemed to be
that the thing on the file system should rather be called "working tree"
such that it sounds very similar but is distinguishable from the command.

However the outcome of the discussion did not yield a bi&complete
refactoring of the code base, such that there are still places with "worktree"
referring to the thing on the FS, "working trees".

I am not aware that "working directory" is an official term we use in
any documentation for Git, but it sounds like you mean a "working tree".
(From a point of view not based on the version control, "working directory"
may sound more correct, note however as the directories in Git are
named trees, the working "tree" sounds as if you can make changes to
Git trees, ... which you can. :) )

> If anyone wanted to pick up and finish the work feel free to do so,
> otherwise please comment and I'll try to address issues as time permits.

First let's review this series. :)

Thanks,
Stefan

^ permalink raw reply	[relevance 4%]

* Re: [RFC PATCH 02/10] submodule: factor out a config_gitmodules_set function
      [irrelevant] ` <20180514105823.8378-3-ao2@ao2.it>
@ 2018-05-15  1:20   ` Stefan Beller
  2018-06-20 18:41     ` Antonio Ospite
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-15  1:20 UTC (permalink / raw)
  To: Antonio Ospite
  Cc: git, Brandon Williams, Daniel Graña, Jonathan Nieder,
	Richard Hartmann

On Mon, May 14, 2018 at 3:58 AM, Antonio Ospite <ao2@ao2.it> wrote:
> Introduce a new config_gitmodules_set function to write config values to the
> .gitmodules file.
>
> This is in preparation for a future change which will use the function
> to write to the .gitmodules file in a more controlled way instead of
> using "git config -f .gitmodules".
>
> Signed-off-by: Antonio Ospite <ao2@ao2.it>
> ---
>
> Not sure about the name, and maybe it can go in config.c for symmetry with
> config_from_gitmodules?

What is the function about (in the end state and now) ?
is it more of a
* configure_submodules_config()
  which would convey it is a generic function to configure submodules
  (i.e. it may not even write to *the* .gitmodules file but somewhere else,
  such as a helper ref)
  This doesn't sound like it as we make use of the function in
  update_path_in_gitmodules() that is used from git-mv, which would want
  to ensure that the specific .gitmodules file is changed.
* gitmodules_file_set()
  that focuses on the specific file that we want to modify?
* ...

Let's continue reading the series to see the end state for
a good name.

Stefan

^ permalink raw reply	[relevance 7%]

* Re: [RFC PATCH 03/10] t7411: be nicer to other tests and really clean things up
      [irrelevant] ` <20180514105823.8378-4-ao2@ao2.it>
@ 2018-05-15  1:23   ` Stefan Beller
  2018-06-20 21:16     ` Antonio Ospite
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-15  1:23 UTC (permalink / raw)
  To: Antonio Ospite
  Cc: git, Brandon Williams, Daniel Graña, Jonathan Nieder,
	Richard Hartmann

On Mon, May 14, 2018 at 3:58 AM, Antonio Ospite <ao2@ao2.it> wrote:
> Tests 5 and 8 in t/t7411-submodule-config.sh add two commits with
> invalid lines in .gitmodules but then only the second commit is removed.
>
> This may affect subsequent tests if they assume that the .gitmodules
> file has no errors.
>
> Since those commits are not needed anymore remove both of them.
>
> Signed-off-by: Antonio Ospite <ao2@ao2.it>
> ---
>
> I am putting these fixups to the test-suite before the patch that actually
> needs them so that the test-suite passes after each commit.
>
>  t/t7411-submodule-config.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/t/t7411-submodule-config.sh b/t/t7411-submodule-config.sh
> index 0bde5850a..a648de6a9 100755
> --- a/t/t7411-submodule-config.sh
> +++ b/t/t7411-submodule-config.sh
> @@ -135,7 +135,7 @@ test_expect_success 'error in history in fetchrecursesubmodule lets continue' '
>                         HEAD submodule \
>                                 >actual &&
>                 test_cmp expect_error actual  &&
> -               git reset --hard HEAD^
> +               git reset --hard HEAD~2
>         )
>  '

As this is the last test in this file, we do not change any subsequent
tests in a subtle way.
Good!

This is
Reviewed-by: Stefan Beller <sbeller@google.com>

FYI:
This test -- of course -- doesn't quite follow the latest coding guidelines,
as usually we'd prefer a test_when_finished "<cmd to restore>"
at the beginning of a test.

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 2/2] merge-recursive: i18n submodule merge output and respect verbosity
  2018-05-10 21:19         ` [PATCH 2/2] merge-recursive: i18n submodule merge output and respect verbosity Stefan Beller
@ 2018-05-15  1:25           ` Elijah Newren
  0 siblings, 0 replies; 200+ results
From: Elijah Newren @ 2018-05-15  1:25 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Leif Middelschulte, Git Mailing List, Junio C Hamano

I know I said the patches looked okay earlier, but I just noticed something...

On Thu, May 10, 2018 at 2:19 PM, Stefan Beller <sbeller@google.com> wrote:

>         case 1:
> -               MERGE_WARNING(path, "not fast-forward");
> -               fprintf(stderr, "Found a possible merge resolution "
> -                               "for the submodule:\n");
> +               output(o, 1, _("Failed to merge submodule %s (not fast-forward)"), path);

We allow folks to set GIT_MERGE_VERBOSITY to change how much output
they get.  A setting of 1 should only show conflicts or major
warnings.  2 is the default and adds a few more messages (e.g.
"Auto-merging $PATH", "Adding $PATH" for one-sided adds, etc.), higher
levels show even more.

Anyway this output message is correct to use level 1 since this is a
conflict, but...

> +               output(o, 1, _("Found a possible merge resolution for the submodule:\n"));

I think this should use level 2.

>                 print_commit((struct commit *) merges.objects[0].item);
> -               fprintf(stderr,
> +               output(o, 1, _(
>                        "If this is correct simply add it to the index "
>                        "for example\n"
>                        "by using:\n\n"
>                        "  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
>-                       "which will accept this suggestion.\n",
>+                       "which will accept this suggestion.\n"),
>                        oid_to_hex(&merges.objects[0].item->oid), path);

and so should this one (in fact, I'm tempted to say these last two
should use level 3, but since it looks like a command users may have
difficulty finding on their own, I'm okay with going with 2).

^ permalink raw reply	[relevance 4%]

* Re: [RFC PATCH 04/10] submodule--helper: add a new 'config' subcommand
      [irrelevant] ` <20180514105823.8378-5-ao2@ao2.it>
@ 2018-05-15  1:33   ` Stefan Beller
  2018-06-20 21:32     ` Antonio Ospite
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-15  1:33 UTC (permalink / raw)
  To: Antonio Ospite
  Cc: git, Brandon Williams, Daniel Graña, Jonathan Nieder,
	Richard Hartmann

On Mon, May 14, 2018 at 3:58 AM, Antonio Ospite <ao2@ao2.it> wrote:
> Add a new 'config' subcommand to 'submodule--helper', this extra level
> of indirection makes it possible to add some flexibility to how the
> submodules configuration is handled.
>
> Signed-off-by: Antonio Ospite <ao2@ao2.it>
> ---
>  builtin/submodule--helper.c | 39 +++++++++++++++++++++++++++++++++++++
>  t/t7411-submodule-config.sh | 26 +++++++++++++++++++++++++
>  2 files changed, 65 insertions(+)
>
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 9e8f2acd5..b32110e3b 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -1825,6 +1825,44 @@ static int is_active(int argc, const char **argv, const char *prefix)
>         return !is_submodule_active(the_repository, argv[1]);
>  }
>
> +static int config_print_callback(const char *key_, const char *value_, void *cb_data)
> +{
> +       char *key = cb_data;
> +
> +       if (!strcmp(key, key_))
> +               printf("%s\n", value_);
> +
> +       return 0;
> +}
> +
> +static int module_config(int argc, const char **argv, const char *prefix)
> +{
> +       int ret;
> +
> +       if (argc < 2 || argc > 3)
> +               die("submodule--helper config takes 1 or 2 arguments: name [value]");
> +
> +       /* Equivalent to ACTION_GET in builtin/config.c */
> +       if (argc == 2) {
> +               char *key;
> +
> +               ret = git_config_parse_key(argv[1], &key, NULL);
> +               if (ret < 0)
> +                       return CONFIG_INVALID_KEY;
> +
> +               config_from_gitmodules(config_print_callback, the_repository, key);
> +
> +               free(key);
> +               return 0;
> +       }
> +
> +       /* Equivalent to ACTION_SET in builtin/config.c */
> +       if (argc == 3)
> +               return config_gitmodules_set(argv[1], argv[2]);

Ah, here we definitely want to set it in the .gitmodules file?
(Or does that change later in this series?)

> +
> +       return 0;
> +}
> +
>  #define SUPPORT_SUPER_PREFIX (1<<0)
>
>  struct cmd_struct {
> @@ -1850,6 +1888,7 @@ static struct cmd_struct commands[] = {
>         {"push-check", push_check, 0},
>         {"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
>         {"is-active", is_active, 0},
> +       {"config", module_config, 0},
>  };
>
>  int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
> diff --git a/t/t7411-submodule-config.sh b/t/t7411-submodule-config.sh
> index a648de6a9..dfe019f05 100755
> --- a/t/t7411-submodule-config.sh
> +++ b/t/t7411-submodule-config.sh
> @@ -139,4 +139,30 @@ test_expect_success 'error in history in fetchrecursesubmodule lets continue' '
>         )
>  '
>
> +test_expect_success 'reading submodules config with "submodule--helper config"' '
> +       (cd super &&

I think the project prefers a style
of the cd at the same level of the echo and the following commands.

However we might not need the (cd super && ...) via

  echo "../submodule"  >expected
  git -C super ubmodule--helper config submodule.submodule.url >../actual
  test_cmp expected actual

Our friends developing git on Windows will thank us for saving
to spawn a shell as spawning processes is expensive on Windows. :)
Also we have fewer lines of code.

The patch looks good to me,
Thanks,
Stefan

^ permalink raw reply	[relevance 8%]

* Re: [RFC PATCH 09/10] submodule: support reading .gitmodules even when it's not checked out
      [irrelevant] ` <20180514105823.8378-10-ao2@ao2.it>
@ 2018-05-15  1:45   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15  1:45 UTC (permalink / raw)
  To: Antonio Ospite
  Cc: git, Brandon Williams, Daniel Graña, Jonathan Nieder,
	Richard Hartmann

On Mon, May 14, 2018 at 3:58 AM, Antonio Ospite <ao2@ao2.it> wrote:
> When the .gitmodules file is not available in the working directory, try
> using HEAD:.gitmodules from the index.

I think HEAD:.gitmodules is different than the index (the former is
part of the latest commit, whereas the index could have changed via
git-add, that is not committed yet).

> This covers the case when the
> file is part of the repository but for some reason it is not checked
> out, for example because of a sparse checkout.
>
> This makes it possible to use at least the 'git submodule' commands
> which *read* the gitmodules configuration file without fully populating
> the work dir.

Instead of checking for an explicit sparse "hidden" could we just rely on
the file missing? Then I could continue using submodules if I just
"rm .gitmodules".

>
> Writing to .gitmodules wills still require that the file is checked out,
> so check for that in config_gitmodules_set.

That makes sense!

>
> Signed-off-by: Antonio Ospite <ao2@ao2.it>
> ---
>
> I am doing the is_gitmodules_hidden() check in the open for now, I am not sure
> whether it is approprate to do that inside stage_updated_gitmodules.

Why do we need that check at all?

In your use case, you want to checkout *a* .gitmodules file, not necessarily
the .gitmodules file of that repo you're currently working on. So it
sort of makes
sense to prevent cross-repo changes (i.e. committing the .gitmodules
accidentally
into the wrong repo)

Stefan

^ permalink raw reply	[relevance 7%]

* Re: [PATCH] grep: handle corrupt index files early
      [irrelevant] ` <CACsJy8AcG6zGa9vLCwm2B4ishyJVWdFQ2YV0FOZTor_0x8Q64g@mail.gmail.com>
@ 2018-05-15 16:44   ` Stefan Beller
  2018-05-16 15:24     ` Duy Nguyen
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-15 16:44 UTC (permalink / raw)
  To: Duy Nguyen
  Cc: Brandon Williams, Junio C Hamano, Git Mailing List, Antonio Ospite

On Tue, May 15, 2018 at 6:13 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Tue, May 15, 2018 at 3:04 AM, Stefan Beller <sbeller@google.com> wrote:
>> Any other caller of 'repo_read_index' dies upon a negative return of
>> it, so grep should, too.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>
>> Found while reviewing the series
>> https://public-inbox.org/git/20180514105823.8378-1-ao2@ao2.it/
>>
>>  builtin/grep.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/builtin/grep.c b/builtin/grep.c
>> index 6e7bc76785a..69f0743619f 100644
>> --- a/builtin/grep.c
>> +++ b/builtin/grep.c
>> @@ -488,7 +488,8 @@ static int grep_cache(struct grep_opt *opt, struct repository *repo,
>>                 strbuf_addstr(&name, repo->submodule_prefix);
>>         }
>>
>> -       repo_read_index(repo);
>> +       if (repo_read_index(repo) < 0)
>> +               die("index file corrupt");
>
> _() the string (and maybe reuse an existing phrase if found to reduce
> workload on translators)

sbeller@sbeller:/u/git$ git grep -A 1 repo_read_index
builtin/grep.c:491:     if (repo_read_index(repo) < 0)
builtin/grep.c-492-             die("index file corrupt");
--
builtin/ls-files.c:213: if (repo_read_index(&submodule) < 0)
builtin/ls-files.c-214-         die("index file corrupt");
--
builtin/ls-files.c:582: if (repo_read_index(the_repository) < 0)
builtin/ls-files.c-583-         die("index file corrupt");
--
dir.c:3028:     if (repo_read_index(&subrepo) < 0)
dir.c-3029-             die("index file corrupt in repo %s", subrepo.gitdir);
--
repository.c:245:int repo_read_index(struct repository *repo)
repository.c-246-{
--
repository.h:70:         * 'repo_read_index()' can be used to populate 'index'.
repository.h-71-         */
--
repository.h:119:extern int repo_read_index(struct repository *repo);
repository.h-120-
--
submodule-config.c:583:         if (repo_read_index(repo) < 0)
submodule-config.c-584-                 return;
--
submodule.c:1336:       if (repo_read_index(r) < 0)
submodule.c-1337-               die("index file corrupt");

I think this is as good as it gets for using an existing phrase.
None of them are translated, which I would defer to a follow up patch
that translates all(?) of them or just the porcelains.

Thanks,
Stefan

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 35/35] submodule: convert push_unpushed_submodules to take a struct refspec
      [irrelevant]   ` <87k1s52va3.fsf@evledraar.gmail.com>
@ 2018-05-15 16:52     ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15 16:52 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Brandon Williams, git

On Tue, May 15, 2018 at 1:11 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Mon, May 14 2018, Brandon Williams wrote:
>
>> Convert 'push_unpushed_submodules()' to take a 'struct refspec' as a
>> parameter instead of an array of 'const char *'.
>> [...]
>> diff --git a/submodule.h b/submodule.h
>> index e5526f6aa..aae0c9c8f 100644
>> --- a/submodule.h
>> +++ b/submodule.h
>> @@ -100,9 +100,10 @@ extern int submodule_touches_in_range(struct object_id *a,
>>  extern int find_unpushed_submodules(struct oid_array *commits,
>>                                   const char *remotes_name,
>>                                   struct string_list *needs_pushing);
>> +struct refspec;
>>  extern int push_unpushed_submodules(struct oid_array *commits,
>>                                   const struct remote *remote,
>> -                                 const char **refspec, int refspec_nr,
>> +                                 const struct refspec *rs,
>>                                   const struct string_list *push_options,
>>                                   int dry_run);
>>  /*
>
> Why do you prefer doing this to having this on top?:

The fewer includes in header files the better, as then the headers
themselves don't have dependencies. (Otherwise we'd end up
multiplying cache.h ;)

In the source files we have to include all needed headers, but for
the headers, it is better if we can just get away with declaring the
existence of a struct.

This way we reduce compile time, so I'd am not keen on your patch
on top.

This is discussed a lot on stackoverflow, e.g.:
https://softwareengineering.stackexchange.com/questions/195806/forward-declaration-vs-include
https://stackoverflow.com/a/15828094

Stefan

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] git-submodule.sh: try harder to fetch a submodule
      [irrelevant]   ` <xmqqmux5g2pa.fsf@gitster-ct.c.googlers.com>
@ 2018-05-15 19:07     ` Stefan Beller
  2018-05-15 19:40       ` Stefan Beller
  2018-05-15 20:04       ` Jonathan Nieder
  0 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2018-05-15 19:07 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Jonathan Nieder, git

On Fri, May 11, 2018 at 5:03 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> A more typical example would be if the ref simply doesn't exist (i.e.,
>> is a branch yet to be born).
>
> Indeed this is interesting.  At first glance I thought this was
> about underlying "git clone" failing to grab things from a
> repository with unborn HEAD, but that part works perfectly OK.

ok.

> So it probably should be more like
>
>         guard1 || action1 || warn
>         guard2 || action2 || die
>
> so that no matter what the outcome of the action1 is, the second set
> gets executed.
>

I'll resend it with a warning (using say()).

I think we have 2 bugs and this is merely fixing the second bug.

The first bug:
We had a call chain as
  git clone --recurse-submodules=<path spec>
    -> git submodule update --init --recursive $(pathspec)
      -> git submodule--helper update-clone # will clone
        -> git submodule helper clone
          -> git clone --no-checkout --separate-git-dir ...

The call to the "git clone" produces an interesting
submodule state:

  $ git init confused-head
  $ (cd confused-head && git branch test \
        $(git commit-tree $(git write-tree) -m test))
  $ git clone --no-checkout  --depth=1 \
        --separate-git-dir=test.git confused-head/.git test
Cloning into 'test'...
warning: --depth is ignored in local clones; use file:// instead.
done.

  $ git -C test.git config remote.origin.fetch
  $ echo $?
1

(A) Despite the warning of --depth having no impact, the
  omission thereof changes the repository state.
(B) There is no remote.origin.fetch configuration, which
  is weird. See builtin/clone.c:830, that states for this case:

    /*
     * otherwise, the next "git fetch" will
     * simply fetch from HEAD without updating
     * any remote-tracking branch, which is what
     * we want.
     */

I disagree as the next fetch will be confused
(HEAD is not advertised on the next ls-remote)

The patch that is under discussion here is merely
papering over the effect of having no fetch spec,
by allowing the second fetch (fetching the sha1 directly)
to run, which ignores the configuration as a refspec is
given.

However it is still a bug, as such repositories are out there,
which is why I said there are 2 bugs initially. It's just that
the first bug enables the second bug.

Thanks,
Stefan

^ permalink raw reply	[relevance 8%]

* [PATCH] git-submodule.sh: try harder to fetch a submodule
  2018-05-15 19:07     ` Stefan Beller
@ 2018-05-15 19:40       ` Stefan Beller
  2018-05-15 20:04       ` Jonathan Nieder
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15 19:40 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jrnieder

This is the logical continuum of fb43e31f2b4 (submodule: try harder to
fetch needed sha1 by direct fetching sha1, 2016-02-23) and fixes it as
some assumptions were not correct.

The commit states:
> If $sha1 was not part of the default fetch ... fail ourselves here
> assumes that the fetch_in_submodule only fails when the serverside does
> not support fetching by sha1.

There are other failures, why such a fetch may fail, such as
    fatal: Couldn't find remote ref HEAD
which can happen if the remote side doesn't advertise HEAD and we do not
have a local fetch refspec.

Not advertising HEAD is allowed by the protocol spec and would happen,
if HEAD points at an unborn branch for example.

Not having a local fetch refspec can happen when submodules are fetched
shallowly, as then git-clone doesn't setup a fetch refspec.

So do try even harder for a submodule by ignoring the exit code of the
first fetch and rather relying on the following is_tip_reachable to
see if we try fetching again.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 24914963ca2..00fcd69138f 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -614,7 +614,7 @@ cmd_update()
 				# is not reachable from a ref.
 				is_tip_reachable "$sm_path" "$sha1" ||
 				fetch_in_submodule "$sm_path" $depth ||
-				die "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
+				say "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
 
 				# Now we tried the usual fetch, but $sha1 may
 				# not be reachable from any of the refs
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 20%]

* [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive
@ 2018-05-15 20:00 Stefan Beller
  2018-05-15 20:00 ` [PATCH] git-submodule.sh: try harder to fetch a submodule Stefan Beller
                   ` (5 more replies)
  0 siblings, 6 replies; 200+ results
From: Stefan Beller @ 2018-05-15 20:00 UTC (permalink / raw)
  To: git, leif.middelschulte; +Cc: gitster, newren, Stefan Beller

This rerolls the two commits found at [1] with the feedback of Eliah
and puts Leifs patch[2] on top, that I edited according to Eliahs feedback,
but kept Leifs ownership. 

This has addressed all of Eliahs feedback AFAICT.
You'll find a branch-diff below[3], which lacks
the new patch of Leif in that series, but is part of the reroll?

Leif, what do you think?

Thanks,
Stefan

[1] https://public-inbox.org/git/20180510211917.138518-1-sbeller@google.com/
[2] https://public-inbox.org/git/20180514205737.21313-2-leif.middelschulte@gmail.com/
[3] git branch-diff origin/master..origin/sb/submodule-merge-in-merge-recursive origin/master..HEAD  >>0000-cover-letter.patch

Leif Middelschulte (1):
  Inform about fast-forwarding of submodules during merge

Stefan Beller (2):
  submodule.c: move submodule merging to merge-recursive.c
  merge-recursive: i18n submodule merge output and respect verbosity

 merge-recursive.c | 185 +++++++++++++++++++++++++++++++++++++++++++++-
 submodule.c       | 168 +----------------------------------------
 submodule.h       |   6 +-
 3 files changed, 186 insertions(+), 173 deletions(-)

-- 
2.17.0.582.gccdcbd54c44.dirty



1:  e022c7976ae ! 1:  3b638ccac64 submodule.c: move submodule merging to merge-recursive.c
    @@ -20,7 +20,6 @@
         This commit is best viewed with --color-moved.
     
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/merge-recursive.c b/merge-recursive.c
     --- a/merge-recursive.c
2:  2c02ece7e01 ! 2:  eb43110df9d merge-recursive: i18n submodule merge output and respect verbosity
    @@ -7,7 +7,6 @@
         internationalisation as well as the verbosity setting.
     
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/merge-recursive.c b/merge-recursive.c
     --- a/merge-recursive.c
    @@ -73,10 +72,10 @@
     -		fprintf(stderr, "Found a possible merge resolution "
     -				"for the submodule:\n");
     +		output(o, 1, _("Failed to merge submodule %s (not fast-forward)"), path);
    -+		output(o, 1, _("Found a possible merge resolution for the submodule:\n"));
    ++		output(o, 2, _("Found a possible merge resolution for the submodule:\n"));
      		print_commit((struct commit *) merges.objects[0].item);
     -		fprintf(stderr,
    -+		output(o, 1, _(
    ++		output(o, 2, _(
      			"If this is correct simply add it to the index "
      			"for example\n"
      			"by using:\n\n"
-:  ----------- > 3:  4a3bc435023 Inform about fast-forwarding of submodules during merge

^ permalink raw reply	[relevance 11%]

* [PATCH] git-submodule.sh: try harder to fetch a submodule
  2018-05-15 20:00 [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
@ 2018-05-15 20:00 ` Stefan Beller
  2018-05-15 20:00 ` [PATCH 1/3] submodule.c: move submodule merging to merge-recursive.c Stefan Beller
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15 20:00 UTC (permalink / raw)
  To: git, leif.middelschulte; +Cc: gitster, newren, Stefan Beller

This is the logical continuum of fb43e31f2b4 (submodule: try harder to
fetch needed sha1 by direct fetching sha1, 2016-02-23) and fixes it as
some assumptions were not correct.

The commit states:
> If $sha1 was not part of the default fetch ... fail ourselves here
> assumes that the fetch_in_submodule only fails when the serverside does
> not support fetching by sha1.

There are other failures, why such a fetch may fail, such as
    fatal: Couldn't find remote ref HEAD
which can happen if the remote side doesn't advertise HEAD and we do not
have a local fetch refspec.

Not advertising HEAD is allowed by the protocol spec and would happen,
if HEAD points at an unborn branch for example.

Not having a local fetch refspec can happen when submodules are fetched
shallowly, as then git-clone doesn't setup a fetch refspec.

So do try even harder for a submodule by ignoring the exit code of the
first fetch and rather relying on the following is_tip_reachable to
see if we try fetching again.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 git-submodule.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 24914963ca2..00fcd69138f 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -614,7 +614,7 @@ cmd_update()
 				# is not reachable from a ref.
 				is_tip_reachable "$sm_path" "$sha1" ||
 				fetch_in_submodule "$sm_path" $depth ||
-				die "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
+				say "$(eval_gettext "Unable to fetch in submodule path '\$displaypath'")"
 
 				# Now we tried the usual fetch, but $sha1 may
 				# not be reachable from any of the refs
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 19%]

* [PATCH 2/3] merge-recursive: i18n submodule merge output and respect verbosity
  2018-05-15 20:00 [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
  2018-05-15 20:00 ` [PATCH] git-submodule.sh: try harder to fetch a submodule Stefan Beller
  2018-05-15 20:00 ` [PATCH 1/3] submodule.c: move submodule merging to merge-recursive.c Stefan Beller
@ 2018-05-15 20:00 ` Stefan Beller
  2018-05-16  1:17   ` Junio C Hamano
  2018-05-15 20:00 ` [PATCH 3/3] Inform about fast-forwarding of submodules during merge Stefan Beller
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-15 20:00 UTC (permalink / raw)
  To: git, leif.middelschulte; +Cc: gitster, newren, Stefan Beller

The submodule merge code now uses the output() function that is used by
all the rest of the merge-recursive-code. This allows for respecting
internationalisation as well as the verbosity setting.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 merge-recursive.c | 33 +++++++++++++++------------------
 1 file changed, 15 insertions(+), 18 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 700ba15bf88..0571919ee0a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1048,18 +1048,17 @@ static void print_commit(struct commit *commit)
 	strbuf_release(&sb);
 }
 
-#define MERGE_WARNING(path, msg) \
-	warning("Failed to merge submodule %s (%s)", path, msg);
-
-static int merge_submodule(struct object_id *result, const char *path,
+static int merge_submodule(struct merge_options *o,
+			   struct object_id *result, const char *path,
 			   const struct object_id *base, const struct object_id *a,
-			   const struct object_id *b, int search)
+			   const struct object_id *b)
 {
 	struct commit *commit_base, *commit_a, *commit_b;
 	int parent_count;
 	struct object_array merges;
 
 	int i;
+	int search = !o->call_depth;
 
 	/* store a in result in case we fail */
 	oidcpy(result, a);
@@ -1073,21 +1072,21 @@ static int merge_submodule(struct object_id *result, const char *path,
 		return 0;
 
 	if (add_submodule_odb(path)) {
-		MERGE_WARNING(path, "not checked out");
+		output(o, 1, _("Failed to merge submodule %s (not checked out)"), path);
 		return 0;
 	}
 
 	if (!(commit_base = lookup_commit_reference(base)) ||
 	    !(commit_a = lookup_commit_reference(a)) ||
 	    !(commit_b = lookup_commit_reference(b))) {
-		MERGE_WARNING(path, "commits not present");
+		output(o, 1, _("Failed to merge submodule %s (commits not present)"), path);
 		return 0;
 	}
 
 	/* check whether both changes are forward */
 	if (!in_merge_bases(commit_base, commit_a) ||
 	    !in_merge_bases(commit_base, commit_b)) {
-		MERGE_WARNING(path, "commits don't follow merge-base");
+		output(o, 1, _("Failed to merge submodule %s (commits don't follow merge-base)"), path);
 		return 0;
 	}
 
@@ -1116,25 +1115,24 @@ static int merge_submodule(struct object_id *result, const char *path,
 	parent_count = find_first_merges(&merges, path, commit_a, commit_b);
 	switch (parent_count) {
 	case 0:
-		MERGE_WARNING(path, "merge following commits not found");
+		output(o, 1, _("Failed to merge submodule %s (merge following commits not found)"), path);
 		break;
 
 	case 1:
-		MERGE_WARNING(path, "not fast-forward");
-		fprintf(stderr, "Found a possible merge resolution "
-				"for the submodule:\n");
+		output(o, 1, _("Failed to merge submodule %s (not fast-forward)"), path);
+		output(o, 2, _("Found a possible merge resolution for the submodule:\n"));
 		print_commit((struct commit *) merges.objects[0].item);
-		fprintf(stderr,
+		output(o, 2, _(
 			"If this is correct simply add it to the index "
 			"for example\n"
 			"by using:\n\n"
 			"  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
-			"which will accept this suggestion.\n",
+			"which will accept this suggestion.\n"),
 			oid_to_hex(&merges.objects[0].item->oid), path);
 		break;
 
 	default:
-		MERGE_WARNING(path, "multiple merges found");
+		output(o, 1, _("Failed to merge submodule %s (multiple merges found)"), path);
 		for (i = 0; i < merges.nr; i++)
 			print_commit((struct commit *) merges.objects[i].item);
 	}
@@ -1205,12 +1203,11 @@ static int merge_file_1(struct merge_options *o,
 				return ret;
 			result->clean = (merge_status == 0);
 		} else if (S_ISGITLINK(a->mode)) {
-			result->clean = merge_submodule(&result->oid,
+			result->clean = merge_submodule(o, &result->oid,
 						       one->path,
 						       &one->oid,
 						       &a->oid,
-						       &b->oid,
-						       !o->call_depth);
+						       &b->oid);
 		} else if (S_ISLNK(a->mode)) {
 			switch (o->recursive_variant) {
 			case MERGE_RECURSIVE_NORMAL:
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 15%]

* [PATCH 3/3] Inform about fast-forwarding of submodules during merge
  2018-05-15 20:00 [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
                   ` (2 preceding siblings ...)
  2018-05-15 20:00 ` [PATCH 2/3] merge-recursive: i18n submodule merge output and respect verbosity Stefan Beller
@ 2018-05-15 20:00 ` Stefan Beller
  2018-05-16  1:36   ` Elijah Newren
                     ` (2 more replies)
  2018-05-15 20:02 ` [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
  2018-05-15 20:15 ` Leif Middelschulte
  5 siblings, 3 replies; 200+ results
From: Stefan Beller @ 2018-05-15 20:00 UTC (permalink / raw)
  To: git, leif.middelschulte
  Cc: gitster, newren, Leif Middelschulte, Stefan Beller

From: Leif Middelschulte <Leif.Middelschulte@gmail.com>

Inform the user about an automatically fast-forwarded submodule. The
silent merge behavior was introduced by commit 68d03e4a6e44 ("Implement
automatic fast-forward merge for submodules", 2010-07-07)).

Signed-off-by: Leif Middelschulte <Leif.Middelschulte@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 merge-recursive.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0571919ee0a..29a430c418a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1093,10 +1093,26 @@ static int merge_submodule(struct merge_options *o,
 	/* Case #1: a is contained in b or vice versa */
 	if (in_merge_bases(commit_a, commit_b)) {
 		oidcpy(result, b);
+		if (show(o, 3)) {
+			output(o, 1, _("Fast-forwarding submodule %s to the following commit:"), path);
+			output_commit_title(o, commit_b);
+		} else if (show(o, 2))
+			output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(b));
+		else
+			; /* no output */
+
 		return 1;
 	}
 	if (in_merge_bases(commit_b, commit_a)) {
 		oidcpy(result, a);
+		if (show(o, 3)) {
+			output(o, 1, _("Fast-forwarding submodule %s to the following commit:"), path);
+			output_commit_title(o, commit_a);
+		} else if (show(o, 2))
+			output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(a));
+		else
+			; /* no output */
+
 		return 1;
 	}
 
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 14%]

* [PATCH 1/3] submodule.c: move submodule merging to merge-recursive.c
  2018-05-15 20:00 [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
  2018-05-15 20:00 ` [PATCH] git-submodule.sh: try harder to fetch a submodule Stefan Beller
@ 2018-05-15 20:00 ` Stefan Beller
  2018-05-15 20:00 ` [PATCH 2/3] merge-recursive: i18n submodule merge output and respect verbosity Stefan Beller
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15 20:00 UTC (permalink / raw)
  To: git, leif.middelschulte; +Cc: gitster, newren, Stefan Beller

In a later patch we want to improve submodule merging by using the output()
function in merge-recursive.c for submodule merges to deliver a consistent
UI to users.

To do so we could either make the output() function globally available
so we can use it in submodule.c#merge_submodule(), or we could integrate
the submodule merging into the merging code. Choose the later as we
generally want to move submodules closer into the core.

Therefore we move any function related to merging submodules
(merge_submodule(), find_first_merges() and print_commit) to
merge-recursive.c.  We'll keep add_submodule_odb() in submodule.c as it
is used by other submodule functions. While at it, add a TODO note that
we do not really like the function add_submodule_odb().

This commit is best viewed with --color-moved.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 merge-recursive.c | 166 +++++++++++++++++++++++++++++++++++++++++++++
 submodule.c       | 168 +---------------------------------------------
 submodule.h       |   6 +-
 3 files changed, 170 insertions(+), 170 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0c0d48624da..700ba15bf88 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -23,6 +23,7 @@
 #include "merge-recursive.h"
 #include "dir.h"
 #include "submodule.h"
+#include "revision.h"
 
 struct path_hashmap_entry {
 	struct hashmap_entry e;
@@ -977,6 +978,171 @@ static int merge_3way(struct merge_options *o,
 	return merge_status;
 }
 
+static int find_first_merges(struct object_array *result, const char *path,
+		struct commit *a, struct commit *b)
+{
+	int i, j;
+	struct object_array merges = OBJECT_ARRAY_INIT;
+	struct commit *commit;
+	int contains_another;
+
+	char merged_revision[42];
+	const char *rev_args[] = { "rev-list", "--merges", "--ancestry-path",
+				   "--all", merged_revision, NULL };
+	struct rev_info revs;
+	struct setup_revision_opt rev_opts;
+
+	memset(result, 0, sizeof(struct object_array));
+	memset(&rev_opts, 0, sizeof(rev_opts));
+
+	/* get all revisions that merge commit a */
+	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
+			oid_to_hex(&a->object.oid));
+	init_revisions(&revs, NULL);
+	rev_opts.submodule = path;
+	/* FIXME: can't handle linked worktrees in submodules yet */
+	revs.single_worktree = path != NULL;
+	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
+
+	/* save all revisions from the above list that contain b */
+	if (prepare_revision_walk(&revs))
+		die("revision walk setup failed");
+	while ((commit = get_revision(&revs)) != NULL) {
+		struct object *o = &(commit->object);
+		if (in_merge_bases(b, commit))
+			add_object_array(o, NULL, &merges);
+	}
+	reset_revision_walk();
+
+	/* Now we've got all merges that contain a and b. Prune all
+	 * merges that contain another found merge and save them in
+	 * result.
+	 */
+	for (i = 0; i < merges.nr; i++) {
+		struct commit *m1 = (struct commit *) merges.objects[i].item;
+
+		contains_another = 0;
+		for (j = 0; j < merges.nr; j++) {
+			struct commit *m2 = (struct commit *) merges.objects[j].item;
+			if (i != j && in_merge_bases(m2, m1)) {
+				contains_another = 1;
+				break;
+			}
+		}
+
+		if (!contains_another)
+			add_object_array(merges.objects[i].item, NULL, result);
+	}
+
+	object_array_clear(&merges);
+	return result->nr;
+}
+
+static void print_commit(struct commit *commit)
+{
+	struct strbuf sb = STRBUF_INIT;
+	struct pretty_print_context ctx = {0};
+	ctx.date_mode.type = DATE_NORMAL;
+	format_commit_message(commit, " %h: %m %s", &sb, &ctx);
+	fprintf(stderr, "%s\n", sb.buf);
+	strbuf_release(&sb);
+}
+
+#define MERGE_WARNING(path, msg) \
+	warning("Failed to merge submodule %s (%s)", path, msg);
+
+static int merge_submodule(struct object_id *result, const char *path,
+			   const struct object_id *base, const struct object_id *a,
+			   const struct object_id *b, int search)
+{
+	struct commit *commit_base, *commit_a, *commit_b;
+	int parent_count;
+	struct object_array merges;
+
+	int i;
+
+	/* store a in result in case we fail */
+	oidcpy(result, a);
+
+	/* we can not handle deletion conflicts */
+	if (is_null_oid(base))
+		return 0;
+	if (is_null_oid(a))
+		return 0;
+	if (is_null_oid(b))
+		return 0;
+
+	if (add_submodule_odb(path)) {
+		MERGE_WARNING(path, "not checked out");
+		return 0;
+	}
+
+	if (!(commit_base = lookup_commit_reference(base)) ||
+	    !(commit_a = lookup_commit_reference(a)) ||
+	    !(commit_b = lookup_commit_reference(b))) {
+		MERGE_WARNING(path, "commits not present");
+		return 0;
+	}
+
+	/* check whether both changes are forward */
+	if (!in_merge_bases(commit_base, commit_a) ||
+	    !in_merge_bases(commit_base, commit_b)) {
+		MERGE_WARNING(path, "commits don't follow merge-base");
+		return 0;
+	}
+
+	/* Case #1: a is contained in b or vice versa */
+	if (in_merge_bases(commit_a, commit_b)) {
+		oidcpy(result, b);
+		return 1;
+	}
+	if (in_merge_bases(commit_b, commit_a)) {
+		oidcpy(result, a);
+		return 1;
+	}
+
+	/*
+	 * Case #2: There are one or more merges that contain a and b in
+	 * the submodule. If there is only one, then present it as a
+	 * suggestion to the user, but leave it marked unmerged so the
+	 * user needs to confirm the resolution.
+	 */
+
+	/* Skip the search if makes no sense to the calling context.  */
+	if (!search)
+		return 0;
+
+	/* find commit which merges them */
+	parent_count = find_first_merges(&merges, path, commit_a, commit_b);
+	switch (parent_count) {
+	case 0:
+		MERGE_WARNING(path, "merge following commits not found");
+		break;
+
+	case 1:
+		MERGE_WARNING(path, "not fast-forward");
+		fprintf(stderr, "Found a possible merge resolution "
+				"for the submodule:\n");
+		print_commit((struct commit *) merges.objects[0].item);
+		fprintf(stderr,
+			"If this is correct simply add it to the index "
+			"for example\n"
+			"by using:\n\n"
+			"  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
+			"which will accept this suggestion.\n",
+			oid_to_hex(&merges.objects[0].item->oid), path);
+		break;
+
+	default:
+		MERGE_WARNING(path, "multiple merges found");
+		for (i = 0; i < merges.nr; i++)
+			print_commit((struct commit *) merges.objects[i].item);
+	}
+
+	object_array_clear(&merges);
+	return 0;
+}
+
 static int merge_file_1(struct merge_options *o,
 					   const struct diff_filespec *one,
 					   const struct diff_filespec *a,
diff --git a/submodule.c b/submodule.c
index 74d35b25779..654089b3647 100644
--- a/submodule.c
+++ b/submodule.c
@@ -153,7 +153,8 @@ void stage_updated_gitmodules(struct index_state *istate)
 		die(_("staging updated .gitmodules failed"));
 }
 
-static int add_submodule_odb(const char *path)
+/* TODO: remove this function, use repo_submodule_init instead. */
+int add_submodule_odb(const char *path)
 {
 	struct strbuf objects_directory = STRBUF_INIT;
 	int ret = 0;
@@ -1701,171 +1702,6 @@ int submodule_move_head(const char *path,
 	return ret;
 }
 
-static int find_first_merges(struct object_array *result, const char *path,
-		struct commit *a, struct commit *b)
-{
-	int i, j;
-	struct object_array merges = OBJECT_ARRAY_INIT;
-	struct commit *commit;
-	int contains_another;
-
-	char merged_revision[42];
-	const char *rev_args[] = { "rev-list", "--merges", "--ancestry-path",
-				   "--all", merged_revision, NULL };
-	struct rev_info revs;
-	struct setup_revision_opt rev_opts;
-
-	memset(result, 0, sizeof(struct object_array));
-	memset(&rev_opts, 0, sizeof(rev_opts));
-
-	/* get all revisions that merge commit a */
-	xsnprintf(merged_revision, sizeof(merged_revision), "^%s",
-			oid_to_hex(&a->object.oid));
-	init_revisions(&revs, NULL);
-	rev_opts.submodule = path;
-	/* FIXME: can't handle linked worktrees in submodules yet */
-	revs.single_worktree = path != NULL;
-	setup_revisions(ARRAY_SIZE(rev_args)-1, rev_args, &revs, &rev_opts);
-
-	/* save all revisions from the above list that contain b */
-	if (prepare_revision_walk(&revs))
-		die("revision walk setup failed");
-	while ((commit = get_revision(&revs)) != NULL) {
-		struct object *o = &(commit->object);
-		if (in_merge_bases(b, commit))
-			add_object_array(o, NULL, &merges);
-	}
-	reset_revision_walk();
-
-	/* Now we've got all merges that contain a and b. Prune all
-	 * merges that contain another found merge and save them in
-	 * result.
-	 */
-	for (i = 0; i < merges.nr; i++) {
-		struct commit *m1 = (struct commit *) merges.objects[i].item;
-
-		contains_another = 0;
-		for (j = 0; j < merges.nr; j++) {
-			struct commit *m2 = (struct commit *) merges.objects[j].item;
-			if (i != j && in_merge_bases(m2, m1)) {
-				contains_another = 1;
-				break;
-			}
-		}
-
-		if (!contains_another)
-			add_object_array(merges.objects[i].item, NULL, result);
-	}
-
-	object_array_clear(&merges);
-	return result->nr;
-}
-
-static void print_commit(struct commit *commit)
-{
-	struct strbuf sb = STRBUF_INIT;
-	struct pretty_print_context ctx = {0};
-	ctx.date_mode.type = DATE_NORMAL;
-	format_commit_message(commit, " %h: %m %s", &sb, &ctx);
-	fprintf(stderr, "%s\n", sb.buf);
-	strbuf_release(&sb);
-}
-
-#define MERGE_WARNING(path, msg) \
-	warning("Failed to merge submodule %s (%s)", path, msg);
-
-int merge_submodule(struct object_id *result, const char *path,
-		    const struct object_id *base, const struct object_id *a,
-		    const struct object_id *b, int search)
-{
-	struct commit *commit_base, *commit_a, *commit_b;
-	int parent_count;
-	struct object_array merges;
-
-	int i;
-
-	/* store a in result in case we fail */
-	oidcpy(result, a);
-
-	/* we can not handle deletion conflicts */
-	if (is_null_oid(base))
-		return 0;
-	if (is_null_oid(a))
-		return 0;
-	if (is_null_oid(b))
-		return 0;
-
-	if (add_submodule_odb(path)) {
-		MERGE_WARNING(path, "not checked out");
-		return 0;
-	}
-
-	if (!(commit_base = lookup_commit_reference(base)) ||
-	    !(commit_a = lookup_commit_reference(a)) ||
-	    !(commit_b = lookup_commit_reference(b))) {
-		MERGE_WARNING(path, "commits not present");
-		return 0;
-	}
-
-	/* check whether both changes are forward */
-	if (!in_merge_bases(commit_base, commit_a) ||
-	    !in_merge_bases(commit_base, commit_b)) {
-		MERGE_WARNING(path, "commits don't follow merge-base");
-		return 0;
-	}
-
-	/* Case #1: a is contained in b or vice versa */
-	if (in_merge_bases(commit_a, commit_b)) {
-		oidcpy(result, b);
-		return 1;
-	}
-	if (in_merge_bases(commit_b, commit_a)) {
-		oidcpy(result, a);
-		return 1;
-	}
-
-	/*
-	 * Case #2: There are one or more merges that contain a and b in
-	 * the submodule. If there is only one, then present it as a
-	 * suggestion to the user, but leave it marked unmerged so the
-	 * user needs to confirm the resolution.
-	 */
-
-	/* Skip the search if makes no sense to the calling context.  */
-	if (!search)
-		return 0;
-
-	/* find commit which merges them */
-	parent_count = find_first_merges(&merges, path, commit_a, commit_b);
-	switch (parent_count) {
-	case 0:
-		MERGE_WARNING(path, "merge following commits not found");
-		break;
-
-	case 1:
-		MERGE_WARNING(path, "not fast-forward");
-		fprintf(stderr, "Found a possible merge resolution "
-				"for the submodule:\n");
-		print_commit((struct commit *) merges.objects[0].item);
-		fprintf(stderr,
-			"If this is correct simply add it to the index "
-			"for example\n"
-			"by using:\n\n"
-			"  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
-			"which will accept this suggestion.\n",
-			oid_to_hex(&merges.objects[0].item->oid), path);
-		break;
-
-	default:
-		MERGE_WARNING(path, "multiple merges found");
-		for (i = 0; i < merges.nr; i++)
-			print_commit((struct commit *) merges.objects[i].item);
-	}
-
-	object_array_clear(&merges);
-	return 0;
-}
-
 /*
  * Embeds a single submodules git directory into the superprojects git dir,
  * non recursively.
diff --git a/submodule.h b/submodule.h
index e5526f6aaab..b96689ac0db 100644
--- a/submodule.h
+++ b/submodule.h
@@ -89,10 +89,8 @@ extern int submodule_uses_gitfile(const char *path);
 #define SUBMODULE_REMOVAL_IGNORE_UNTRACKED (1<<1)
 #define SUBMODULE_REMOVAL_IGNORE_IGNORED_UNTRACKED (1<<2)
 extern int bad_to_remove_submodule(const char *path, unsigned flags);
-extern int merge_submodule(struct object_id *result, const char *path,
-			   const struct object_id *base,
-			   const struct object_id *a,
-			   const struct object_id *b, int search);
+
+int add_submodule_odb(const char *path);
 
 /* Checks if there are submodule changes in a..b. */
 extern int submodule_touches_in_range(struct object_id *a,
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 18%]

* Re: [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive
  2018-05-15 20:00 [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
                   ` (3 preceding siblings ...)
  2018-05-15 20:00 ` [PATCH 3/3] Inform about fast-forwarding of submodules during merge Stefan Beller
@ 2018-05-15 20:02 ` Stefan Beller
  2018-05-15 20:15 ` Leif Middelschulte
  5 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15 20:02 UTC (permalink / raw)
  To: git, Leif Middelschulte; +Cc: Junio C Hamano, Elijah Newren, Stefan Beller

And I resent two of my earlier patches, please ignore those
(0001-grep-handle-corrupt-index-files-early.patch and
0001-git-submodule.sh-try-harder-to-fetch-a-submodule.patch)

Stefan

^ permalink raw reply	[relevance 8%]

* Re: [PATCH] git-submodule.sh: try harder to fetch a submodule
  2018-05-15 19:07     ` Stefan Beller
  2018-05-15 19:40       ` Stefan Beller
@ 2018-05-15 20:04       ` Jonathan Nieder
  1 sibling, 0 replies; 200+ results
From: Jonathan Nieder @ 2018-05-15 20:04 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Junio C Hamano, git

Stefan Beller wrote:

> I'll resend it with a warning (using say()).

Thanks, makes sense.

> I think we have 2 bugs and this is merely fixing the second bug.

I'm fearing that there are more than two.

[...]
>   $ git init confused-head
>   $ (cd confused-head && git branch test \
>         $(git commit-tree $(git write-tree) -m test))
>   $ git clone --no-checkout  --depth=1 \
>         --separate-git-dir=test.git confused-head/.git test
> Cloning into 'test'...
> warning: --depth is ignored in local clones; use file:// instead.
> done.
>
>   $ git -C test.git config remote.origin.fetch
>   $ echo $?
> 1
>
> (A) Despite the warning of --depth having no impact, the
>   omission thereof changes the repository state.
> (B) There is no remote.origin.fetch configuration, which
>   is weird. See builtin/clone.c:830, that states for this case:

I can reproduce the issue without submodules and without --local,
as follows:

	git init --bare empty.git
	git init --bare almost-empty.git
	git -C ~/src/git push $(pwd)/almost-empty HEAD:refs/heads/upstream

	git clone --single-branch file://$(pwd)/empty.git
	git clone --single-branch file://$(pwd)/almost-empty.git

	git -C almost-empty.git branch -D upstream

	git -C empty fetch
	git -C almost-empty fetch

Expected result:
Both fetches succeed.

Actual result:
First fetch succeeds, second produces
"fatal: Couldn't find remote ref HEAD".

Note that empty.git and almost-empty.git are basically identical.
The difference instead lies in the clones' .git/config files:

diff --git 1/empty/.git/config 2/almost-empty/.git/config
index b51bb0d..ee21198 100644
--- 1/empty/.git/config
+++ 2/almost-empty/.git/config
@@ -4,7 +4,4 @@
        bare = false
        logallrefupdates = true
 [remote "origin"]
-       url = file:///tmp/t/empty.git
-[branch "master"]
-       remote = origin
-       merge = refs/heads/master
+       url = file:///tmp/t/almost-empty.git

Thanks,
Jonathan

^ permalink raw reply	[relevance 7%]

* Re: [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive
  2018-05-15 20:00 [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
                   ` (4 preceding siblings ...)
  2018-05-15 20:02 ` [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive Stefan Beller
@ 2018-05-15 20:15 ` Leif Middelschulte
  2018-05-15 20:49   ` Stefan Beller
  5 siblings, 1 reply; 200+ results
From: Leif Middelschulte @ 2018-05-15 20:15 UTC (permalink / raw)
  To: Stefan Beller, git; +Cc: newren, gitster

Hello Stefan,

thank you once again for your effort.

Am 15. Mai 2018 um 22:00:34, Stefan Beller
(sbeller@google.com(mailto:sbeller@google.com)) schrieb:

> This rerolls the two commits found at [1] with the feedback of Eliah
> and puts Leifs patch[2] on top, that I edited according to Eliahs feedback,
> but kept Leifs ownership.
>
> This has addressed all of Eliahs feedback AFAICT.
> You'll find a branch-diff below[3], which lacks
> the new patch of Leif in that series, but is part of the reroll?
>
> Leif, what do you think?

Seems great to me. Thank you for picking up and improving my changes :)
One Question though: Shouldn’t an enum (like
NOTES_MERGE_VERBOSITY_DEFAULT) be used instead of numbers?


Cheers,


Leif

>
> Thanks,
> Stefan
>
> [1] https://public-inbox.org/git/20180510211917.138518-1-sbeller@google.com/
> [2] https://public-inbox.org/git/20180514205737.21313-2-leif.middelschulte@gmail.com/
> [3] git branch-diff origin/master..origin/sb/submodule-merge-in-merge-recursive origin/master..HEAD >>0000-cover-letter.patch
>
> Leif Middelschulte (1):
> Inform about fast-forwarding of submodules during merge
>
> Stefan Beller (2):
> submodule.c: move submodule merging to merge-recursive.c
> merge-recursive: i18n submodule merge output and respect verbosity
>
> merge-recursive.c | 185 +++++++++++++++++++++++++++++++++++++++++++++-
> submodule.c | 168 +----------------------------------------
> submodule.h | 6 +-
> 3 files changed, 186 insertions(+), 173 deletions(-)
>
> --
> 2.17.0.582.gccdcbd54c44.dirty
>
>
>
> 1: e022c7976ae ! 1: 3b638ccac64 submodule.c: move submodule merging to merge-recursive.c
> @@ -20,7 +20,6 @@
> This commit is best viewed with --color-moved.
>
> Signed-off-by: Stefan Beller
> - Signed-off-by: Junio C Hamano
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> --- a/merge-recursive.c
> 2: 2c02ece7e01 ! 2: eb43110df9d merge-recursive: i18n submodule merge output and respect verbosity
> @@ -7,7 +7,6 @@
> internationalisation as well as the verbosity setting.
>
> Signed-off-by: Stefan Beller
> - Signed-off-by: Junio C Hamano
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> --- a/merge-recursive.c
> @@ -73,10 +72,10 @@
> - fprintf(stderr, "Found a possible merge resolution "
> - "for the submodule:\n");
> + output(o, 1, _("Failed to merge submodule %s (not fast-forward)"), path);
> -+ output(o, 1, _("Found a possible merge resolution for the submodule:\n"));
> ++ output(o, 2, _("Found a possible merge resolution for the submodule:\n"));
> print_commit((struct commit *) merges.objects[0].item);
> - fprintf(stderr,
> -+ output(o, 1, _(
> ++ output(o, 2, _(
> "If this is correct simply add it to the index "
> "for example\n"
> "by using:\n\n"
> -: ----------- > 3: 4a3bc435023 Inform about fast-forwarding of submodules during merge

^ permalink raw reply	[relevance 5%]

* Re: [PATCHv2 0/3] Reroll of sb/submodule-merge-in-merge-recursive
  2018-05-15 20:15 ` Leif Middelschulte
@ 2018-05-15 20:49   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15 20:49 UTC (permalink / raw)
  To: Leif Middelschulte; +Cc: git, Elijah Newren, Junio C Hamano

On Tue, May 15, 2018 at 1:15 PM, Leif Middelschulte
<leif.middelschulte@gmail.com> wrote:
> Hello Stefan,
>
> thank you once again for your effort.
>
> Am 15. Mai 2018 um 22:00:34, Stefan Beller
> (sbeller@google.com(mailto:sbeller@google.com)) schrieb:
>
>> This rerolls the two commits found at [1] with the feedback of Eliah
>> and puts Leifs patch[2] on top, that I edited according to Eliahs feedback,
>> but kept Leifs ownership.
>>
>> This has addressed all of Eliahs feedback AFAICT.
>> You'll find a branch-diff below[3], which lacks
>> the new patch of Leif in that series, but is part of the reroll?
>>
>> Leif, what do you think?
>
> Seems great to me. Thank you for picking up and improving my changes :)
> One Question though: Shouldn’t an enum (like
> NOTES_MERGE_VERBOSITY_DEFAULT) be used instead of numbers?

Hah! I did not know that existed.

$ git grep NOTES_MERGE_VERBOSITY_DEFAULT
builtin/notes.c:810:    o.verbosity = verbosity + NOTES_MERGE_VERBOSITY_DEFAULT;
notes-merge.c:22:       o->verbosity = NOTES_MERGE_VERBOSITY_DEFAULT;
notes-merge.h:9:        NOTES_MERGE_VERBOSITY_DEFAULT = 2,

It doesn't seem to be used much, as opposed to numbers:

$ git grep show -- merge-recursive.c
merge-recursive.c:201:static int show(struct merge_options *o, int v)
merge-recursive.c:211:  if (!show(o, v))
merge-recursive.c:570:  opts.show_rename_progress = o->show_rename_progress;
merge-recursive.c:1096:         if (show(o, 3)) {
merge-recursive.c:1099:         } else if (show(o, 2))
merge-recursive.c:1108:         if (show(o, 3)) {
merge-recursive.c:1111:         } else if (show(o, 2))
merge-recursive.c:2178:         if (show(o, 4) || o->call_depth)
merge-recursive.c:2275: if (show(o, 4)) {
merge-recursive.c:2286: if (show(o, 5)) {
merge-recursive.c:2351: if (show(o, 2))

(The first two are the implementation of show/output, third is
somewhat unrelated to show() and all the rest is numbers).

If we'd want to use  NOTES_MERGE_VERBOSITY_DEFAULT,
I would suggest to send a followup series on top of this?

I would think numbers are fine for now.

Thanks,
Stefan

^ permalink raw reply	[relevance 4%]

* [RFC PATCH 00/19] object store: grafts and shallow.
@ 2018-05-15 23:42 Stefan Beller
  2018-05-15 23:42 ` [PATCH 01/19] object-store: move object access functions to object-store.h Stefan Beller
  2018-05-17 22:51 ` [PATCH 00/19] object store: grafts and shallow Stefan Beller
  0 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2018-05-15 23:42 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

This applies on top of sb/object-store-alloc[1], and is the next part of the
object store series. 

I think we're getting close to actually being done in the object store series,
as the next series to build on top of this will convert the
lookup_{object, commit, ...} functions.

However I marked this series as RFC, as I expect heavy conflicts with the
code base. sb/object-store-alloc builds on an older base of the code base
and there have been some series that will conflict with this one.
For example the patch to migrate path functions into a repository world
will collide with one of Dschos series as he added another path helper
function there. I am happy to reroll on top of that, but for now I chose
sb/object-store-alloc to keep the momentum in the object-store series'.

There is another object store series that is not part of the critical path:
There is a mem leak in the pack files stored in the raw object store.
Upon free'ing the repository we do not free its pack files. We cannot
free the packfiles, as otherwise the bitmap code would have dangling
pointers into packfiles that no longer exists, leading to segfaults.
So we'll need to have an object store series covering the bitmap code.
I have something local, which I'll send out shortly.

Thanks,
Stefan

[1] with the latest patch replaced as in
    https://public-inbox.org/git/20180515214842.108713-1-sbeller@google.com/

Brandon Williams (3):
  commit: convert commit_graft_pos() to handle arbitrary repositories
  commit: convert register_commit_graft to handle arbitrary repositories
  commit: convert read_graft_file to handle arbitrary repositories

Jonathan Nieder (6):
  object: move grafts to object parser
  commit: add repository argument to commit_graft_pos
  commit: add repository argument to register_commit_graft
  commit: add repository argument to read_graft_file
  commit: add repository argument to prepare_commit_graft
  commit: add repository argument to lookup_commit_graft

Stefan Beller (10):
  object-store: move object access functions to object-store.h
  shallow: add repository argument to set_alternate_shallow_file
  shallow: add repository argument to register_shallow
  shallow: add repository argument to check_shallow_file_for_update
  shallow: add repository argument to is_repository_shallow
  cache: convert get_graft_file to handle arbitrary repositories
  path.c: migrate git_path_ to take a repository argument
  shallow: migrate shallow information into the object parser
  commit: allow prepare_commit_graft to handle arbitrary repositories
  commit: allow lookup_commit_graft to handle arbitrary repositories

 apply.c                  |   1 +
 archive-tar.c            |   1 +
 archive-zip.c            |   1 +
 archive.c                |   1 +
 blame.c                  |   9 ++-
 branch.c                 |  14 ++---
 builtin/blame.c          |   4 +-
 builtin/cat-file.c       |   1 +
 builtin/checkout.c       |   1 +
 builtin/clone.c          |   1 +
 builtin/commit-tree.c    |   1 +
 builtin/commit.c         |  38 ++++++-------
 builtin/describe.c       |   1 +
 builtin/difftool.c       |   1 +
 builtin/fast-export.c    |   1 +
 builtin/fetch.c          |   7 ++-
 builtin/fmt-merge-msg.c  |   1 +
 builtin/hash-object.c    |   1 +
 builtin/log.c            |   1 +
 builtin/ls-tree.c        |   1 +
 builtin/merge-tree.c     |   1 +
 builtin/merge.c          |  37 ++++++------
 builtin/mktag.c          |   1 +
 builtin/mktree.c         |   1 +
 builtin/notes.c          |   1 +
 builtin/pack-objects.c   |   6 +-
 builtin/prune.c          |   3 +-
 builtin/pull.c           |   4 +-
 builtin/receive-pack.c   |   3 +-
 builtin/reflog.c         |   1 +
 builtin/remote.c         |   1 +
 builtin/reset.c          |   2 +-
 builtin/rev-list.c       |   1 +
 builtin/rev-parse.c      |   3 +-
 builtin/show-ref.c       |   1 +
 builtin/tag.c            |   1 +
 builtin/unpack-file.c    |   1 +
 builtin/unpack-objects.c |   1 +
 builtin/verify-commit.c  |   1 +
 bulk-checkin.c           |   1 +
 bundle.c                 |   1 +
 cache-tree.c             |   1 +
 cache.h                  | 119 +--------------------------------------
 combine-diff.c           |   1 +
 commit.c                 |  77 +++++++++++++------------
 commit.h                 |  10 ++--
 config.c                 |   1 +
 convert.c                |   1 +
 diff.c                   |   1 +
 diffcore-rename.c        |   1 +
 dir.c                    |   1 +
 entry.c                  |   1 +
 environment.c            |   8 +--
 fetch-pack.c             |   9 +--
 fsck.c                   |   3 +-
 git.c                    |   2 +-
 grep.c                   |   1 +
 list-objects-filter.c    |   1 +
 list-objects.c           |   1 +
 log-tree.c               |   1 +
 mailmap.c                |   1 +
 match-trees.c            |   1 +
 merge-blobs.c            |   1 +
 merge-recursive.c        |   1 +
 notes-cache.c            |   1 +
 notes-merge.c            |   1 +
 notes.c                  |   1 +
 object-store.h           | 117 ++++++++++++++++++++++++++++++++++++++
 object.c                 |   4 ++
 object.h                 |  10 ++++
 pack-bitmap-write.c      |   1 +
 packfile.h               |   5 ++
 path.c                   |  18 +++---
 path.h                   |  40 ++++++++++---
 read-cache.c             |   1 +
 ref-filter.c             |   1 +
 refs.c                   |   1 +
 remote-testsvn.c         |   1 +
 remote.c                 |   1 +
 repository.h             |   5 ++
 rerere.c                 |   8 ++-
 revision.c               |   1 +
 send-pack.c              |   7 ++-
 sequencer.c              |  38 +++++++------
 shallow.c                |  74 ++++++++++++------------
 submodule-config.c       |   1 +
 tag.c                    |   1 +
 tree-walk.c              |   1 +
 tree.c                   |   1 +
 unpack-trees.c           |   1 +
 upload-pack.c            |  10 ++--
 walker.c                 |   1 +
 wt-status.c              |   8 +--
 xdiff-interface.c        |   1 +
 94 files changed, 450 insertions(+), 314 deletions(-)

-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 4%]

* [PATCH 01/19] object-store: move object access functions to object-store.h
  2018-05-15 23:42 [RFC PATCH 00/19] object store: grafts and shallow Stefan Beller
@ 2018-05-15 23:42 ` Stefan Beller
  2018-05-17 22:51 ` [PATCH 00/19] object store: grafts and shallow Stefan Beller
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-15 23:42 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

This should make these functions easier to find and cache.h less
overwhelming to read.

In particular, this moves:
- read_object_file
- oid_object_info
- write_object_file

As a result, most of the codebase needs to #include object-store.h.
In this patch the #include is only added to files that would fail to
compile otherwise.  It would be better to #include wherever
identifiers from the header are used.  That can happen later
when we have better tooling for it.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 apply.c                  |   1 +
 archive-tar.c            |   1 +
 archive-zip.c            |   1 +
 archive.c                |   1 +
 blame.c                  |   1 +
 builtin/blame.c          |   1 +
 builtin/cat-file.c       |   1 +
 builtin/checkout.c       |   1 +
 builtin/clone.c          |   1 +
 builtin/commit-tree.c    |   1 +
 builtin/describe.c       |   1 +
 builtin/difftool.c       |   1 +
 builtin/fast-export.c    |   1 +
 builtin/fetch.c          |   1 +
 builtin/fmt-merge-msg.c  |   1 +
 builtin/hash-object.c    |   1 +
 builtin/log.c            |   1 +
 builtin/ls-tree.c        |   1 +
 builtin/merge-tree.c     |   1 +
 builtin/mktag.c          |   1 +
 builtin/mktree.c         |   1 +
 builtin/notes.c          |   1 +
 builtin/prune.c          |   1 +
 builtin/receive-pack.c   |   1 +
 builtin/reflog.c         |   1 +
 builtin/remote.c         |   1 +
 builtin/rev-list.c       |   1 +
 builtin/show-ref.c       |   1 +
 builtin/tag.c            |   1 +
 builtin/unpack-file.c    |   1 +
 builtin/unpack-objects.c |   1 +
 builtin/verify-commit.c  |   1 +
 bulk-checkin.c           |   1 +
 bundle.c                 |   1 +
 cache-tree.c             |   1 +
 cache.h                  | 117 ---------------------------------------
 combine-diff.c           |   1 +
 commit.c                 |   1 +
 config.c                 |   1 +
 convert.c                |   1 +
 diff.c                   |   1 +
 diffcore-rename.c        |   1 +
 dir.c                    |   1 +
 entry.c                  |   1 +
 fetch-pack.c             |   1 +
 fsck.c                   |   1 +
 grep.c                   |   1 +
 list-objects-filter.c    |   1 +
 list-objects.c           |   1 +
 log-tree.c               |   1 +
 mailmap.c                |   1 +
 match-trees.c            |   1 +
 merge-blobs.c            |   1 +
 merge-recursive.c        |   1 +
 notes-cache.c            |   1 +
 notes-merge.c            |   1 +
 notes.c                  |   1 +
 object-store.h           | 117 +++++++++++++++++++++++++++++++++++++++
 object.c                 |   1 +
 pack-bitmap-write.c      |   1 +
 packfile.h               |   5 ++
 read-cache.c             |   1 +
 ref-filter.c             |   1 +
 refs.c                   |   1 +
 remote-testsvn.c         |   1 +
 remote.c                 |   1 +
 rerere.c                 |   1 +
 revision.c               |   1 +
 send-pack.c              |   1 +
 sequencer.c              |   1 +
 shallow.c                |   1 +
 submodule-config.c       |   1 +
 tag.c                    |   1 +
 tree-walk.c              |   1 +
 tree.c                   |   1 +
 unpack-trees.c           |   1 +
 upload-pack.c            |   1 +
 walker.c                 |   1 +
 xdiff-interface.c        |   1 +
 79 files changed, 198 insertions(+), 117 deletions(-)

diff --git a/apply.c b/apply.c
index 7e5792c996f..cbc45fa1b0e 100644
--- a/apply.c
+++ b/apply.c
@@ -9,6 +9,7 @@
 
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "blob.h"
 #include "delta.h"
 #include "diff.h"
diff --git a/archive-tar.c b/archive-tar.c
index f93409324f9..e38435eb4ef 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -5,6 +5,7 @@
 #include "config.h"
 #include "tar.h"
 #include "archive.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "run-command.h"
 
diff --git a/archive-zip.c b/archive-zip.c
index 74f3fe91034..abc556e5a75 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -6,6 +6,7 @@
 #include "archive.h"
 #include "streaming.h"
 #include "utf8.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
 
diff --git a/archive.c b/archive.c
index 93ab175b0b4..9da1e3664a6 100644
--- a/archive.c
+++ b/archive.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree-walk.h"
 #include "attr.h"
diff --git a/blame.c b/blame.c
index 3a11f1ce52b..f689bde31cd 100644
--- a/blame.c
+++ b/blame.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "cache-tree.h"
 #include "mergesort.h"
 #include "diff.h"
diff --git a/builtin/blame.c b/builtin/blame.c
index bfdf7cc1325..0ffd1d443ea 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -22,6 +22,7 @@
 #include "line-log.h"
 #include "dir.h"
 #include "progress.h"
+#include "object-store.h"
 #include "blame.h"
 
 static char blame_usage[] = N_("git blame [<options>] [<rev-opts>] [<rev>] [--] <file>");
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index b8ecbea98e9..91e7764243e 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -13,6 +13,7 @@
 #include "tree-walk.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "object-store.h"
 
 struct batch_options {
 	int enabled;
diff --git a/builtin/checkout.c b/builtin/checkout.c
index b49b5820718..105e07981ff 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -4,6 +4,7 @@
 #include "lockfile.h"
 #include "parse-options.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/builtin/clone.c b/builtin/clone.c
index 7df5932b855..29998c02ece 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -14,6 +14,7 @@
 #include "parse-options.h"
 #include "fetch-pack.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tree.h"
 #include "tree-walk.h"
 #include "unpack-trees.h"
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index ecf42191da1..9fbd3529fb1 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -5,6 +5,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "builtin.h"
diff --git a/builtin/describe.c b/builtin/describe.c
index 66c497f7896..65b0edc473c 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -13,6 +13,7 @@
 #include "hashmap.h"
 #include "argv-array.h"
 #include "run-command.h"
+#include "object-store.h"
 #include "revision.h"
 #include "list-objects.h"
 
diff --git a/builtin/difftool.c b/builtin/difftool.c
index ee8dce019e1..df7e75f797b 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -20,6 +20,7 @@
 #include "argv-array.h"
 #include "strbuf.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "dir.h"
 
 static char *diff_gui_tool;
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 373c794873e..f593e57b9d4 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "tag.h"
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 73be393b2ea..c1f2df97965 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -5,6 +5,7 @@
 #include "config.h"
 #include "repository.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "builtin.h"
 #include "string-list.h"
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index bd680be6874..1b526adb3a9 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index 526da5c1856..d5c018eabd5 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -6,6 +6,7 @@
  */
 #include "builtin.h"
 #include "config.h"
+#include "object-store.h"
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/log.c b/builtin/log.c
index 71f68a3e4f5..9656578f58e 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "color.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index 409da4e8351..fe3b952cb30 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -5,6 +5,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 32736e0b101..1b702d44c9f 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "object-store.h"
 #include "blob.h"
 #include "exec_cmd.h"
 #include "merge-blobs.h"
diff --git a/builtin/mktag.c b/builtin/mktag.c
index 82a6e860775..6fb7dc8578d 100644
--- a/builtin/mktag.c
+++ b/builtin/mktag.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "tag.h"
 #include "replace-object.h"
+#include "object-store.h"
 
 /*
  * A signature file has a very simple fixed format: four lines
diff --git a/builtin/mktree.c b/builtin/mktree.c
index bb76b469fd1..2dc4ad6ba8f 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -7,6 +7,7 @@
 #include "quote.h"
 #include "tree.h"
 #include "parse-options.h"
+#include "object-store.h"
 
 static struct treeent {
 	unsigned mode;
diff --git a/builtin/notes.c b/builtin/notes.c
index 921e08d5bf5..73b680ee13a 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -11,6 +11,7 @@
 #include "config.h"
 #include "builtin.h"
 #include "notes.h"
+#include "object-store.h"
 #include "blob.h"
 #include "pretty.h"
 #include "refs.h"
diff --git a/builtin/prune.c b/builtin/prune.c
index 518ffbea139..8cc8659612f 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -6,6 +6,7 @@
 #include "reachable.h"
 #include "parse-options.h"
 #include "progress.h"
+#include "object-store.h"
 
 static const char * const prune_usage[] = {
 	N_("git prune [-n] [-v] [--progress] [--expire <time>] [--] [<head>...]"),
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index c4272fbc96d..36906fd5e98 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -25,6 +25,7 @@
 #include "tmp-objdir.h"
 #include "oidset.h"
 #include "packfile.h"
+#include "object-store.h"
 #include "protocol.h"
 
 static const char * const receive_pack_usage[] = {
diff --git a/builtin/reflog.c b/builtin/reflog.c
index a89bd1dd252..bee4795a936 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "config.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "commit.h"
 #include "refs.h"
 #include "dir.h"
diff --git a/builtin/remote.c b/builtin/remote.c
index 805ffc05cdb..9487c019092 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -7,6 +7,7 @@
 #include "strbuf.h"
 #include "run-command.h"
 #include "refs.h"
+#include "object-store.h"
 #include "argv-array.h"
 
 static const char * const builtin_remote_usage[] = {
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index fadd3ec14cb..e9bd4e378ad 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -6,6 +6,7 @@
 #include "list-objects.h"
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-bitmap.h"
 #include "builtin.h"
diff --git a/builtin/show-ref.c b/builtin/show-ref.c
index f2eb1a77240..2f13f1316fa 100644
--- a/builtin/show-ref.c
+++ b/builtin/show-ref.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "string-list.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 26d7729f57b..f872f14f65e 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -10,6 +10,7 @@
 #include "config.h"
 #include "builtin.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tag.h"
 #include "run-command.h"
 #include "parse-options.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index 300eb59657e..58652229f27 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "config.h"
+#include "object-store.h"
 
 static char *create_temp_file(struct object_id *oid)
 {
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index cfe9019f800..ded798b72ff 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "object.h"
 #include "delta.h"
 #include "pack.h"
diff --git a/builtin/verify-commit.c b/builtin/verify-commit.c
index dcdaada1110..f6922da16d6 100644
--- a/builtin/verify-commit.c
+++ b/builtin/verify-commit.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "config.h"
 #include "builtin.h"
+#include "object-store.h"
 #include "commit.h"
 #include "run-command.h"
 #include <signal.h>
diff --git a/bulk-checkin.c b/bulk-checkin.c
index de1f4040c78..918d58a7ebf 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -8,6 +8,7 @@
 #include "pack.h"
 #include "strbuf.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static struct bulk_checkin_state {
 	unsigned plugged:1;
diff --git a/bundle.c b/bundle.c
index 902c9b54485..00ada05f46c 100644
--- a/bundle.c
+++ b/bundle.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "lockfile.h"
 #include "bundle.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/cache-tree.c b/cache-tree.c
index 6a555f4d431..1c338c41f3a 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -3,6 +3,7 @@
 #include "tree.h"
 #include "tree-walk.h"
 #include "cache-tree.h"
+#include "object-store.h"
 
 #ifndef DEBUG
 #define DEBUG 0
diff --git a/cache.h b/cache.h
index c75559b7d38..ab716011b7e 100644
--- a/cache.h
+++ b/cache.h
@@ -1183,32 +1183,6 @@ extern char *xdg_config_home(const char *filename);
  */
 extern char *xdg_cache_home(const char *filename);
 
-extern void *read_object_file_extended(const struct object_id *oid,
-				       enum object_type *type,
-				       unsigned long *size, int lookup_replace);
-static inline void *read_object_file(const struct object_id *oid, enum object_type *type, unsigned long *size)
-{
-	return read_object_file_extended(oid, type, size, 1);
-}
-
-/* Read and unpack an object file into memory, write memory to an object file */
-int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
-
-extern int hash_object_file(const void *buf, unsigned long len,
-			    const char *type, struct object_id *oid);
-
-extern int write_object_file(const void *buf, unsigned long len,
-			     const char *type, struct object_id *oid);
-
-extern int hash_object_file_literally(const void *buf, unsigned long len,
-				      const char *type, struct object_id *oid,
-				      unsigned flags);
-
-extern int pretend_object_file(void *, unsigned long, enum object_type,
-			       struct object_id *oid);
-
-extern int force_object_loose(const struct object_id *oid, time_t mtime);
-
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
@@ -1218,43 +1192,6 @@ extern int check_object_signature(const struct object_id *oid, void *buf, unsign
 
 extern int finalize_object_file(const char *tmpfile, const char *filename);
 
-/*
- * Open the loose object at path, check its hash, and return the contents,
- * type, and size. If the object is a blob, then "contents" may return NULL,
- * to allow streaming of large blobs.
- *
- * Returns 0 on success, negative on error (details may be written to stderr).
- */
-int read_loose_object(const char *path,
-		      const struct object_id *expected_oid,
-		      enum object_type *type,
-		      unsigned long *size,
-		      void **contents);
-
-/*
- * Convenience for sha1_object_info_extended() with a NULL struct
- * object_info. OBJECT_INFO_SKIP_CACHED is automatically set; pass
- * nonzero flags to also set other flags.
- */
-extern int has_sha1_file_with_flags(const unsigned char *sha1, int flags);
-static inline int has_sha1_file(const unsigned char *sha1)
-{
-	return has_sha1_file_with_flags(sha1, 0);
-}
-
-/* Same as the above, except for struct object_id. */
-extern int has_object_file(const struct object_id *oid);
-extern int has_object_file_with_flags(const struct object_id *oid, int flags);
-
-/*
- * Return true iff an alternate object database has a loose object
- * with the specified name.  This function does not respect replace
- * references.
- */
-extern int has_loose_object_nonlocal(const unsigned char *sha1);
-
-extern void assert_oid_type(const struct object_id *oid, enum object_type expect);
-
 /* Helper to check and "touch" a file */
 extern int check_and_freshen_file(const char *fn, int freshen);
 
@@ -1624,60 +1561,6 @@ int for_each_loose_file_in_objdir_buf(struct strbuf *path,
 #define FOR_EACH_OBJECT_LOCAL_ONLY 0x1
 extern int for_each_loose_object(each_loose_object_fn, void *, unsigned flags);
 
-struct object_info {
-	/* Request */
-	enum object_type *typep;
-	unsigned long *sizep;
-	off_t *disk_sizep;
-	unsigned char *delta_base_sha1;
-	struct strbuf *type_name;
-	void **contentp;
-
-	/* Response */
-	enum {
-		OI_CACHED,
-		OI_LOOSE,
-		OI_PACKED,
-		OI_DBCACHED
-	} whence;
-	union {
-		/*
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } cached;
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } loose;
-		 */
-		struct {
-			struct packed_git *pack;
-			off_t offset;
-			unsigned int is_delta;
-		} packed;
-	} u;
-};
-
-/*
- * Initializer for a "struct object_info" that wants no items. You may
- * also memset() the memory to all-zeroes.
- */
-#define OBJECT_INFO_INIT {NULL}
-
-/* Invoke lookup_replace_object() on the given hash */
-#define OBJECT_INFO_LOOKUP_REPLACE 1
-/* Allow reading from a loose object file of unknown/bogus type */
-#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
-/* Do not check cached storage */
-#define OBJECT_INFO_SKIP_CACHED 4
-/* Do not retry packed storage after checking packed and loose storage */
-#define OBJECT_INFO_QUICK 8
-/* Do not check loose object */
-#define OBJECT_INFO_IGNORE_LOOSE 16
-
-int oid_object_info_extended(struct repository *r,
-			     const struct object_id *,
-			     struct object_info *, unsigned flags);
-
 /*
  * Set this to 0 to prevent sha1_object_info_extended() from fetching missing
  * blobs. This has a difference only if extensions.partialClone is set.
diff --git a/combine-diff.c b/combine-diff.c
index 2ef495963fc..de7695e7282 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "blob.h"
 #include "diff.h"
diff --git a/commit.c b/commit.c
index 5eb4d2f08f8..b053f07f305 100644
--- a/commit.c
+++ b/commit.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "tag.h"
 #include "commit.h"
+#include "object-store.h"
 #include "pkt-line.h"
 #include "utf8.h"
 #include "diff.h"
diff --git a/config.c b/config.c
index c698988f5e1..041db3ce7d4 100644
--- a/config.c
+++ b/config.c
@@ -14,6 +14,7 @@
 #include "quote.h"
 #include "hashmap.h"
 #include "string-list.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "dir.h"
 
diff --git a/convert.c b/convert.c
index c480097a2a0..7f5afd6dd66 100644
--- a/convert.c
+++ b/convert.c
@@ -1,6 +1,7 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "attr.h"
 #include "run-command.h"
 #include "quote.h"
diff --git a/diff.c b/diff.c
index 4753170fe12..18d478c5190 100644
--- a/diff.c
+++ b/diff.c
@@ -13,6 +13,7 @@
 #include "attr.h"
 #include "run-command.h"
 #include "utf8.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "submodule-config.h"
 #include "submodule.h"
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 0b7e4989a87..d775183c2fd 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -4,6 +4,7 @@
 #include "cache.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "object-store.h"
 #include "hashmap.h"
 #include "progress.h"
 
diff --git a/dir.c b/dir.c
index 63a917be45d..9e7c90cf50b 100644
--- a/dir.c
+++ b/dir.c
@@ -11,6 +11,7 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "object-store.h"
 #include "attr.h"
 #include "refs.h"
 #include "wildmatch.h"
diff --git a/entry.c b/entry.c
index 2101201a111..b5d1d3cf231 100644
--- a/entry.c
+++ b/entry.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "blob.h"
+#include "object-store.h"
 #include "dir.h"
 #include "streaming.h"
 #include "submodule.h"
diff --git a/fetch-pack.c b/fetch-pack.c
index adc1b68dd3a..a1535b37b9b 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -19,6 +19,7 @@
 #include "sha1-array.h"
 #include "oidset.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static int transfer_unpack_limit = -1;
 static int fetch_unpack_limit = -1;
diff --git a/fsck.c b/fsck.c
index 9218c2a643b..59b0c7d640e 100644
--- a/fsck.c
+++ b/fsck.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "object.h"
 #include "blob.h"
 #include "tree.h"
diff --git a/grep.c b/grep.c
index 65b90c10a38..438987220ea 100644
--- a/grep.c
+++ b/grep.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "grep.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
 #include "diff.h"
diff --git a/list-objects-filter.c b/list-objects-filter.c
index ea94fe8af28..44b3a49bdb3 100644
--- a/list-objects-filter.c
+++ b/list-objects-filter.c
@@ -11,6 +11,7 @@
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
 #include "oidset.h"
+#include "object-store.h"
 
 /* Remember to update object flag allocation in object.h */
 /*
diff --git a/list-objects.c b/list-objects.c
index 168bef688a8..7ae0eb8ebc7 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -10,6 +10,7 @@
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static void process_blob(struct rev_info *revs,
 			 struct blob *blob,
diff --git a/log-tree.c b/log-tree.c
index d1c0bedf244..5660ee7c785 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "diff.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "graph.h"
diff --git a/mailmap.c b/mailmap.c
index 13f0d2884e2..962fd86d6d7 100644
--- a/mailmap.c
+++ b/mailmap.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "string-list.h"
 #include "mailmap.h"
+#include "object-store.h"
 
 #define DEBUG_MAILMAP 0
 #if DEBUG_MAILMAP
diff --git a/match-trees.c b/match-trees.c
index 72cc2baa3f9..4cdeff53e1e 100644
--- a/match-trees.c
+++ b/match-trees.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "tree.h"
 #include "tree-walk.h"
+#include "object-store.h"
 
 static int score_missing(unsigned mode, const char *path)
 {
diff --git a/merge-blobs.c b/merge-blobs.c
index fa49c17287f..fabb8c19ce9 100644
--- a/merge-blobs.c
+++ b/merge-blobs.c
@@ -4,6 +4,7 @@
 #include "ll-merge.h"
 #include "blob.h"
 #include "merge-blobs.h"
+#include "object-store.h"
 
 static int fill_mmfile_blob(mmfile_t *f, struct blob *obj)
 {
diff --git a/merge-recursive.c b/merge-recursive.c
index cbded673c28..455f59d21ac 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -8,6 +8,7 @@
 #include "advice.h"
 #include "lockfile.h"
 #include "cache-tree.h"
+#include "object-store.h"
 #include "commit.h"
 #include "blob.h"
 #include "builtin.h"
diff --git a/notes-cache.c b/notes-cache.c
index e61988e503b..d5770031776 100644
--- a/notes-cache.c
+++ b/notes-cache.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "notes-cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "refs.h"
 
diff --git a/notes-merge.c b/notes-merge.c
index 8e0726a9418..fd96c5ab3d6 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "commit.h"
 #include "refs.h"
+#include "object-store.h"
 #include "diff.h"
 #include "diffcore.h"
 #include "xdiff-interface.h"
diff --git a/notes.c b/notes.c
index a386d450c4c..32d3dbcc1e7 100644
--- a/notes.c
+++ b/notes.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "notes.h"
+#include "object-store.h"
 #include "blob.h"
 #include "tree.h"
 #include "utf8.h"
diff --git a/object-store.h b/object-store.h
index 1ff862c7f93..695266891bf 100644
--- a/object-store.h
+++ b/object-store.h
@@ -137,4 +137,121 @@ void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned cha
 
 void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
+extern void *read_object_file_extended(const struct object_id *oid,
+				       enum object_type *type,
+				       unsigned long *size, int lookup_replace);
+static inline void *read_object_file(const struct object_id *oid, enum object_type *type, unsigned long *size)
+{
+	return read_object_file_extended(oid, type, size, 1);
+}
+
+/* Read and unpack an object file into memory, write memory to an object file */
+int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
+
+extern int hash_object_file(const void *buf, unsigned long len,
+			    const char *type, struct object_id *oid);
+
+extern int write_object_file(const void *buf, unsigned long len,
+			     const char *type, struct object_id *oid);
+
+extern int hash_object_file_literally(const void *buf, unsigned long len,
+				      const char *type, struct object_id *oid,
+				      unsigned flags);
+
+extern int pretend_object_file(void *, unsigned long, enum object_type,
+			       struct object_id *oid);
+
+extern int force_object_loose(const struct object_id *oid, time_t mtime);
+
+/*
+ * Open the loose object at path, check its hash, and return the contents,
+ * type, and size. If the object is a blob, then "contents" may return NULL,
+ * to allow streaming of large blobs.
+ *
+ * Returns 0 on success, negative on error (details may be written to stderr).
+ */
+int read_loose_object(const char *path,
+		      const struct object_id *expected_oid,
+		      enum object_type *type,
+		      unsigned long *size,
+		      void **contents);
+
+/*
+ * Convenience for sha1_object_info_extended() with a NULL struct
+ * object_info. OBJECT_INFO_SKIP_CACHED is automatically set; pass
+ * nonzero flags to also set other flags.
+ */
+extern int has_sha1_file_with_flags(const unsigned char *sha1, int flags);
+static inline int has_sha1_file(const unsigned char *sha1)
+{
+	return has_sha1_file_with_flags(sha1, 0);
+}
+
+/* Same as the above, except for struct object_id. */
+extern int has_object_file(const struct object_id *oid);
+extern int has_object_file_with_flags(const struct object_id *oid, int flags);
+
+/*
+ * Return true iff an alternate object database has a loose object
+ * with the specified name.  This function does not respect replace
+ * references.
+ */
+extern int has_loose_object_nonlocal(const unsigned char *sha1);
+
+extern void assert_oid_type(const struct object_id *oid, enum object_type expect);
+
+struct object_info {
+	/* Request */
+	enum object_type *typep;
+	unsigned long *sizep;
+	off_t *disk_sizep;
+	unsigned char *delta_base_sha1;
+	struct strbuf *type_name;
+	void **contentp;
+
+	/* Response */
+	enum {
+		OI_CACHED,
+		OI_LOOSE,
+		OI_PACKED,
+		OI_DBCACHED
+	} whence;
+	union {
+		/*
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } cached;
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } loose;
+		 */
+		struct {
+			struct packed_git *pack;
+			off_t offset;
+			unsigned int is_delta;
+		} packed;
+	} u;
+};
+
+/*
+ * Initializer for a "struct object_info" that wants no items. You may
+ * also memset() the memory to all-zeroes.
+ */
+#define OBJECT_INFO_INIT {NULL}
+
+/* Invoke lookup_replace_object() on the given hash */
+#define OBJECT_INFO_LOOKUP_REPLACE 1
+/* Allow reading from a loose object file of unknown/bogus type */
+#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
+/* Do not check cached storage */
+#define OBJECT_INFO_SKIP_CACHED 4
+/* Do not retry packed storage after checking packed and loose storage */
+#define OBJECT_INFO_QUICK 8
+/* Do not check loose object */
+#define OBJECT_INFO_IGNORE_LOOSE 16
+
+int oid_object_info_extended(struct repository *r,
+			     const struct object_id *,
+			     struct object_info *, unsigned flags);
+
 #endif /* OBJECT_STORE_H */
diff --git a/object.c b/object.c
index 8e29f63bf23..0116ed6529a 100644
--- a/object.c
+++ b/object.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "object.h"
 #include "replace-object.h"
+#include "object-store.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c
index cd1903e717a..f7fac9d0f24 100644
--- a/pack-bitmap-write.c
+++ b/pack-bitmap-write.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "diff.h"
diff --git a/packfile.h b/packfile.h
index fdfddb89b53..1abbc2c07c8 100644
--- a/packfile.h
+++ b/packfile.h
@@ -3,6 +3,11 @@
 
 #include "oidset.h"
 
+/* in object-store.h */
+struct packed_git;
+struct object_info;
+enum object_type;
+
 /*
  * Generate the filename to be used for a pack file with checksum "sha1" and
  * extension "ext". The result is written into the strbuf "buf", overwriting
diff --git a/read-cache.c b/read-cache.c
index 10f1c6bb8a3..7f10d9ffff9 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -11,6 +11,7 @@
 #include "cache-tree.h"
 #include "refs.h"
 #include "dir.h"
+#include "object-store.h"
 #include "tree.h"
 #include "commit.h"
 #include "blob.h"
diff --git a/ref-filter.c b/ref-filter.c
index 9a333e21b51..e2ae1f47fee 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "refs.h"
 #include "wildmatch.h"
+#include "object-store.h"
 #include "commit.h"
 #include "remote.h"
 #include "color.h"
diff --git a/refs.c b/refs.c
index 27c88ba7689..2361ba9c6ee 100644
--- a/refs.c
+++ b/refs.c
@@ -9,6 +9,7 @@
 #include "iterator.h"
 #include "refs.h"
 #include "refs/refs-internal.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "submodule.h"
diff --git a/remote-testsvn.c b/remote-testsvn.c
index c4bb9a8ba92..515b922b503 100644
--- a/remote-testsvn.c
+++ b/remote-testsvn.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "refs.h"
 #include "remote.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "url.h"
 #include "exec_cmd.h"
diff --git a/remote.c b/remote.c
index 481bf933f39..db438cfbc56 100644
--- a/remote.c
+++ b/remote.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "remote.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/rerere.c b/rerere.c
index 18cae2d11c9..fcb99cc9542 100644
--- a/rerere.c
+++ b/rerere.c
@@ -9,6 +9,7 @@
 #include "ll-merge.h"
 #include "attr.h"
 #include "pathspec.h"
+#include "object-store.h"
 #include "sha1-lookup.h"
 
 #define RESOLVED 0
diff --git a/revision.c b/revision.c
index 1cff11833e7..260b9c276c7 100644
--- a/revision.c
+++ b/revision.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "tag.h"
 #include "blob.h"
 #include "tree.h"
diff --git a/send-pack.c b/send-pack.c
index 19025a7aca8..71600028cdd 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "commit.h"
 #include "refs.h"
+#include "object-store.h"
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/sequencer.c b/sequencer.c
index 44f0518b9c4..cf6456ab75d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "lockfile.h"
 #include "dir.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "sequencer.h"
diff --git a/shallow.c b/shallow.c
index df4d44ea7a3..c2f81a5a5a8 100644
--- a/shallow.c
+++ b/shallow.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "tempfile.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "pkt-line.h"
diff --git a/submodule-config.c b/submodule-config.c
index 3f2075764fe..a1e57c33339 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -4,6 +4,7 @@
 #include "submodule-config.h"
 #include "submodule.h"
 #include "strbuf.h"
+#include "object-store.h"
 #include "parse-options.h"
 
 /*
diff --git a/tag.c b/tag.c
index 7c12426b4ea..3be7206e920 100644
--- a/tag.c
+++ b/tag.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "tag.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
diff --git a/tree-walk.c b/tree-walk.c
index e11b3063afa..cd428a1ee53 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -2,6 +2,7 @@
 #include "tree-walk.h"
 #include "unpack-trees.h"
 #include "dir.h"
+#include "object-store.h"
 #include "tree.h"
 #include "pathspec.h"
 
diff --git a/tree.c b/tree.c
index 8f8ef3189af..dc20a2693fd 100644
--- a/tree.c
+++ b/tree.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "cache-tree.h"
 #include "tree.h"
+#include "object-store.h"
 #include "blob.h"
 #include "commit.h"
 #include "tag.h"
diff --git a/unpack-trees.c b/unpack-trees.c
index e73745051e5..e3f95cc6f95 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -15,6 +15,7 @@
 #include "submodule.h"
 #include "submodule-config.h"
 #include "fsmonitor.h"
+#include "object-store.h"
 #include "fetch-object.h"
 
 /*
diff --git a/upload-pack.c b/upload-pack.c
index 4a82602be5d..a11c6d192ce 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -3,6 +3,7 @@
 #include "refs.h"
 #include "pkt-line.h"
 #include "sideband.h"
+#include "object-store.h"
 #include "tag.h"
 #include "object.h"
 #include "commit.h"
diff --git a/walker.c b/walker.c
index dffb9c8e37c..3fb5fb2a0be 100644
--- a/walker.c
+++ b/walker.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "walker.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/xdiff-interface.c b/xdiff-interface.c
index 9315bc0ede1..ec6e574e4aa 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "xdiff-interface.h"
 #include "xdiff/xtypes.h"
 #include "xdiff/xdiffi.h"
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 5%]

* Re: [PATCH 2/3] merge-recursive: i18n submodule merge output and respect verbosity
  2018-05-15 20:00 ` [PATCH 2/3] merge-recursive: i18n submodule merge output and respect verbosity Stefan Beller
@ 2018-05-16  1:17   ` Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-05-16  1:17 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, leif.middelschulte, newren

Stefan Beller <sbeller@google.com> writes:

> +static int merge_submodule(struct merge_options *o,
> +			   struct object_id *result, const char *path,
>  			   const struct object_id *base, const struct object_id *a,
> -			   const struct object_id *b, int search)
> +			   const struct object_id *b)
>  {
>  	struct commit *commit_base, *commit_a, *commit_b;
>  	int parent_count;
>  	struct object_array merges;
>  
>  	int i;
> +	int search = !o->call_depth;

I kind of like this "while at it" change in this patch ;-)

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 3/3] Inform about fast-forwarding of submodules during merge
  2018-05-15 20:00 ` [PATCH 3/3] Inform about fast-forwarding of submodules during merge Stefan Beller
@ 2018-05-16  1:36   ` Elijah Newren
  2018-05-16  1:36   ` Junio C Hamano
  2018-05-16  1:42   ` Junio C Hamano
  2 siblings, 0 replies; 200+ results
From: Elijah Newren @ 2018-05-16  1:36 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Git Mailing List, Leif Middelschulte, Junio C Hamano

On Tue, May 15, 2018 at 1:00 PM, Stefan Beller <sbeller@google.com> wrote:
> From: Leif Middelschulte <Leif.Middelschulte@gmail.com>
>
> Inform the user about an automatically fast-forwarded submodule. The
> silent merge behavior was introduced by commit 68d03e4a6e44 ("Implement
> automatic fast-forward merge for submodules", 2010-07-07)).
>
> Signed-off-by: Leif Middelschulte <Leif.Middelschulte@gmail.com>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  merge-recursive.c | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 0571919ee0a..29a430c418a 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1093,10 +1093,26 @@ static int merge_submodule(struct merge_options *o,
>         /* Case #1: a is contained in b or vice versa */
>         if (in_merge_bases(commit_a, commit_b)) {
>                 oidcpy(result, b);
> +               if (show(o, 3)) {
> +                       output(o, 1, _("Fast-forwarding submodule %s to the following commit:"), path);

Seems slightly odd to mix 3 and 1 here; although it'll work just fine,
I would have expected use of 3 in both places (much like you did with
the 2 and 2 below).

> +                       output_commit_title(o, commit_b);
> +               } else if (show(o, 2))
> +                       output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(b));
> +               else
> +                       ; /* no output */
> +
>                 return 1;
>         }
>         if (in_merge_bases(commit_b, commit_a)) {
>                 oidcpy(result, a);
> +               if (show(o, 3)) {
> +                       output(o, 1, _("Fast-forwarding submodule %s to the following commit:"), path);

Same.

> +                       output_commit_title(o, commit_a);
> +               } else if (show(o, 2))
> +                       output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(a));
> +               else
> +                       ; /* no output */
> +
>                 return 1;
>         }
>
> --
> 2.17.0.582.gccdcbd54c44.dirty

Other than that nit-pick, looks good to me.

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 3/3] Inform about fast-forwarding of submodules during merge
  2018-05-15 20:00 ` [PATCH 3/3] Inform about fast-forwarding of submodules during merge Stefan Beller
  2018-05-16  1:36   ` Elijah Newren
@ 2018-05-16  1:36   ` Junio C Hamano
  2018-05-16  1:42   ` Junio C Hamano
  2 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-05-16  1:36 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, leif.middelschulte, newren

Stefan Beller <sbeller@google.com> writes:

> From: Leif Middelschulte <Leif.Middelschulte@gmail.com>
>

Subject: merge-recursive: give notice when submodule commit gets fast-forwarded

perhaps?

>  	/* Case #1: a is contained in b or vice versa */
>  	if (in_merge_bases(commit_a, commit_b)) {
>  		oidcpy(result, b);
> +		if (show(o, 3)) {
> +			output(o, 1, _("Fast-forwarding submodule %s to the following commit:"), path);
> +			output_commit_title(o, commit_b);
> +		} else if (show(o, 2))
> +			output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(b));
> +		else
> +			; /* no output */
> +

merge.verbosity::
	Controls the amount of output shown by the recursive merge
	strategy.  Level 0 outputs nothing except a final error
	message if conflicts were detected. Level 1 outputs only
	conflicts, 2 outputs conflicts and file changes.  Level 5 and
	above outputs debugging information.  The default is level 2.
	Can be overridden by the `GIT_MERGE_VERBOSITY` environment variable.

So, by default, we report the fact that we update submodule to a
particular commit, which is quite similar to how we report auto
merged paths using the content level 3-way merge; when you squint
your eyes, the "fast-forward" of submodules look somewhat like a
content-level 3-way merge anyway ;-)

And at level 3, which currently is used to report a non-event that
does not change the result of the merge from what was naturally
expected, we give a bit more detail by citing the commit the
submodule gets fast-forwarded to [*1*].

Sort of makes sense.


[Footnote]

*1* I wonder if that is really necessary, though---we do not give
"here is a diff" or "this is the new contents" after a path gets
merged for normal files.  And if it is needed perhaps because
submodules are so special, I wonder if we also need to give the
commit the submodule gets fast-forwarded from, i.e. the original
one, the same way.

^ permalink raw reply	[relevance 7%]

* Re: [PATCH 3/3] Inform about fast-forwarding of submodules during merge
  2018-05-15 20:00 ` [PATCH 3/3] Inform about fast-forwarding of submodules during merge Stefan Beller
  2018-05-16  1:36   ` Elijah Newren
  2018-05-16  1:36   ` Junio C Hamano
@ 2018-05-16  1:42   ` Junio C Hamano
  2 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-05-16  1:42 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, leif.middelschulte, newren

Stefan Beller <sbeller@google.com> writes:

> From: Leif Middelschulte <Leif.Middelschulte@gmail.com>
>
> Inform the user about an automatically fast-forwarded submodule. The
> silent merge behavior was introduced by commit 68d03e4a6e44 ("Implement
> automatic fast-forward merge for submodules", 2010-07-07)).

Oh, another thing I forgot to mention.

These three lines do not convey much useful information.  The first
sentence can be read from the patch text, and the rest can be read
from "git blame" and "git log" output.

It is correct that the silent behaviour was introduced long time
ago.  The proposed log message does not even say if that silent
behaviour is bad in any way, let alone why it is bad and need to be
changed.

Perhaps Leif can elaborate why this change is a good idea in the
first place?

Thanks.

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] grep: handle corrupt index files early
  2018-05-15 16:44   ` [PATCH] grep: handle corrupt index files early Stefan Beller
@ 2018-05-16 15:24     ` Duy Nguyen
  2018-05-16 22:21       ` [PATCH 00/11] Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Duy Nguyen @ 2018-05-16 15:24 UTC (permalink / raw)
  To: Stefan Beller
  Cc: Brandon Williams, Junio C Hamano, Git Mailing List, Antonio Ospite

On Tue, May 15, 2018 at 6:44 PM, Stefan Beller <sbeller@google.com> wrote:
> On Tue, May 15, 2018 at 6:13 AM, Duy Nguyen <pclouds@gmail.com> wrote:
>> On Tue, May 15, 2018 at 3:04 AM, Stefan Beller <sbeller@google.com> wrote:
>>> Any other caller of 'repo_read_index' dies upon a negative return of
>>> it, so grep should, too.
>>>
>>> Signed-off-by: Stefan Beller <sbeller@google.com>
>>> ---
>>>
>>> Found while reviewing the series
>>> https://public-inbox.org/git/20180514105823.8378-1-ao2@ao2.it/
>>>
>>>  builtin/grep.c | 3 ++-
>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/builtin/grep.c b/builtin/grep.c
>>> index 6e7bc76785a..69f0743619f 100644
>>> --- a/builtin/grep.c
>>> +++ b/builtin/grep.c
>>> @@ -488,7 +488,8 @@ static int grep_cache(struct grep_opt *opt, struct repository *repo,
>>>                 strbuf_addstr(&name, repo->submodule_prefix);
>>>         }
>>>
>>> -       repo_read_index(repo);
>>> +       if (repo_read_index(repo) < 0)
>>> +               die("index file corrupt");
>>
>> _() the string (and maybe reuse an existing phrase if found to reduce
>> workload on translators)
>
> sbeller@sbeller:/u/git$ git grep -A 1 repo_read_index
> builtin/grep.c:491:     if (repo_read_index(repo) < 0)
> builtin/grep.c-492-             die("index file corrupt");
> --
> builtin/ls-files.c:213: if (repo_read_index(&submodule) < 0)
> builtin/ls-files.c-214-         die("index file corrupt");
> --
> builtin/ls-files.c:582: if (repo_read_index(the_repository) < 0)
> builtin/ls-files.c-583-         die("index file corrupt");
> --
> dir.c:3028:     if (repo_read_index(&subrepo) < 0)
> dir.c-3029-             die("index file corrupt in repo %s", subrepo.gitdir);
> --
> repository.c:245:int repo_read_index(struct repository *repo)
> repository.c-246-{
> --
> repository.h:70:         * 'repo_read_index()' can be used to populate 'index'.
> repository.h-71-         */
> --
> repository.h:119:extern int repo_read_index(struct repository *repo);
> repository.h-120-
> --
> submodule-config.c:583:         if (repo_read_index(repo) < 0)
> submodule-config.c-584-                 return;
> --
> submodule.c:1336:       if (repo_read_index(r) < 0)
> submodule.c-1337-               die("index file corrupt");
>
> I think this is as good as it gets for using an existing phrase.
> None of them are translated, which I would defer to a follow up patch
> that translates all(?) of them or just the porcelains.

If you have time, yes translate them all. I don't see how any of these
strings are meant for script. If not, just _() the new string you
added is fine.

With a majority of call sites dying like this though, I wonder if we
should just add repo_read_index_or_die() with die() inside. Then the
next person won't likely accidentally forget _()

>
> Thanks,
> Stefan



-- 
Duy

^ permalink raw reply	[relevance 2%]

* [PATCH 06/11] read_cache: use repo_read_index_or_die with different error messages
  2018-05-16 22:21       ` [PATCH 00/11] Stefan Beller
                           ` (2 preceding siblings ...)
  2018-05-16 22:21         ` [PATCH 05/11] builtin/ls-files: " Stefan Beller
@ 2018-05-16 22:21         ` Stefan Beller
  2018-05-16 22:21         ` [PATCH 11/11] read_cache: convert most calls to repo_read_index_or_die Stefan Beller
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-16 22:21 UTC (permalink / raw)
  To: pclouds; +Cc: ao2, bmwill, git, gitster, sbeller

This replaces all patterns of "if (read_cached() < 0) die(some-msg);"
with repo_read_index_or_die; this changes the output of the error message.

However as all error messages before were translated, these are for human
consumption, so a change in error message is not bad; in fact it makes Git
more consistent.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/add.c       | 4 ++--
 builtin/commit.c    | 9 +++++----
 builtin/rev-parse.c | 4 ++--
 merge.c             | 4 ++--
 4 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index e4751c198c1..910f619b7d5 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -19,6 +19,7 @@
 #include "bulk-checkin.h"
 #include "argv-array.h"
 #include "submodule.h"
+#include "repository.h"
 
 static const char * const builtin_add_usage[] = {
 	N_("git add [<options>] [--] <pathspec>..."),
@@ -229,8 +230,7 @@ static int edit_patch(int argc, const char **argv, const char *prefix)
 
 	git_config(git_diff_basic_config, NULL); /* no "diff" UI options */
 
-	if (read_cache() < 0)
-		die(_("Could not read the index"));
+	repo_read_index_or_die(the_repository);
 
 	init_revisions(&rev, prefix);
 	rev.diffopt.context = 7;
diff --git a/builtin/commit.c b/builtin/commit.c
index 5571d4a3e2b..9ebfb4db415 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -32,6 +32,8 @@
 #include "column.h"
 #include "sequencer.h"
 #include "mailmap.h"
+#include "sigchain.h"
+#include "repository.h"
 
 static const char * const builtin_commit_usage[] = {
 	N_("git commit [<options>] [--] <pathspec>..."),
@@ -428,8 +430,7 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 		exit(1);
 
 	discard_cache();
-	if (read_cache() < 0)
-		die(_("cannot read the index"));
+	repo_read_index_or_die(the_repository);
 
 	hold_locked_index(&index_lock, LOCK_DIE_ON_ERROR);
 	add_remove_files(&partial);
@@ -853,8 +854,8 @@ static int prepare_to_commit(const char *index_file, const char *prefix,
 		struct object_id oid;
 		const char *parent = "HEAD";
 
-		if (!active_nr && read_cache() < 0)
-			die(_("Cannot read index"));
+		if (!active_nr)
+			repo_read_index_or_die(the_repository);
 
 		if (amend)
 			parent = "HEAD^1";
diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index 36b20877828..37f29fd850d 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -14,6 +14,7 @@
 #include "revision.h"
 #include "split-index.h"
 #include "submodule.h"
+#include "repository.h"
 
 #define DO_REVS		1
 #define DO_NOREV	2
@@ -884,8 +885,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix)
 				continue;
 			}
 			if (!strcmp(arg, "--shared-index-path")) {
-				if (read_cache() < 0)
-					die(_("Could not read the index"));
+				repo_read_index_or_die(the_repository);
 				if (the_index.split_index) {
 					const unsigned char *sha1 = the_index.split_index->base_sha1;
 					const char *path = git_path("sharedindex.%s", sha1_to_hex(sha1));
diff --git a/merge.c b/merge.c
index f06a4773d4f..654d049e80d 100644
--- a/merge.c
+++ b/merge.c
@@ -8,6 +8,7 @@
 #include "tree-walk.h"
 #include "unpack-trees.h"
 #include "dir.h"
+#include "repository.h"
 
 static const char *merge_argument(struct commit *commit)
 {
@@ -70,8 +71,7 @@ int try_merge_command(const char *strategy, size_t xopts_nr,
 	argv_array_clear(&args);
 
 	discard_cache();
-	if (read_cache() < 0)
-		die(_("failed to read the cache"));
+	repo_read_index_or_die(the_repository);
 	resolve_undo_clear();
 
 	return ret;
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 7%]

* [PATCH 05/11] builtin/ls-files: use repo_read_index_or_die
  2018-05-16 22:21       ` [PATCH 00/11] Stefan Beller
  2018-05-16 22:21         ` [PATCH 02/11] repository: introduce repo_read_index_or_die Stefan Beller
  2018-05-16 22:21         ` [PATCH 04/11] submodule: use repo_read_index_or_die Stefan Beller
@ 2018-05-16 22:21         ` " Stefan Beller
  2018-05-16 22:21         ` [PATCH 06/11] read_cache: use repo_read_index_or_die with different error messages Stefan Beller
  2018-05-16 22:21         ` [PATCH 11/11] read_cache: convert most calls to repo_read_index_or_die Stefan Beller
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-16 22:21 UTC (permalink / raw)
  To: pclouds; +Cc: ao2, bmwill, git, gitster, sbeller

Despite ls-files being a plumbing command, which promises to not change its
output ever, and to be easy on machines (e.g. non-localized output),
it may make sense to localize the error message for a corrupt index
nevertheless:

1. that is more consistent with the rest of Git.
2. Searching for "ls-tree corrupt index file" on the web doesn't yield
   any hits, that suggest the exact string is parsed for.
   Probably the script authors rely on the exit code of ls-tree anyways.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/ls-files.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/builtin/ls-files.c b/builtin/ls-files.c
index a71f6bd088a..502f2f6db04 100644
--- a/builtin/ls-files.c
+++ b/builtin/ls-files.c
@@ -20,6 +20,7 @@
 #include "run-command.h"
 #include "submodule.h"
 #include "submodule-config.h"
+#include "repository.h"
 
 static int abbrev;
 static int show_deleted;
@@ -210,8 +211,7 @@ static void show_submodule(struct repository *superproject,
 	if (repo_submodule_init(&submodule, superproject, path))
 		return;
 
-	if (repo_read_index(&submodule) < 0)
-		die("index file corrupt");
+	repo_read_index_or_die(&submodule);
 
 	show_files(&submodule, dir);
 
@@ -579,8 +579,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)
 		prefix_len = strlen(prefix);
 	git_config(git_default_config, NULL);
 
-	if (repo_read_index(the_repository) < 0)
-		die("index file corrupt");
+	repo_read_index_or_die(the_repository);
 
 	argc = parse_options(argc, argv, prefix, builtin_ls_files_options,
 			ls_files_usage, 0);
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 15%]

* [PATCH 11/11] read_cache: convert most calls to repo_read_index_or_die
  2018-05-16 22:21       ` [PATCH 00/11] Stefan Beller
                           ` (3 preceding siblings ...)
  2018-05-16 22:21         ` [PATCH 06/11] read_cache: use repo_read_index_or_die with different error messages Stefan Beller
@ 2018-05-16 22:21         ` Stefan Beller
  2018-05-16 22:27           ` Brandon Williams
  4 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-16 22:21 UTC (permalink / raw)
  To: pclouds; +Cc: ao2, bmwill, git, gitster, sbeller

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 blame.c               | 5 +++--
 builtin/am.c          | 3 ++-
 builtin/diff.c        | 3 ++-
 builtin/fsck.c        | 3 ++-
 builtin/merge-index.c | 3 ++-
 check-racy.c          | 2 +-
 diff.c                | 5 +++--
 merge-recursive.c     | 3 ++-
 revision.c            | 5 +++--
 sequencer.c           | 5 +++--
 sha1-name.c           | 2 +-
 11 files changed, 24 insertions(+), 15 deletions(-)

diff --git a/blame.c b/blame.c
index 78c9808bd1a..ebfa1c8efcd 100644
--- a/blame.c
+++ b/blame.c
@@ -5,6 +5,7 @@
 #include "diff.h"
 #include "diffcore.h"
 #include "tag.h"
+#include "repository.h"
 #include "blame.h"
 
 void blame_origin_decref(struct blame_origin *o)
@@ -159,7 +160,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 	unsigned mode;
 	struct strbuf msg = STRBUF_INIT;
 
-	read_cache();
+	repo_read_index_or_die(the_repository);
 	time(&now);
 	commit = alloc_commit_node();
 	commit->object.parsed = 1;
@@ -241,7 +242,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
 	 * want to run "diff-index --cached".
 	 */
 	discard_cache();
-	read_cache();
+	repo_read_index_or_die(the_repository);
 
 	len = strlen(path);
 	if (!mode) {
diff --git a/builtin/am.c b/builtin/am.c
index d834f9e62b6..3c6e77a5369 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -32,6 +32,7 @@
 #include "apply.h"
 #include "string-list.h"
 #include "packfile.h"
+#include "repository.h"
 
 /**
  * Returns 1 if the file is empty or does not exist, 0 otherwise.
@@ -1581,7 +1582,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
 
 	discard_cache();
-	read_cache();
+	repo_read_index_or_die(the_repository);
 
 	/*
 	 * This is not so wrong. Depending on which base we picked, orig_tree
diff --git a/builtin/diff.c b/builtin/diff.c
index 16bfb22f738..4bba211f1c7 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -17,6 +17,7 @@
 #include "builtin.h"
 #include "submodule.h"
 #include "sha1-array.h"
+#include "repository.h"
 
 #define DIFF_NO_INDEX_EXPLICIT 1
 #define DIFF_NO_INDEX_IMPLICIT 2
@@ -210,7 +211,7 @@ static void refresh_index_quietly(void)
 	if (fd < 0)
 		return;
 	discard_cache();
-	read_cache();
+	repo_read_index(the_repository); /* do not die on error */
 	refresh_cache(REFRESH_QUIET|REFRESH_UNMERGED);
 	update_index_if_able(&the_index, &lock_file);
 }
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 087360a6757..a42e98235da 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -18,6 +18,7 @@
 #include "decorate.h"
 #include "packfile.h"
 #include "object-store.h"
+#include "repository.h"
 
 #define REACHABLE 0x0001
 #define SEEN      0x0002
@@ -795,7 +796,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
 	if (keep_cache_objects) {
 		verify_index_checksum = 1;
 		verify_ce_order = 1;
-		read_cache();
+		repo_read_index_or_die(the_repository);
 		for (i = 0; i < active_nr; i++) {
 			unsigned int mode;
 			struct blob *blob;
diff --git a/builtin/merge-index.c b/builtin/merge-index.c
index c99443b095b..2d91c7c3b5e 100644
--- a/builtin/merge-index.c
+++ b/builtin/merge-index.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "run-command.h"
+#include "repository.h"
 
 static const char *pgm;
 static int one_shot, quiet;
@@ -77,7 +78,7 @@ int cmd_merge_index(int argc, const char **argv, const char *prefix)
 	if (argc < 3)
 		usage("git merge-index [-o] [-q] <merge-program> (-a | [--] [<filename>...])");
 
-	read_cache();
+	repo_read_index_or_die(the_repository);
 
 	i = 1;
 	if (!strcmp(argv[i], "-o")) {
diff --git a/check-racy.c b/check-racy.c
index 24b6542352a..9b884639cf4 100644
--- a/check-racy.c
+++ b/check-racy.c
@@ -6,7 +6,7 @@ int main(int ac, char **av)
 	int dirty, clean, racy;
 
 	dirty = clean = racy = 0;
-	read_cache();
+	repo_read_index_or_die(the_repository);
 	for (i = 0; i < active_nr; i++) {
 		struct cache_entry *ce = active_cache[i];
 		struct stat st;
diff --git a/diff.c b/diff.c
index 1289df4b1f9..383f52fa118 100644
--- a/diff.c
+++ b/diff.c
@@ -22,6 +22,7 @@
 #include "argv-array.h"
 #include "graph.h"
 #include "packfile.h"
+#include "repository.h"
 
 #ifdef NO_FAST_WORKING_DIRECTORY
 #define FAST_WORKING_DIRECTORY 0
@@ -4210,13 +4211,13 @@ void diff_setup_done(struct diff_options *options)
 		options->rename_limit = diff_rename_limit_default;
 	if (options->setup & DIFF_SETUP_USE_CACHE) {
 		if (!active_cache)
-			/* read-cache does not die even when it fails
+			/* repo_read_indexe does not die even when it fails
 			 * so it is safe for us to do this here.  Also
 			 * it does not smudge active_cache or active_nr
 			 * when it fails, so we do not have to worry about
 			 * cleaning it up ourselves either.
 			 */
-			read_cache();
+			repo_read_index(the_repository);
 	}
 	if (40 < options->abbrev)
 		options->abbrev = 40; /* full */
diff --git a/merge-recursive.c b/merge-recursive.c
index 0c0d48624da..76911c935c3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2151,7 +2151,8 @@ int merge_recursive(struct merge_options *o,
 
 	discard_cache();
 	if (!o->call_depth)
-		read_cache();
+		if (read_cache() < 0)
+			return err(o, _("index file corrupt"));
 
 	o->ancestor = "merged common ancestors";
 	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
diff --git a/revision.c b/revision.c
index 1cff11833e7..8ad9824143d 100644
--- a/revision.c
+++ b/revision.c
@@ -23,6 +23,7 @@
 #include "packfile.h"
 #include "worktree.h"
 #include "argv-array.h"
+#include "repository.h"
 
 volatile show_early_output_fn_t show_early_output;
 
@@ -1344,7 +1345,7 @@ void add_index_objects_to_pending(struct rev_info *revs, unsigned int flags)
 {
 	struct worktree **worktrees, **p;
 
-	read_cache();
+	repo_read_index_or_die(the_repository);
 	do_add_index_objects_to_pending(revs, &the_index);
 
 	if (revs->single_worktree)
@@ -1486,7 +1487,7 @@ static void prepare_show_merge(struct rev_info *revs)
 	head->object.flags |= SYMMETRIC_LEFT;
 
 	if (!active_nr)
-		read_cache();
+		repo_read_index_or_die(the_repository);
 	for (i = 0; i < active_nr; i++) {
 		const struct cache_entry *ce = active_cache[i];
 		if (!ce_stage(ce))
diff --git a/sequencer.c b/sequencer.c
index 4ce5120e777..773165c8cde 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -23,6 +23,7 @@
 #include "hashmap.h"
 #include "notes-utils.h"
 #include "sigchain.h"
+#include "repository.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -432,7 +433,7 @@ static int fast_forward_to(const struct object_id *to, const struct object_id *f
 	struct strbuf sb = STRBUF_INIT;
 	struct strbuf err = STRBUF_INIT;
 
-	read_cache();
+	repo_read_index_or_die(the_repository);
 	if (checkout_fast_forward(from, to, 1))
 		return -1; /* the callee should have complained already */
 
@@ -489,7 +490,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	if (hold_locked_index(&index_lock, LOCK_REPORT_ON_ERROR) < 0)
 		return -1;
 
-	read_cache();
+	repo_read_index_or_die(the_repository);
 
 	init_merge_options(&o);
 	o.ancestor = base ? base_label : "(empty tree)";
diff --git a/sha1-name.c b/sha1-name.c
index 5b93bf8da36..83d5f945cf1 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -1639,7 +1639,7 @@ static int get_oid_with_context_1(const char *name,
 			oc->path = xstrdup(cp);
 
 		if (!active_cache)
-			read_cache();
+			repo_read_index_or_die(the_repository);
 		pos = cache_name_pos(cp, namelen);
 		if (pos < 0)
 			pos = -pos - 1;
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 5%]

* [PATCH 00/11] 
  2018-05-16 15:24     ` Duy Nguyen
@ 2018-05-16 22:21       ` Stefan Beller
  2018-05-16 22:21         ` [PATCH 02/11] repository: introduce repo_read_index_or_die Stefan Beller
                           ` (4 more replies)
  0 siblings, 5 replies; 200+ results
From: Stefan Beller @ 2018-05-16 22:21 UTC (permalink / raw)
  To: pclouds; +Cc: ao2, bmwill, git, gitster, sbeller

> If you have time, yes translate them all. I don't see how any of these
> strings are meant for script. If not, just _() the new string you
> added is fine.

> With a majority of call sites dying like this though, I wonder if we
> should just add repo_read_index_or_die() with die() inside. Then the
> next person won't likely accidentally forget _()

So this comment tricked me into coming up with a patch series. :)

Each patch is themed, I tried to make each commit special w.r.t. reviewers
attention.

We'd start out with a resend of the origin patch, which is boring.

Then we'll move all similar cases into one function (no-op for
introducing the repo_read_index_or_die function as all callers will have the
same error message and the same localisation).

Any following patch will be more controversial then the previous patches,
I would expect, as we introduce more and more change.

The last patch is just an attempt to finish the series gracefully,
and may contain errors (sometimes we do not want to die() in case of
corrupt index).

Is this series roughly what you had in mind?

Thanks,
Stefan


Stefan Beller (11):
  grep: handle corrupt index files early
  repository: introduce repo_read_index_or_die
  builtin/grep: use repo_read_index_or_die
  submodule: use repo_read_index_or_die
  builtin/ls-files: use repo_read_index_or_die
  read_cache: use repo_read_index_or_die with different error messages
  rerere: use repo_read_index_or_die
  check-attr: switch to repo_read_index_or_die
  checkout-index: switch to repo_read_index
  test helpers: switch to repo_read_index_or_die
  read_cache: convert most calls to repo_read_index_or_die

 blame.c                              |  5 +++--
 builtin/add.c                        |  7 +++----
 builtin/am.c                         |  3 ++-
 builtin/check-attr.c                 |  5 ++---
 builtin/check-ignore.c               |  7 ++++---
 builtin/checkout-index.c             |  5 ++---
 builtin/clean.c                      |  4 ++--
 builtin/commit.c                     |  9 +++++----
 builtin/diff.c                       |  3 ++-
 builtin/fsck.c                       |  3 ++-
 builtin/grep.c                       |  2 +-
 builtin/ls-files.c                   |  7 +++----
 builtin/merge-index.c                |  3 ++-
 builtin/mv.c                         |  3 +--
 builtin/reset.c                      |  3 +--
 builtin/rev-parse.c                  |  4 ++--
 builtin/rm.c                         |  3 +--
 builtin/submodule--helper.c          |  3 +--
 check-racy.c                         |  2 +-
 diff.c                               |  5 +++--
 merge-recursive.c                    |  3 ++-
 merge.c                              |  4 ++--
 repository.c                         |  6 ++++++
 repository.h                         |  8 ++++++++
 rerere.c                             | 10 ++++------
 revision.c                           |  5 +++--
 sequencer.c                          |  5 +++--
 sha1-name.c                          |  2 +-
 submodule.c                          |  3 +--
 t/helper/test-dump-cache-tree.c      |  5 ++---
 t/helper/test-dump-untracked-cache.c |  4 ++--
 t/helper/test-lazy-init-name-hash.c  | 11 ++++++-----
 t/helper/test-read-cache.c           |  3 ++-
 t/helper/test-scrap-cache-tree.c     |  4 ++--
 t/helper/test-write-cache.c          |  3 ++-
 35 files changed, 89 insertions(+), 73 deletions(-)

-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 7%]

* [PATCH 02/11] repository: introduce repo_read_index_or_die
  2018-05-16 22:21       ` [PATCH 00/11] Stefan Beller
@ 2018-05-16 22:21         ` Stefan Beller
  2018-05-19  6:37           ` Duy Nguyen
  2018-05-16 22:21         ` [PATCH 04/11] submodule: use repo_read_index_or_die Stefan Beller
                           ` (3 subsequent siblings)
  4 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-16 22:21 UTC (permalink / raw)
  To: pclouds; +Cc: ao2, bmwill, git, gitster, sbeller

A common pattern with the repo_read_index function is to die if the return
of repo_read_index is negative.  Move this pattern into a function.

This patch replaces the calls of repo_read_index with its _or_die version,
if the error message is exactly the same.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/add.c               | 3 +--
 builtin/check-ignore.c      | 7 ++++---
 builtin/clean.c             | 4 ++--
 builtin/mv.c                | 3 +--
 builtin/reset.c             | 3 +--
 builtin/rm.c                | 3 +--
 builtin/submodule--helper.c | 3 +--
 repository.c                | 6 ++++++
 repository.h                | 8 ++++++++
 9 files changed, 25 insertions(+), 15 deletions(-)

diff --git a/builtin/add.c b/builtin/add.c
index c9e2619a9ad..e4751c198c1 100644
--- a/builtin/add.c
+++ b/builtin/add.c
@@ -445,8 +445,7 @@ int cmd_add(int argc, const char **argv, const char *prefix)
 		return 0;
 	}
 
-	if (read_cache() < 0)
-		die(_("index file corrupt"));
+	repo_read_index_or_die(the_repository);
 
 	die_in_unpopulated_submodule(&the_index, prefix);
 
diff --git a/builtin/check-ignore.c b/builtin/check-ignore.c
index ec9a959e08d..2a46bf9af7f 100644
--- a/builtin/check-ignore.c
+++ b/builtin/check-ignore.c
@@ -6,6 +6,7 @@
 #include "pathspec.h"
 #include "parse-options.h"
 #include "submodule.h"
+#include "repository.h"
 
 static int quiet, verbose, stdin_paths, show_non_matching, no_index;
 static const char * const check_ignore_usage[] = {
@@ -172,9 +173,9 @@ int cmd_check_ignore(int argc, const char **argv, const char *prefix)
 	if (show_non_matching && !verbose)
 		die(_("--non-matching is only valid with --verbose"));
 
-	/* read_cache() is only necessary so we can watch out for submodules. */
-	if (!no_index && read_cache() < 0)
-		die(_("index file corrupt"));
+	/* repo_read_index() is only necessary so we can watch out for submodules. */
+	if (!no_index)
+		repo_read_index_or_die(the_repository);
 
 	memset(&dir, 0, sizeof(dir));
 	setup_standard_excludes(&dir);
diff --git a/builtin/clean.c b/builtin/clean.c
index fad533a0a73..6c1c6fdd7f9 100644
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@ -16,6 +16,7 @@
 #include "column.h"
 #include "color.h"
 #include "pathspec.h"
+#include "repository.h"
 
 static int force = -1; /* unset */
 static int interactive;
@@ -954,8 +955,7 @@ int cmd_clean(int argc, const char **argv, const char *prefix)
 	if (remove_directories)
 		dir.flags |= DIR_SHOW_IGNORED_TOO | DIR_KEEP_UNTRACKED_CONTENTS;
 
-	if (read_cache() < 0)
-		die(_("index file corrupt"));
+	repo_read_index_or_die(the_repository);
 
 	if (!ignored)
 		setup_standard_excludes(&dir);
diff --git a/builtin/mv.c b/builtin/mv.c
index 7a63667d648..7f62e626dda 100644
--- a/builtin/mv.c
+++ b/builtin/mv.c
@@ -140,8 +140,7 @@ int cmd_mv(int argc, const char **argv, const char *prefix)
 		usage_with_options(builtin_mv_usage, builtin_mv_options);
 
 	hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);
-	if (read_cache() < 0)
-		die(_("index file corrupt"));
+	repo_read_index_or_die(the_repository);
 
 	source = internal_prefix_pathspec(prefix, argv, argc, 0);
 	modes = xcalloc(argc, sizeof(enum update_mode));
diff --git a/builtin/reset.c b/builtin/reset.c
index 7f1c3f02a30..fd514eec822 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -237,8 +237,7 @@ static void parse_args(struct pathspec *pathspec,
 	}
 	*rev_ret = rev;
 
-	if (read_cache() < 0)
-		die(_("index file corrupt"));
+	repo_read_index_or_die(the_repository);
 
 	parse_pathspec(pathspec, 0,
 		       PATHSPEC_PREFER_FULL |
diff --git a/builtin/rm.c b/builtin/rm.c
index 5b6fc7ee818..3b90191aa53 100644
--- a/builtin/rm.c
+++ b/builtin/rm.c
@@ -267,8 +267,7 @@ int cmd_rm(int argc, const char **argv, const char *prefix)
 
 	hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);
 
-	if (read_cache() < 0)
-		die(_("index file corrupt"));
+	repo_read_index_or_die(the_repository);
 
 	parse_pathspec(&pathspec, 0,
 		       PATHSPEC_PREFER_CWD,
diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index c2403a915ff..7aebed9bd66 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -323,8 +323,7 @@ static int module_list_compute(int argc, const char **argv,
 	if (pathspec->nr)
 		ps_matched = xcalloc(pathspec->nr, 1);
 
-	if (read_cache() < 0)
-		die(_("index file corrupt"));
+	repo_read_index_or_die(the_repository);
 
 	for (i = 0; i < active_nr; i++) {
 		const struct cache_entry *ce = active_cache[i];
diff --git a/repository.c b/repository.c
index beff3caa9e2..ceca14ba718 100644
--- a/repository.c
+++ b/repository.c
@@ -249,3 +249,9 @@ int repo_read_index(struct repository *repo)
 
 	return read_index_from(repo->index, repo->index_file, repo->gitdir);
 }
+
+void repo_read_index_or_die(struct repository *repo)
+{
+	if (repo_read_index(repo) < 0)
+		die(_("index file corrupt"));
+}
diff --git a/repository.h b/repository.h
index f2646f0c52a..8efd8979ad3 100644
--- a/repository.h
+++ b/repository.h
@@ -118,4 +118,12 @@ extern void repo_clear(struct repository *repo);
  */
 extern int repo_read_index(struct repository *repo);
 
+/*
+ * Populates the index in the repository from its index file,
+ * allocating the struct index if needed.
+ *
+ * If the index file cannot be read, die.
+ */
+void repo_read_index_or_die(struct repository *r);
+
 #endif /* REPOSITORY_H */
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 14%]

* [PATCH 04/11] submodule: use repo_read_index_or_die
  2018-05-16 22:21       ` [PATCH 00/11] Stefan Beller
  2018-05-16 22:21         ` [PATCH 02/11] repository: introduce repo_read_index_or_die Stefan Beller
@ 2018-05-16 22:21         ` Stefan Beller
  2018-05-16 22:21         ` [PATCH 05/11] builtin/ls-files: " Stefan Beller
                           ` (2 subsequent siblings)
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-16 22:21 UTC (permalink / raw)
  To: pclouds; +Cc: ao2, bmwill, git, gitster, sbeller

The code is used by the fetch command, which is a main porcelain command,
so we should localize its error messages.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/submodule.c b/submodule.c
index 74d35b25779..71c042e1371 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1333,8 +1333,7 @@ int fetch_populated_submodules(struct repository *r,
 	if (!r->worktree)
 		goto out;
 
-	if (repo_read_index(r) < 0)
-		die("index file corrupt");
+	repo_read_index_or_die(r);
 
 	argv_array_push(&spf.args, "fetch");
 	for (i = 0; i < options->argc; i++)
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 13%]

* Re: [PATCH 11/11] read_cache: convert most calls to repo_read_index_or_die
  2018-05-16 22:21         ` [PATCH 11/11] read_cache: convert most calls to repo_read_index_or_die Stefan Beller
@ 2018-05-16 22:27           ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2018-05-16 22:27 UTC (permalink / raw)
  To: Stefan Beller; +Cc: pclouds, ao2, git, gitster

On 05/16, Stefan Beller wrote:
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  blame.c               | 5 +++--
>  builtin/am.c          | 3 ++-
>  builtin/diff.c        | 3 ++-
>  builtin/fsck.c        | 3 ++-
>  builtin/merge-index.c | 3 ++-
>  check-racy.c          | 2 +-
>  diff.c                | 5 +++--
>  merge-recursive.c     | 3 ++-
>  revision.c            | 5 +++--
>  sequencer.c           | 5 +++--
>  sha1-name.c           | 2 +-
>  11 files changed, 24 insertions(+), 15 deletions(-)
> 
> diff --git a/blame.c b/blame.c
> index 78c9808bd1a..ebfa1c8efcd 100644
> --- a/blame.c
> +++ b/blame.c
> @@ -5,6 +5,7 @@
>  #include "diff.h"
>  #include "diffcore.h"
>  #include "tag.h"
> +#include "repository.h"
>  #include "blame.h"
>  
>  void blame_origin_decref(struct blame_origin *o)
> @@ -159,7 +160,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
>  	unsigned mode;
>  	struct strbuf msg = STRBUF_INIT;
>  
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  	time(&now);
>  	commit = alloc_commit_node();
>  	commit->object.parsed = 1;
> @@ -241,7 +242,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,
>  	 * want to run "diff-index --cached".
>  	 */
>  	discard_cache();
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  
>  	len = strlen(path);
>  	if (!mode) {
> diff --git a/builtin/am.c b/builtin/am.c
> index d834f9e62b6..3c6e77a5369 100644
> --- a/builtin/am.c
> +++ b/builtin/am.c
> @@ -32,6 +32,7 @@
>  #include "apply.h"
>  #include "string-list.h"
>  #include "packfile.h"
> +#include "repository.h"
>  
>  /**
>   * Returns 1 if the file is empty or does not exist, 0 otherwise.
> @@ -1581,7 +1582,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
>  	say(state, stdout, _("Falling back to patching base and 3-way merge..."));
>  
>  	discard_cache();
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  
>  	/*
>  	 * This is not so wrong. Depending on which base we picked, orig_tree
> diff --git a/builtin/diff.c b/builtin/diff.c
> index 16bfb22f738..4bba211f1c7 100644
> --- a/builtin/diff.c
> +++ b/builtin/diff.c
> @@ -17,6 +17,7 @@
>  #include "builtin.h"
>  #include "submodule.h"
>  #include "sha1-array.h"
> +#include "repository.h"
>  
>  #define DIFF_NO_INDEX_EXPLICIT 1
>  #define DIFF_NO_INDEX_IMPLICIT 2
> @@ -210,7 +211,7 @@ static void refresh_index_quietly(void)
>  	if (fd < 0)
>  		return;
>  	discard_cache();
> -	read_cache();
> +	repo_read_index(the_repository); /* do not die on error */
>  	refresh_cache(REFRESH_QUIET|REFRESH_UNMERGED);
>  	update_index_if_able(&the_index, &lock_file);
>  }
> diff --git a/builtin/fsck.c b/builtin/fsck.c
> index 087360a6757..a42e98235da 100644
> --- a/builtin/fsck.c
> +++ b/builtin/fsck.c
> @@ -18,6 +18,7 @@
>  #include "decorate.h"
>  #include "packfile.h"
>  #include "object-store.h"
> +#include "repository.h"
>  
>  #define REACHABLE 0x0001
>  #define SEEN      0x0002
> @@ -795,7 +796,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)
>  	if (keep_cache_objects) {
>  		verify_index_checksum = 1;
>  		verify_ce_order = 1;
> -		read_cache();
> +		repo_read_index_or_die(the_repository);
>  		for (i = 0; i < active_nr; i++) {
>  			unsigned int mode;
>  			struct blob *blob;
> diff --git a/builtin/merge-index.c b/builtin/merge-index.c
> index c99443b095b..2d91c7c3b5e 100644
> --- a/builtin/merge-index.c
> +++ b/builtin/merge-index.c
> @@ -1,5 +1,6 @@
>  #include "builtin.h"
>  #include "run-command.h"
> +#include "repository.h"
>  
>  static const char *pgm;
>  static int one_shot, quiet;
> @@ -77,7 +78,7 @@ int cmd_merge_index(int argc, const char **argv, const char *prefix)
>  	if (argc < 3)
>  		usage("git merge-index [-o] [-q] <merge-program> (-a | [--] [<filename>...])");
>  
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  
>  	i = 1;
>  	if (!strcmp(argv[i], "-o")) {
> diff --git a/check-racy.c b/check-racy.c
> index 24b6542352a..9b884639cf4 100644
> --- a/check-racy.c
> +++ b/check-racy.c
> @@ -6,7 +6,7 @@ int main(int ac, char **av)
>  	int dirty, clean, racy;
>  
>  	dirty = clean = racy = 0;
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  	for (i = 0; i < active_nr; i++) {
>  		struct cache_entry *ce = active_cache[i];
>  		struct stat st;
> diff --git a/diff.c b/diff.c
> index 1289df4b1f9..383f52fa118 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -22,6 +22,7 @@
>  #include "argv-array.h"
>  #include "graph.h"
>  #include "packfile.h"
> +#include "repository.h"
>  
>  #ifdef NO_FAST_WORKING_DIRECTORY
>  #define FAST_WORKING_DIRECTORY 0
> @@ -4210,13 +4211,13 @@ void diff_setup_done(struct diff_options *options)
>  		options->rename_limit = diff_rename_limit_default;
>  	if (options->setup & DIFF_SETUP_USE_CACHE) {
>  		if (!active_cache)
> -			/* read-cache does not die even when it fails
> +			/* repo_read_indexe does not die even when it fails

s/repo_read_indexe/repo_read_index

>  			 * so it is safe for us to do this here.  Also
>  			 * it does not smudge active_cache or active_nr
>  			 * when it fails, so we do not have to worry about
>  			 * cleaning it up ourselves either.
>  			 */
> -			read_cache();
> +			repo_read_index(the_repository);
>  	}
>  	if (40 < options->abbrev)
>  		options->abbrev = 40; /* full */
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 0c0d48624da..76911c935c3 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2151,7 +2151,8 @@ int merge_recursive(struct merge_options *o,
>  
>  	discard_cache();
>  	if (!o->call_depth)
> -		read_cache();
> +		if (read_cache() < 0)
> +			return err(o, _("index file corrupt"));

if we're already moving to change read_cache() calls then we could
substitute it for repo_read_index here as well.

Maybe we can kill read_cache() while at it?

>  
>  	o->ancestor = "merged common ancestors";
>  	clean = merge_trees(o, h1->tree, h2->tree, merged_common_ancestors->tree,
> diff --git a/revision.c b/revision.c
> index 1cff11833e7..8ad9824143d 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -23,6 +23,7 @@
>  #include "packfile.h"
>  #include "worktree.h"
>  #include "argv-array.h"
> +#include "repository.h"
>  
>  volatile show_early_output_fn_t show_early_output;
>  
> @@ -1344,7 +1345,7 @@ void add_index_objects_to_pending(struct rev_info *revs, unsigned int flags)
>  {
>  	struct worktree **worktrees, **p;
>  
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  	do_add_index_objects_to_pending(revs, &the_index);
>  
>  	if (revs->single_worktree)
> @@ -1486,7 +1487,7 @@ static void prepare_show_merge(struct rev_info *revs)
>  	head->object.flags |= SYMMETRIC_LEFT;
>  
>  	if (!active_nr)
> -		read_cache();
> +		repo_read_index_or_die(the_repository);
>  	for (i = 0; i < active_nr; i++) {
>  		const struct cache_entry *ce = active_cache[i];
>  		if (!ce_stage(ce))
> diff --git a/sequencer.c b/sequencer.c
> index 4ce5120e777..773165c8cde 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -23,6 +23,7 @@
>  #include "hashmap.h"
>  #include "notes-utils.h"
>  #include "sigchain.h"
> +#include "repository.h"
>  
>  #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>  
> @@ -432,7 +433,7 @@ static int fast_forward_to(const struct object_id *to, const struct object_id *f
>  	struct strbuf sb = STRBUF_INIT;
>  	struct strbuf err = STRBUF_INIT;
>  
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  	if (checkout_fast_forward(from, to, 1))
>  		return -1; /* the callee should have complained already */
>  
> @@ -489,7 +490,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
>  	if (hold_locked_index(&index_lock, LOCK_REPORT_ON_ERROR) < 0)
>  		return -1;
>  
> -	read_cache();
> +	repo_read_index_or_die(the_repository);
>  
>  	init_merge_options(&o);
>  	o.ancestor = base ? base_label : "(empty tree)";
> diff --git a/sha1-name.c b/sha1-name.c
> index 5b93bf8da36..83d5f945cf1 100644
> --- a/sha1-name.c
> +++ b/sha1-name.c
> @@ -1639,7 +1639,7 @@ static int get_oid_with_context_1(const char *name,
>  			oc->path = xstrdup(cp);
>  
>  		if (!active_cache)
> -			read_cache();
> +			repo_read_index_or_die(the_repository);
>  		pos = cache_name_pos(cp, namelen);
>  		if (pos < 0)
>  			pos = -pos - 1;
> -- 
> 2.17.0.582.gccdcbd54c44.dirty
> 

-- 
Brandon Williams

^ permalink raw reply	[relevance 2%]

* Re: What's cooking in git.git (May 2018, #02; Thu, 17)
      [irrelevant] <xmqqvabm6csb.fsf@gitster-ct.c.googlers.com>
@ 2018-05-17 18:20 ` Stefan Beller
  2018-05-17 18:40   ` [PATCH] merge-recursive: give notice when submodule commit gets fast-forwarded Stefan Beller
                     ` (2 more replies)
      [irrelevant] ` <87lgc77wc7.fsf@evledraar.gmail.com>
  1 sibling, 3 replies; 200+ results
From: Stefan Beller @ 2018-05-17 18:20 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

> * sb/object-store-replace (2018-05-10) 2 commits
>   (merged to 'next' on 2018-05-16 at 41bbedcc81)
>  + replace-object.c: remove the_repository from prepare_replace_object
>  + object.c: free replace map in raw_object_store_clear
>
>  Hotfix.
>
>  Will merge to 'master'.

Please do not.
(Or do, but then be prepared for another hotfix.)

The commit sb/object-store-replace^ needs more free'ing and shall be
replaced with
https://public-inbox.org/git/20180510195849.28023-4-sbeller@google.com/
I'll resend shortly.

>
> * sb/submodule-merge-in-merge-recursive (2018-05-16) 3 commits
>  - merge-recursive: give notice when submodule commit gets fast-forwarded
>  - merge-recursive: i18n submodule merge output and respect verbosity
>  - submodule.c: move submodule merging to merge-recursive.c
>
>  By code restructuring of submodule merge in merge-recursive,
>  informational messages from the codepath are now given using the
>  same mechanism as other output, and honor the merge.verbosity
>  configuration.  The code also learned to give a few new messages
>  when a submodule three-way merge resolves cleanly when one side
>  records a descendant of the commit chosen by the other side.
>
>  Will merge to 'next'.

Merging would be ok, but I would rather not.
A resend will be only for cosmetic effect, as I messed up the last commit

So, please hold in pu.

> * sb/diff-color-move-more (2018-04-25) 7 commits
>  - diff.c: add --color-moved-ignore-space-delta option
>  - diff.c: decouple white space treatment from move detection algorithm
>  - diff.c: add a blocks mode for moved code detection
>  - diff.c: adjust hash function signature to match hashmap expectation
>  - diff.c: do not pass diff options as keydata to hashmap
>  - xdiff/xdiffi.c: remove unneeded function declarations
>  - xdiff/xdiff.h: remove unused flags
>
>  "git diff --color-moved" feature has further been tweaked.
>
>  Will merge to 'next'.

I did not get around to fix it up, there are still review
comments outstanding. (The test is broken in the last commit.)

Please hold in pu;

Thanks,
Stefan

^ permalink raw reply	[relevance 2%]

* [PATCH] merge-recursive: give notice when submodule commit gets fast-forwarded
  2018-05-17 18:20 ` What's cooking in git.git (May 2018, #02; Thu, 17) Stefan Beller
@ 2018-05-17 18:40   ` Stefan Beller
  2018-05-18 19:43     ` [PATCH v2 0/1] rebased: inform about auto submodule ff Leif Middelschulte
  2018-05-18 19:48     ` [PATCH v3 " Leif Middelschulte
      [irrelevant]   ` <20180517194653.48928-1-sbeller@google.com>
  2018-05-17 22:36   ` What's cooking in git.git (May 2018, #02; Thu, 17) Junio C Hamano
  2 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2018-05-17 18:40 UTC (permalink / raw)
  To: leif.middelschulte; +Cc: git, gitster, Leif Middelschulte, Stefan Beller

From: Leif Middelschulte <Leif.Middelschulte@gmail.com>

Inform the user about an automatically fast-forwarded submodule. The
silent merge behavior was introduced by commit 68d03e4a6e44 ("Implement
automatic fast-forward merge for submodules", 2010-07-07)).

Signed-off-by: Leif Middelschulte <Leif.Middelschulte@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 merge-recursive.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

>> * sb/submodule-merge-in-merge-recursive (2018-05-16) 3 commits
>>  - merge-recursive: give notice when submodule commit gets fast-forwarded
>>  - merge-recursive: i18n submodule merge output and respect verbosity
>>  - submodule.c: move submodule merging to merge-recursive.c
>>
>>  By code restructuring of submodule merge in merge-recursive,
>>  informational messages from the codepath are now given using the
>>  same mechanism as other output, and honor the merge.verbosity
>>  configuration.  The code also learned to give a few new messages
>>  when a submodule three-way merge resolves cleanly when one side
>>  records a descendant of the commit chosen by the other side.
>>
>>  Will merge to 'next'.
>
>Merging would be ok, but I would rather not.
>A resend will be only for cosmetic effect, as I messed up the last commit
>
>So, please hold in pu.

This is the resend, with an interdiff as follows.

Junio wrote in http://public-inbox.org/git/xmqqk1s474vx.fsf@gitster-ct.c.googlers.com:

> Perhaps Leif can elaborate why this change is a good idea in the
> first place?

which is also outstanding.

Leif can you pick this patch and resend it with a proper commit message?


    # diff --git c/merge-recursive.c w/merge-recursive.c
    # index 29a430c418a..a9aecccb8c3 100644
    # --- c/merge-recursive.c
    # +++ w/merge-recursive.c
    # @@ -1094,7 +1094,7 @@ static int merge_submodule(struct merge_options *o,
    #  	if (in_merge_bases(commit_a, commit_b)) {
    #  		oidcpy(result, b);
    #  		if (show(o, 3)) {
    # -			output(o, 1, _("Fast-forwarding submodule %s to the following commit:"), path);
    # +			output(o, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
    #  			output_commit_title(o, commit_b);
    #  		} else if (show(o, 2))
    #  			output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(b));
    # @@ -1106,7 +1106,7 @@ static int merge_submodule(struct merge_options *o,
    #  	if (in_merge_bases(commit_b, commit_a)) {
    #  		oidcpy(result, a);
    #  		if (show(o, 3)) {
    # -			output(o, 1, _("Fast-forwarding submodule %s to the following commit:"), path);
    # +			output(o, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
    #  			output_commit_title(o, commit_a);
    #  		} else if (show(o, 2))
    #  			output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(a));


diff --git a/merge-recursive.c b/merge-recursive.c
index 0571919ee0a..a9aecccb8c3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1093,10 +1093,26 @@ static int merge_submodule(struct merge_options *o,
 	/* Case #1: a is contained in b or vice versa */
 	if (in_merge_bases(commit_a, commit_b)) {
 		oidcpy(result, b);
+		if (show(o, 3)) {
+			output(o, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
+			output_commit_title(o, commit_b);
+		} else if (show(o, 2))
+			output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(b));
+		else
+			; /* no output */
+
 		return 1;
 	}
 	if (in_merge_bases(commit_b, commit_a)) {
 		oidcpy(result, a);
+		if (show(o, 3)) {
+			output(o, 3, _("Fast-forwarding submodule %s to the following commit:"), path);
+			output_commit_title(o, commit_a);
+		} else if (show(o, 2))
+			output(o, 2, _("Fast-forwarding submodule %s to %s"), path, oid_to_hex(a));
+		else
+			; /* no output */
+
 		return 1;
 	}
 
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 13%]

* [PATCH 6/8] diff.c: decouple white space treatment from move detection algorithm
      [irrelevant]   ` <20180517194653.48928-1-sbeller@google.com>
@ 2018-05-17 19:46     ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-17 19:46 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, jonathantanmy

In the original implementation of the move detection logic the choice for
ignoring white space changes is the same for the move detection as it is
for the regular diff.  Some cases came up where different treatment would
have been nice.

Allow the user to specify that whitespace should be ignored differently
during detection of moved lines than during generation of added and removed
lines. This is done by providing analogs to the --ignore-space-at-eol,
-b, and -w options (namely,
  --color-moved-[no-]ignore-space-at-eol
  --color-moved-[no-]ignore-space-change
  --color-moved-[no-]ignore-all-space) that affect only the color of the
output, and making the existing --ignore-space-at-eol, -b, and -w options
no longer affect the color of the output.

As we change the default, we'll adjust the tests.

For now we do not infer any options to treat whitespaces in the move
detection from the generic white space options given to diff.
This can be tuned later to reasonable default.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/diff-options.txt | 13 +++++
 diff.c                         | 19 ++++++-
 diff.h                         |  1 +
 t/t4015-diff-whitespace.sh     | 90 +++++++++++++++++++++++++++++++---
 4 files changed, 114 insertions(+), 9 deletions(-)

diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index bb9f1b7cd82..7b2527b9a19 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -292,6 +292,19 @@ dimmed_zebra::
 	blocks are considered interesting, the rest is uninteresting.
 --
 
+--color-moved-[no-]ignore-space-at-eol::
+	Ignore changes in whitespace at EOL when performing the move
+	detection for --color-moved.
+--color-moved-[no-]ignore-space-change::
+	Ignore changes in amount of whitespace when performing the move
+	detection for --color-moved.  This ignores whitespace
+	at line end, and considers all other sequences of one or
+	more whitespace characters to be equivalent.
+--color-moved-[no-]ignore-all-space::
+	Ignore whitespace when comparing lines when performing the move
+	detection for --color-moved.  This ignores differences even if
+	one line has whitespace where the other line has none.
+
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
 	By default, words are delimited by whitespace; see
diff --git a/diff.c b/diff.c
index 95c51c0b7df..b5819dd538f 100644
--- a/diff.c
+++ b/diff.c
@@ -717,10 +717,12 @@ static int moved_entry_cmp(const void *hashmap_cmp_fn_data,
 	const struct diff_options *diffopt = hashmap_cmp_fn_data;
 	const struct moved_entry *a = entry;
 	const struct moved_entry *b = entry_or_key;
+	unsigned flags = diffopt->color_moved_ws_handling
+			 & XDF_WHITESPACE_FLAGS;
 
 	return !xdiff_compare_lines(a->es->line, a->es->len,
 				    b->es->line, b->es->len,
-				    diffopt->xdl_opts);
+				    flags);
 }
 
 static struct moved_entry *prepare_entry(struct diff_options *o,
@@ -728,8 +730,9 @@ static struct moved_entry *prepare_entry(struct diff_options *o,
 {
 	struct moved_entry *ret = xmalloc(sizeof(*ret));
 	struct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];
+	unsigned flags = o->color_moved_ws_handling & XDF_WHITESPACE_FLAGS;
 
-	ret->ent.hash = xdiff_hash_string(l->line, l->len, o->xdl_opts);
+	ret->ent.hash = xdiff_hash_string(l->line, l->len, flags);
 	ret->es = l;
 	ret->next_line = NULL;
 
@@ -4638,6 +4641,18 @@ int diff_opt_parse(struct diff_options *options,
 		DIFF_XDL_SET(options, IGNORE_CR_AT_EOL);
 	else if (!strcmp(arg, "--ignore-blank-lines"))
 		DIFF_XDL_SET(options, IGNORE_BLANK_LINES);
+	else if (!strcmp(arg, "--color-moved-no-ignore-all-space"))
+		options->color_moved_ws_handling &= ~XDF_IGNORE_WHITESPACE;
+	else if (!strcmp(arg, "--color-moved-no-ignore-space-change"))
+		options->color_moved_ws_handling &= ~XDF_IGNORE_WHITESPACE_CHANGE;
+	else if (!strcmp(arg, "--color-moved-no-ignore-space-at-eol"))
+		options->color_moved_ws_handling &= ~XDF_IGNORE_WHITESPACE_AT_EOL;
+	else if (!strcmp(arg, "--color-moved-ignore-all-space"))
+		options->color_moved_ws_handling |= XDF_IGNORE_WHITESPACE;
+	else if (!strcmp(arg, "--color-moved-ignore-space-change"))
+		options->color_moved_ws_handling |= XDF_IGNORE_WHITESPACE_CHANGE;
+	else if (!strcmp(arg, "--color-moved-ignore-space-at-eol"))
+		options->color_moved_ws_handling |= XDF_IGNORE_WHITESPACE_AT_EOL;
 	else if (!strcmp(arg, "--indent-heuristic"))
 		DIFF_XDL_SET(options, INDENT_HEURISTIC);
 	else if (!strcmp(arg, "--no-indent-heuristic"))
diff --git a/diff.h b/diff.h
index 7bd4f182c33..de5dc680051 100644
--- a/diff.h
+++ b/diff.h
@@ -214,6 +214,7 @@ struct diff_options {
 	} color_moved;
 	#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA
 	#define COLOR_MOVED_MIN_ALNUM_COUNT 20
+	int color_moved_ws_handling;
 };
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line);
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 45091abb192..751fc478dde 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -1441,7 +1441,10 @@ test_expect_success 'move detection ignoring whitespace ' '
 	line 4
 	line 5
 	EOF
-	git diff HEAD --no-renames --color-moved --color |
+	git diff HEAD --no-renames --color-moved --color \
+		--color-moved-no-ignore-all-space \
+		--color-moved-no-ignore-space-change \
+		--color-moved-no-ignore-space-at-eol |
 		grep -v "index" |
 		test_decode_color >actual &&
 	cat <<-\EOF >expected &&
@@ -1465,7 +1468,10 @@ test_expect_success 'move detection ignoring whitespace ' '
 	EOF
 	test_cmp expected actual &&
 
-	git diff HEAD --no-renames -w --color-moved --color |
+	git diff HEAD --no-renames --color-moved --color \
+		--color-moved-ignore-all-space \
+		--color-moved-no-ignore-space-change \
+		--color-moved-no-ignore-space-at-eol |
 		grep -v "index" |
 		test_decode_color >actual &&
 	cat <<-\EOF >expected &&
@@ -1505,7 +1511,10 @@ test_expect_success 'move detection ignoring whitespace changes' '
 	line 5
 	EOF
 
-	git diff HEAD --no-renames --color-moved --color |
+	git diff HEAD --no-renames --color-moved --color \
+		--color-moved-no-ignore-all-space \
+		--color-moved-no-ignore-space-change \
+		--color-moved-no-ignore-space-at-eol |
 		grep -v "index" |
 		test_decode_color >actual &&
 	cat <<-\EOF >expected &&
@@ -1529,7 +1538,10 @@ test_expect_success 'move detection ignoring whitespace changes' '
 	EOF
 	test_cmp expected actual &&
 
-	git diff HEAD --no-renames -b --color-moved --color |
+	git diff HEAD --no-renames --color-moved --color \
+		--color-moved-no-ignore-all-space \
+		--color-moved-no-ignore-space-at-eol \
+		--color-moved-ignore-space-change |
 		grep -v "index" |
 		test_decode_color >actual &&
 	cat <<-\EOF >expected &&
@@ -1572,7 +1584,10 @@ test_expect_success 'move detection ignoring whitespace at eol' '
 	# avoid cluttering the output with complaints about our eol whitespace
 	test_config core.whitespace -blank-at-eol &&
 
-	git diff HEAD --no-renames --color-moved --color |
+	git diff HEAD --no-renames --color-moved --color \
+		--color-moved-no-ignore-all-space \
+		--color-moved-no-ignore-space-change \
+		--color-moved-no-ignore-space-at-eol |
 		grep -v "index" |
 		test_decode_color >actual &&
 	cat <<-\EOF >expected &&
@@ -1596,7 +1611,10 @@ test_expect_success 'move detection ignoring whitespace at eol' '
 	EOF
 	test_cmp expected actual &&
 
-	git diff HEAD --no-renames --ignore-space-at-eol --color-moved --color |
+	git diff HEAD --no-renames --color-moved --color \
+		--color-moved-no-ignore-all-space \
+		--color-moved-no-ignore-space-change \
+		--color-moved-ignore-space-at-eol |
 		grep -v "index" |
 		test_decode_color >actual &&
 	cat <<-\EOF >expected &&
@@ -1768,7 +1786,65 @@ test_expect_success 'move detection with submodules' '
 
 	# nor did we mess with it another way
 	git diff --submodule=diff --color | test_decode_color >expect &&
-	test_cmp expect decoded_actual
+	test_cmp expect decoded_actual &&
+	rm -rf bananas &&
+	git submodule deinit bananas
+'
+
+test_expect_success 'only move detection ignores white spaces' '
+	git reset --hard &&
+	q_to_tab <<-\EOF >text.txt &&
+		a long line to exceed per-line minimum
+		another long line to exceed per-line minimum
+		original file
+	EOF
+	git add text.txt &&
+	git commit -m "add text" &&
+	q_to_tab <<-\EOF >text.txt &&
+		Qa long line to exceed per-line minimum
+		Qanother long line to exceed per-line minimum
+		new file
+	EOF
+
+	# Make sure we get a different diff using -w
+	git diff --color --color-moved -w \
+		--color-moved-no-ignore-all-space \
+		--color-moved-no-ignore-space-change \
+		--color-moved-no-ignore-space-at-eol |
+		grep -v "index" |
+		test_decode_color >actual &&
+	q_to_tab <<-\EOF >expected &&
+	<BOLD>diff --git a/text.txt b/text.txt<RESET>
+	<BOLD>--- a/text.txt<RESET>
+	<BOLD>+++ b/text.txt<RESET>
+	<CYAN>@@ -1,3 +1,3 @@<RESET>
+	 Qa long line to exceed per-line minimum<RESET>
+	 Qanother long line to exceed per-line minimum<RESET>
+	<RED>-original file<RESET>
+	<GREEN>+<RESET><GREEN>new file<RESET>
+	EOF
+	test_cmp expected actual &&
+
+	# And now ignoring white space only in the move detection
+	git diff --color --color-moved \
+		--color-moved-ignore-all-space \
+		--color-moved-ignore-space-change \
+		--color-moved-ignore-space-at-eol |
+		grep -v "index" |
+		test_decode_color >actual &&
+	q_to_tab <<-\EOF >expected &&
+	<BOLD>diff --git a/text.txt b/text.txt<RESET>
+	<BOLD>--- a/text.txt<RESET>
+	<BOLD>+++ b/text.txt<RESET>
+	<CYAN>@@ -1,3 +1,3 @@<RESET>
+	<BOLD;MAGENTA>-a long line to exceed per-line minimum<RESET>
+	<BOLD;MAGENTA>-another long line to exceed per-line minimum<RESET>
+	<RED>-original file<RESET>
+	<BOLD;YELLOW>+<RESET>Q<BOLD;YELLOW>a long line to exceed per-line minimum<RESET>
+	<BOLD;YELLOW>+<RESET>Q<BOLD;YELLOW>another long line to exceed per-line minimum<RESET>
+	<GREEN>+<RESET><GREEN>new file<RESET>
+	EOF
+	test_cmp expected actual
 '
 
 test_done
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 6%]

* Re: What's cooking in git.git (May 2018, #02; Thu, 17)
  2018-05-17 18:20 ` What's cooking in git.git (May 2018, #02; Thu, 17) Stefan Beller
  2018-05-17 18:40   ` [PATCH] merge-recursive: give notice when submodule commit gets fast-forwarded Stefan Beller
      [irrelevant]   ` <20180517194653.48928-1-sbeller@google.com>
@ 2018-05-17 22:36   ` Junio C Hamano
  2 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-05-17 22:36 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git

Stefan Beller <sbeller@google.com> writes:

>> * sb/object-store-replace (2018-05-10) 2 commits
>>   (merged to 'next' on 2018-05-16 at 41bbedcc81)
>>  + replace-object.c: remove the_repository from prepare_replace_object
>>  + object.c: free replace map in raw_object_store_clear
>>
>>  Hotfix.
>>
>>  Will merge to 'master'.
>
> Please do not.
> (Or do, but then be prepared for another hotfix.)
>
> The commit sb/object-store-replace^ needs more free'ing and shall be
> replaced with

Please do not replace what already hit 'next'.

> https://public-inbox.org/git/20180510195849.28023-4-sbeller@google.com/
> I'll resend shortly.
>
>>
>> * sb/submodule-merge-in-merge-recursive (2018-05-16) 3 commits
>>  - merge-recursive: give notice when submodule commit gets fast-forwarded
>>  - merge-recursive: i18n submodule merge output and respect verbosity
>>  - submodule.c: move submodule merging to merge-recursive.c
>>
>>  By code restructuring of submodule merge in merge-recursive,
>>  informational messages from the codepath are now given using the
>>  same mechanism as other output, and honor the merge.verbosity
>>  configuration.  The code also learned to give a few new messages
>>  when a submodule three-way merge resolves cleanly when one side
>>  records a descendant of the commit chosen by the other side.
>>
>>  Will merge to 'next'.
>
> Merging would be ok, but I would rather not.
> A resend will be only for cosmetic effect, as I messed up the last commit
>
> So, please hold in pu.
>
>> * sb/diff-color-move-more (2018-04-25) 7 commits
>>  - diff.c: add --color-moved-ignore-space-delta option
>>  - diff.c: decouple white space treatment from move detection algorithm
>>  - diff.c: add a blocks mode for moved code detection
>>  - diff.c: adjust hash function signature to match hashmap expectation
>>  - diff.c: do not pass diff options as keydata to hashmap
>>  - xdiff/xdiffi.c: remove unneeded function declarations
>>  - xdiff/xdiff.h: remove unused flags
>>
>>  "git diff --color-moved" feature has further been tweaked.
>>
>>  Will merge to 'next'.
>
> I did not get around to fix it up, there are still review
> comments outstanding. (The test is broken in the last commit.)
>
> Please hold in pu;
>
> Thanks,
> Stefan

^ permalink raw reply	[relevance 2%]

* [PATCH 00/19] object store: grafts and shallow.
  2018-05-15 23:42 [RFC PATCH 00/19] object store: grafts and shallow Stefan Beller
  2018-05-15 23:42 ` [PATCH 01/19] object-store: move object access functions to object-store.h Stefan Beller
@ 2018-05-17 22:51 ` Stefan Beller
  2018-05-17 22:51   ` [PATCH 01/19] object-store: move object access functions to object-store.h Stefan Beller
  1 sibling, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-17 22:51 UTC (permalink / raw)
  To: sbeller; +Cc: git

v1:

This reroll:
* includes the fixup by Ramsay
  http://public-inbox.org/git/ae96f1c4-add2-d9d8-f08b-a765fe277232@ramsayjones.plus.com
* fixes commit messages
* changes the macro for the global git_path definitions such that we need fewer
  characters (omits the duplicates "path_" that were the same for every function)
* the range diff is below
  (per https://public-inbox.org/git/20180508034815.GB7242@sigill.intra.peff.net/)

Thanks,
Stefan

v0/RFC:
This applies on top of sb/object-store-alloc[1], and is the next part of the
object store series. 

I think we're getting close to actually being done in the object store series,
as the next series to build on top of this will convert the
lookup_{object, commit, ...} functions.

However I marked this series as RFC, as I expect heavy conflicts with the
code base. sb/object-store-alloc builds on an older base of the code base
and there have been some series that will conflict with this one.
For example the patch to migrate path functions into a repository world
will collide with one of Dschos series as he added another path helper
function there. I am happy to reroll on top of that, but for now I chose
sb/object-store-alloc to keep the momentum in the object-store series'.

There is another object store series that is not part of the critical path:
There is a mem leak in the pack files stored in the raw object store.
Upon free'ing the repository we do not free its pack files. We cannot
free the packfiles, as otherwise the bitmap code would have dangling
pointers into packfiles that no longer exists, leading to segfaults.
So we'll need to have an object store series covering the bitmap code.
I have something local, which I'll send out shortly.

Thanks,
Stefan

[1] with the latest patch replaced as in
    https://public-inbox.org/git/20180515214842.108713-1-sbeller@google.com/

Brandon Williams (3):
  commit: convert commit_graft_pos() to handle arbitrary repositories
  commit: convert register_commit_graft to handle arbitrary repositories
  commit: convert read_graft_file to handle arbitrary repositories

Jonathan Nieder (6):
  object: move grafts to object parser
  commit: add repository argument to commit_graft_pos
  commit: add repository argument to register_commit_graft
  commit: add repository argument to read_graft_file
  commit: add repository argument to prepare_commit_graft
  commit: add repository argument to lookup_commit_graft

Stefan Beller (10):
  object-store: move object access functions to object-store.h
  shallow: add repository argument to set_alternate_shallow_file
  shallow: add repository argument to register_shallow
  shallow: add repository argument to check_shallow_file_for_update
  shallow: add repository argument to is_repository_shallow
  cache: convert get_graft_file to handle arbitrary repositories
  path.c: migrate global git_path_* to take a repository argument
  shallow: migrate shallow information into the object parser
  commit: allow prepare_commit_graft to handle arbitrary repositories
  commit: allow lookup_commit_graft to handle arbitrary repositories

 apply.c                  |   1 +
 archive-tar.c            |   1 +
 archive-zip.c            |   1 +
 archive.c                |   1 +
 blame.c                  |   9 ++-
 branch.c                 |  14 ++---
 builtin/blame.c          |   4 +-
 builtin/cat-file.c       |   1 +
 builtin/checkout.c       |   1 +
 builtin/clone.c          |   1 +
 builtin/commit-tree.c    |   1 +
 builtin/commit.c         |  38 ++++++-------
 builtin/describe.c       |   1 +
 builtin/difftool.c       |   1 +
 builtin/fast-export.c    |   1 +
 builtin/fetch.c          |   7 ++-
 builtin/fmt-merge-msg.c  |   1 +
 builtin/hash-object.c    |   1 +
 builtin/log.c            |   1 +
 builtin/ls-tree.c        |   1 +
 builtin/merge-tree.c     |   1 +
 builtin/merge.c          |  37 ++++++------
 builtin/mktag.c          |   1 +
 builtin/mktree.c         |   1 +
 builtin/notes.c          |   1 +
 builtin/pack-objects.c   |   6 +-
 builtin/prune.c          |   3 +-
 builtin/pull.c           |   4 +-
 builtin/receive-pack.c   |   3 +-
 builtin/reflog.c         |   1 +
 builtin/remote.c         |   1 +
 builtin/reset.c          |   2 +-
 builtin/rev-list.c       |   1 +
 builtin/rev-parse.c      |   3 +-
 builtin/show-ref.c       |   1 +
 builtin/tag.c            |   1 +
 builtin/unpack-file.c    |   1 +
 builtin/unpack-objects.c |   1 +
 builtin/verify-commit.c  |   1 +
 bulk-checkin.c           |   1 +
 bundle.c                 |   1 +
 cache-tree.c             |   1 +
 cache.h                  | 119 +--------------------------------------
 combine-diff.c           |   1 +
 commit.c                 |  77 +++++++++++++------------
 commit.h                 |  10 ++--
 config.c                 |   1 +
 convert.c                |   1 +
 diff.c                   |   1 +
 diffcore-rename.c        |   1 +
 dir.c                    |   1 +
 entry.c                  |   1 +
 environment.c            |   8 +--
 fetch-pack.c             |   9 +--
 fsck.c                   |   3 +-
 git.c                    |   2 +-
 grep.c                   |   1 +
 list-objects-filter.c    |   1 +
 list-objects.c           |   1 +
 log-tree.c               |   1 +
 mailmap.c                |   1 +
 match-trees.c            |   1 +
 merge-blobs.c            |   1 +
 merge-recursive.c        |   1 +
 notes-cache.c            |   1 +
 notes-merge.c            |   1 +
 notes.c                  |   1 +
 object-store.h           | 117 ++++++++++++++++++++++++++++++++++++++
 object.c                 |   4 ++
 object.h                 |  10 ++++
 pack-bitmap-write.c      |   1 +
 packfile.h               |   5 ++
 path.c                   |  18 +++---
 path.h                   |  40 ++++++++++---
 read-cache.c             |   1 +
 ref-filter.c             |   1 +
 refs.c                   |   1 +
 remote-testsvn.c         |   1 +
 remote.c                 |   1 +
 repository.h             |   5 ++
 rerere.c                 |   8 ++-
 revision.c               |   1 +
 send-pack.c              |   7 ++-
 sequencer.c              |  38 +++++++------
 shallow.c                |  74 ++++++++++++------------
 submodule-config.c       |   1 +
 tag.c                    |   1 +
 tree-walk.c              |   1 +
 tree.c                   |   1 +
 unpack-trees.c           |   1 +
 upload-pack.c            |  10 ++--
 walker.c                 |   1 +
 wt-status.c              |   8 +--
 xdiff-interface.c        |   1 +
 94 files changed, 449 insertions(+), 315 deletions(-)

-- 
2.17.0.582.gccdcbd54c44.dirty

1:  cbd53a2193d = 1:  20b6bef31b7 object-store: move object access functions to object-store.h
2:  6a1a79fd146 = 2:  6855b203842 object: move grafts to object parser
3:  be479e801da = 3:  0afdfac30da commit: add repository argument to commit_graft_pos
4:  3f5787f8066 = 4:  af84c3bc1dd commit: add repository argument to register_commit_graft
5:  02ba3e1a057 = 5:  154ec8bacc1 commit: add repository argument to read_graft_file
6:  3ee37656ee6 = 6:  c12bd3a7870 commit: add repository argument to prepare_commit_graft
7:  e6c898f460c ! 7:  aef9cd4ffba commit: add repository argument to lookup_commit_graft
    @@ -10,9 +10,6 @@
         As with the previous commits, use a macro to catch callers passing a
         repository other than the_repository at compile time.
     
    -    The included coccinelle semantic patch will adapt any new callers in
    -    the diff produced by `make coccicheck`.
    -
         Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
         Signed-off-by: Stefan Beller <sbeller@google.com>
         Signed-off-by: Junio C Hamano <gitster@pobox.com>
8:  f95ee308eb7 ! 8:  0025ae11097 shallow: add repository argument to set_alternate_shallow_file
    @@ -2,8 +2,15 @@
     
         shallow: add repository argument to set_alternate_shallow_file
     
    +    Add a repository argument to allow callers of set_alternate_shallow_file
    +    to be more specific about which repository to handle. This is a small
    +    mechanical change; it doesn't change the implementation to handle
    +    repositories other than the_repository yet.
    +
    +    As with the previous commits, use a macro to catch callers passing a
    +    repository other than the_repository at compile time.
    +
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/commit.h b/commit.h
     --- a/commit.h
9:  11948c4278a ! 9:  3ee6c801d16 shallow: add repository argument to register_shallow
    @@ -2,8 +2,15 @@
     
         shallow: add repository argument to register_shallow
     
    +    Add a repository argument to allow callers of register_shallow
    +    to be more specific about which repository to handle. This is a small
    +    mechanical change; it doesn't change the implementation to handle
    +    repositories other than the_repository yet.
    +
    +    As with the previous commits, use a macro to catch callers passing a
    +    repository other than the_repository at compile time.
    +
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
     --- a/builtin/pack-objects.c
10:  c69182008bd ! 10:  f0d9eb428ef shallow: add repository argument to check_shallow_file_for_update
    @@ -2,8 +2,15 @@
     
         shallow: add repository argument to check_shallow_file_for_update
     
    +    Add a repository argument to allow callers of check_shallow_file_for_update
    +    to be more specific about which repository to handle. This is a small
    +    mechanical change; it doesn't change the implementation to handle
    +    repositories other than the_repository yet.
    +
    +    As with the previous commits, use a macro to catch callers passing a
    +    repository other than the_repository at compile time.
    +
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/shallow.c b/shallow.c
     --- a/shallow.c
11:  2194490111e ! 11:  5ba578b5386 shallow: add repository argument to is_repository_shallow
    @@ -2,8 +2,15 @@
     
         shallow: add repository argument to is_repository_shallow
     
    +    Add a repository argument to allow callers of is_repository_shallow
    +    to be more specific about which repository to handle. This is a small
    +    mechanical change; it doesn't change the implementation to handle
    +    repositories other than the_repository yet.
    +
    +    As with the previous commits, use a macro to catch callers passing a
    +    repository other than the_repository at compile time.
    +
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/builtin/fetch.c b/builtin/fetch.c
     --- a/builtin/fetch.c
12:  7560733756e ! 12:  8432e847c05 commit: convert commit_graft_pos() to handle arbitrary repositories
    @@ -4,7 +4,6 @@
     
         Signed-off-by: Brandon Williams <bmwill@google.com>
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/commit.c b/commit.c
     --- a/commit.c
13:  12a4071c6a5 ! 13:  8763c86af52 commit: convert register_commit_graft to handle arbitrary repositories
    @@ -4,7 +4,6 @@
     
         Signed-off-by: Brandon Williams <bmwill@google.com>
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/commit.c b/commit.c
     --- a/commit.c
14:  a333e619158 ! 14:  2eea80787c6 commit: convert read_graft_file to handle arbitrary repositories
    @@ -4,7 +4,6 @@
     
         Signed-off-by: Brandon Williams <bmwill@google.com>
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/commit.c b/commit.c
     --- a/commit.c
15:  5c8f9317393 ! 15:  4f41b5cc4e8 cache: convert get_graft_file to handle arbitrary repositories
    @@ -2,6 +2,10 @@
     
         cache: convert get_graft_file to handle arbitrary repositories
     
    +    This conversion was done without the #define trick used in the earlier
    +    series refactoring to have better repository access, because this function
    +    is easy to review, as all lines are converted and it has only one caller.
    +
         Signed-off-by: Stefan Beller <sbeller@google.com>
         Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
16:  adbe83bff1f ! 16:  77cc05fde8a path.c: migrate git_path_ to take a repository argument
    @@ -1,6 +1,14 @@
     Author: Stefan Beller <sbeller@google.com>
     
    -    path.c: migrate git_path_ to take a repository argument
    +    path.c: migrate global git_path_* to take a repository argument
    +
    +    Migrate all git_path_* functions that are defined in path.c to take a
    +    repository argument. Unlike other patches in this series, do not use the
    +     #define trick, as we rewrite the whole function, which is rather small.
    +
    +    This doesn't migrate all the functions, as other builtins have their own
    +    local path functions defined using GIT_PATH_FUNC. So keep that macro
    +    around to serve the other locations.
     
         Signed-off-by: Stefan Beller <sbeller@google.com>
         Signed-off-by: Junio C Hamano <gitster@pobox.com>
    @@ -374,15 +382,15 @@
     -GIT_PATH_FUNC(git_path_merge_head, "MERGE_HEAD")
     -GIT_PATH_FUNC(git_path_fetch_head, "FETCH_HEAD")
     -GIT_PATH_FUNC(git_path_shallow, "shallow")
    -+REPO_GIT_PATH_FUNC(path_cherry_pick_head, "CHERRY_PICK_HEAD")
    -+REPO_GIT_PATH_FUNC(path_revert_head, "REVERT_HEAD")
    -+REPO_GIT_PATH_FUNC(path_squash_msg, "SQUASH_MSG")
    -+REPO_GIT_PATH_FUNC(path_merge_msg, "MERGE_MSG")
    -+REPO_GIT_PATH_FUNC(path_merge_rr, "MERGE_RR")
    -+REPO_GIT_PATH_FUNC(path_merge_mode, "MERGE_MODE")
    -+REPO_GIT_PATH_FUNC(path_merge_head, "MERGE_HEAD")
    -+REPO_GIT_PATH_FUNC(path_fetch_head, "FETCH_HEAD")
    -+REPO_GIT_PATH_FUNC(path_shallow, "shallow")
    ++REPO_GIT_PATH_FUNC(cherry_pick_head, "CHERRY_PICK_HEAD")
    ++REPO_GIT_PATH_FUNC(revert_head, "REVERT_HEAD")
    ++REPO_GIT_PATH_FUNC(squash_msg, "SQUASH_MSG")
    ++REPO_GIT_PATH_FUNC(merge_msg, "MERGE_MSG")
    ++REPO_GIT_PATH_FUNC(merge_rr, "MERGE_RR")
    ++REPO_GIT_PATH_FUNC(merge_mode, "MERGE_MODE")
    ++REPO_GIT_PATH_FUNC(merge_head, "MERGE_HEAD")
    ++REPO_GIT_PATH_FUNC(fetch_head, "FETCH_HEAD")
    ++REPO_GIT_PATH_FUNC(shallow, "shallow")
     
     diff --git a/path.h b/path.h
     --- a/path.h
    @@ -401,7 +409,7 @@
     -const char *git_path_fetch_head(void);
     -const char *git_path_shallow(void);
     +#define REPO_GIT_PATH_FUNC(var, filename) \
    -+	const char *git_##var(struct repository *r) \
    ++	const char *git_path_##var(struct repository *r) \
     +	{ \
     +		if (!r->cached_paths.var) \
     +			r->cached_paths.var = git_pathdup(filename); \
    @@ -409,15 +417,15 @@
     +	}
     +
     +struct path_cache {
    -+	const char *path_cherry_pick_head;
    -+	const char *path_revert_head;
    -+	const char *path_squash_msg;
    -+	const char *path_merge_msg;
    -+	const char *path_merge_rr;
    -+	const char *path_merge_mode;
    -+	const char *path_merge_head;
    -+	const char *path_fetch_head;
    -+	const char *path_shallow;
    ++	const char *cherry_pick_head;
    ++	const char *revert_head;
    ++	const char *squash_msg;
    ++	const char *merge_msg;
    ++	const char *merge_rr;
    ++	const char *merge_mode;
    ++	const char *merge_head;
    ++	const char *fetch_head;
    ++	const char *shallow;
     +};
     +
     +#define PATH_CACHE_INIT { NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL }
17:  d73b49b7078 ! 17:  d9b50420d32 shallow: migrate shallow information into the object parser
    @@ -2,8 +2,10 @@
     
         shallow: migrate shallow information into the object parser
     
    +    We need to convert the shallow functions all at the same time
    +    as we move the data structures they operate on into the repository.
    +
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/commit.h b/commit.h
     --- a/commit.h
    @@ -72,8 +74,7 @@
     -static int is_shallow = -1;
     -static struct stat_validity shallow_stat;
     -static char *alternate_shallow_file;
    -+struct stat_validity the_repository_shallow_stat;
    - 
    +-
     -void set_alternate_shallow_file_the_repository(const char *path, int override)
     +void set_alternate_shallow_file(struct repository *r, const char *path, int override)
      {
18:  cede061c90f ! 18:  0458d3b9fac commit: allow prepare_commit_graft to handle arbitrary repositories
    @@ -2,8 +2,11 @@
     
         commit: allow prepare_commit_graft to handle arbitrary repositories
     
    +    Move the global variable 'commit_graft_prepared' into the object
    +    pool and convert the function prepare_commit_graft to work
    +    an arbitrary repositories.
    +
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/commit.c b/commit.c
     --- a/commit.c
19:  7dc1693a492 ! 19:  39c2ddce2e1 commit: allow lookup_commit_graft to handle arbitrary repositories
    @@ -3,7 +3,6 @@
         commit: allow lookup_commit_graft to handle arbitrary repositories
     
         Signed-off-by: Stefan Beller <sbeller@google.com>
    -    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
     diff --git a/commit.c b/commit.c
     --- a/commit.c

^ permalink raw reply	[relevance 5%]

* [PATCH 01/19] object-store: move object access functions to object-store.h
  2018-05-17 22:51 ` [PATCH 00/19] object store: grafts and shallow Stefan Beller
@ 2018-05-17 22:51   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-17 22:51 UTC (permalink / raw)
  To: sbeller; +Cc: git, Junio C Hamano

This should make these functions easier to find and cache.h less
overwhelming to read.

In particular, this moves:
- read_object_file
- oid_object_info
- write_object_file

As a result, most of the codebase needs to #include object-store.h.
In this patch the #include is only added to files that would fail to
compile otherwise.  It would be better to #include wherever
identifiers from the header are used.  That can happen later
when we have better tooling for it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 apply.c                  |   1 +
 archive-tar.c            |   1 +
 archive-zip.c            |   1 +
 archive.c                |   1 +
 blame.c                  |   1 +
 builtin/blame.c          |   1 +
 builtin/cat-file.c       |   1 +
 builtin/checkout.c       |   1 +
 builtin/clone.c          |   1 +
 builtin/commit-tree.c    |   1 +
 builtin/describe.c       |   1 +
 builtin/difftool.c       |   1 +
 builtin/fast-export.c    |   1 +
 builtin/fetch.c          |   1 +
 builtin/fmt-merge-msg.c  |   1 +
 builtin/hash-object.c    |   1 +
 builtin/log.c            |   1 +
 builtin/ls-tree.c        |   1 +
 builtin/merge-tree.c     |   1 +
 builtin/mktag.c          |   1 +
 builtin/mktree.c         |   1 +
 builtin/notes.c          |   1 +
 builtin/prune.c          |   1 +
 builtin/receive-pack.c   |   1 +
 builtin/reflog.c         |   1 +
 builtin/remote.c         |   1 +
 builtin/rev-list.c       |   1 +
 builtin/show-ref.c       |   1 +
 builtin/tag.c            |   1 +
 builtin/unpack-file.c    |   1 +
 builtin/unpack-objects.c |   1 +
 builtin/verify-commit.c  |   1 +
 bulk-checkin.c           |   1 +
 bundle.c                 |   1 +
 cache-tree.c             |   1 +
 cache.h                  | 117 ---------------------------------------
 combine-diff.c           |   1 +
 commit.c                 |   1 +
 config.c                 |   1 +
 convert.c                |   1 +
 diff.c                   |   1 +
 diffcore-rename.c        |   1 +
 dir.c                    |   1 +
 entry.c                  |   1 +
 fetch-pack.c             |   1 +
 fsck.c                   |   1 +
 grep.c                   |   1 +
 list-objects-filter.c    |   1 +
 list-objects.c           |   1 +
 log-tree.c               |   1 +
 mailmap.c                |   1 +
 match-trees.c            |   1 +
 merge-blobs.c            |   1 +
 merge-recursive.c        |   1 +
 notes-cache.c            |   1 +
 notes-merge.c            |   1 +
 notes.c                  |   1 +
 object-store.h           | 117 +++++++++++++++++++++++++++++++++++++++
 object.c                 |   1 +
 pack-bitmap-write.c      |   1 +
 packfile.h               |   5 ++
 read-cache.c             |   1 +
 ref-filter.c             |   1 +
 refs.c                   |   1 +
 remote-testsvn.c         |   1 +
 remote.c                 |   1 +
 rerere.c                 |   1 +
 revision.c               |   1 +
 send-pack.c              |   1 +
 sequencer.c              |   1 +
 shallow.c                |   1 +
 submodule-config.c       |   1 +
 tag.c                    |   1 +
 tree-walk.c              |   1 +
 tree.c                   |   1 +
 unpack-trees.c           |   1 +
 upload-pack.c            |   1 +
 walker.c                 |   1 +
 xdiff-interface.c        |   1 +
 79 files changed, 198 insertions(+), 117 deletions(-)

diff --git a/apply.c b/apply.c
index 7e5792c996f..cbc45fa1b0e 100644
--- a/apply.c
+++ b/apply.c
@@ -9,6 +9,7 @@
 
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "blob.h"
 #include "delta.h"
 #include "diff.h"
diff --git a/archive-tar.c b/archive-tar.c
index f93409324f9..e38435eb4ef 100644
--- a/archive-tar.c
+++ b/archive-tar.c
@@ -5,6 +5,7 @@
 #include "config.h"
 #include "tar.h"
 #include "archive.h"
+#include "object-store.h"
 #include "streaming.h"
 #include "run-command.h"
 
diff --git a/archive-zip.c b/archive-zip.c
index 74f3fe91034..abc556e5a75 100644
--- a/archive-zip.c
+++ b/archive-zip.c
@@ -6,6 +6,7 @@
 #include "archive.h"
 #include "streaming.h"
 #include "utf8.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
 
diff --git a/archive.c b/archive.c
index 93ab175b0b4..9da1e3664a6 100644
--- a/archive.c
+++ b/archive.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree-walk.h"
 #include "attr.h"
diff --git a/blame.c b/blame.c
index 3a11f1ce52b..f689bde31cd 100644
--- a/blame.c
+++ b/blame.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "cache-tree.h"
 #include "mergesort.h"
 #include "diff.h"
diff --git a/builtin/blame.c b/builtin/blame.c
index bfdf7cc1325..0ffd1d443ea 100644
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@ -22,6 +22,7 @@
 #include "line-log.h"
 #include "dir.h"
 #include "progress.h"
+#include "object-store.h"
 #include "blame.h"
 
 static char blame_usage[] = N_("git blame [<options>] [<rev-opts>] [<rev>] [--] <file>");
diff --git a/builtin/cat-file.c b/builtin/cat-file.c
index b8ecbea98e9..91e7764243e 100644
--- a/builtin/cat-file.c
+++ b/builtin/cat-file.c
@@ -13,6 +13,7 @@
 #include "tree-walk.h"
 #include "sha1-array.h"
 #include "packfile.h"
+#include "object-store.h"
 
 struct batch_options {
 	int enabled;
diff --git a/builtin/checkout.c b/builtin/checkout.c
index b49b5820718..105e07981ff 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -4,6 +4,7 @@
 #include "lockfile.h"
 #include "parse-options.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/builtin/clone.c b/builtin/clone.c
index 7df5932b855..29998c02ece 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -14,6 +14,7 @@
 #include "parse-options.h"
 #include "fetch-pack.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tree.h"
 #include "tree-walk.h"
 #include "unpack-trees.h"
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index ecf42191da1..9fbd3529fb1 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -5,6 +5,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "builtin.h"
diff --git a/builtin/describe.c b/builtin/describe.c
index 66c497f7896..65b0edc473c 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -13,6 +13,7 @@
 #include "hashmap.h"
 #include "argv-array.h"
 #include "run-command.h"
+#include "object-store.h"
 #include "revision.h"
 #include "list-objects.h"
 
diff --git a/builtin/difftool.c b/builtin/difftool.c
index ee8dce019e1..df7e75f797b 100644
--- a/builtin/difftool.c
+++ b/builtin/difftool.c
@@ -20,6 +20,7 @@
 #include "argv-array.h"
 #include "strbuf.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "dir.h"
 
 static char *diff_gui_tool;
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 373c794873e..f593e57b9d4 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "object.h"
 #include "tag.h"
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 73be393b2ea..c1f2df97965 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -5,6 +5,7 @@
 #include "config.h"
 #include "repository.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "builtin.h"
 #include "string-list.h"
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index bd680be6874..1b526adb3a9 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index 526da5c1856..d5c018eabd5 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -6,6 +6,7 @@
  */
 #include "builtin.h"
 #include "config.h"
+#include "object-store.h"
 #include "blob.h"
 #include "quote.h"
 #include "parse-options.h"
diff --git a/builtin/log.c b/builtin/log.c
index 71f68a3e4f5..9656578f58e 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -7,6 +7,7 @@
 #include "cache.h"
 #include "config.h"
 #include "refs.h"
+#include "object-store.h"
 #include "color.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c
index 409da4e8351..fe3b952cb30 100644
--- a/builtin/ls-tree.c
+++ b/builtin/ls-tree.c
@@ -5,6 +5,7 @@
  */
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 32736e0b101..1b702d44c9f 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "object-store.h"
 #include "blob.h"
 #include "exec_cmd.h"
 #include "merge-blobs.h"
diff --git a/builtin/mktag.c b/builtin/mktag.c
index 82a6e860775..6fb7dc8578d 100644
--- a/builtin/mktag.c
+++ b/builtin/mktag.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "tag.h"
 #include "replace-object.h"
+#include "object-store.h"
 
 /*
  * A signature file has a very simple fixed format: four lines
diff --git a/builtin/mktree.c b/builtin/mktree.c
index bb76b469fd1..2dc4ad6ba8f 100644
--- a/builtin/mktree.c
+++ b/builtin/mktree.c
@@ -7,6 +7,7 @@
 #include "quote.h"
 #include "tree.h"
 #include "parse-options.h"
+#include "object-store.h"
 
 static struct treeent {
 	unsigned mode;
diff --git a/builtin/notes.c b/builtin/notes.c
index 921e08d5bf5..73b680ee13a 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -11,6 +11,7 @@
 #include "config.h"
 #include "builtin.h"
 #include "notes.h"
+#include "object-store.h"
 #include "blob.h"
 #include "pretty.h"
 #include "refs.h"
diff --git a/builtin/prune.c b/builtin/prune.c
index 518ffbea139..8cc8659612f 100644
--- a/builtin/prune.c
+++ b/builtin/prune.c
@@ -6,6 +6,7 @@
 #include "reachable.h"
 #include "parse-options.h"
 #include "progress.h"
+#include "object-store.h"
 
 static const char * const prune_usage[] = {
 	N_("git prune [-n] [-v] [--progress] [--expire <time>] [--] [<head>...]"),
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index c4272fbc96d..36906fd5e98 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -25,6 +25,7 @@
 #include "tmp-objdir.h"
 #include "oidset.h"
 #include "packfile.h"
+#include "object-store.h"
 #include "protocol.h"
 
 static const char * const receive_pack_usage[] = {
diff --git a/builtin/reflog.c b/builtin/reflog.c
index a89bd1dd252..bee4795a936 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "config.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "commit.h"
 #include "refs.h"
 #include "dir.h"
diff --git a/builtin/remote.c b/builtin/remote.c
index 805ffc05cdb..9487c019092 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -7,6 +7,7 @@
 #include "strbuf.h"
 #include "run-command.h"
 #include "refs.h"
+#include "object-store.h"
 #include "argv-array.h"
 
 static const char * const builtin_remote_usage[] = {
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index fadd3ec14cb..e9bd4e378ad 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -6,6 +6,7 @@
 #include "list-objects.h"
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
+#include "object-store.h"
 #include "pack.h"
 #include "pack-bitmap.h"
 #include "builtin.h"
diff --git a/builtin/show-ref.c b/builtin/show-ref.c
index f2eb1a77240..2f13f1316fa 100644
--- a/builtin/show-ref.c
+++ b/builtin/show-ref.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "cache.h"
 #include "refs.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "string-list.h"
diff --git a/builtin/tag.c b/builtin/tag.c
index 26d7729f57b..f872f14f65e 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -10,6 +10,7 @@
 #include "config.h"
 #include "builtin.h"
 #include "refs.h"
+#include "object-store.h"
 #include "tag.h"
 #include "run-command.h"
 #include "parse-options.h"
diff --git a/builtin/unpack-file.c b/builtin/unpack-file.c
index 300eb59657e..58652229f27 100644
--- a/builtin/unpack-file.c
+++ b/builtin/unpack-file.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "config.h"
+#include "object-store.h"
 
 static char *create_temp_file(struct object_id *oid)
 {
diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index cfe9019f800..ded798b72ff 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -1,6 +1,7 @@
 #include "builtin.h"
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "object.h"
 #include "delta.h"
 #include "pack.h"
diff --git a/builtin/verify-commit.c b/builtin/verify-commit.c
index dcdaada1110..f6922da16d6 100644
--- a/builtin/verify-commit.c
+++ b/builtin/verify-commit.c
@@ -8,6 +8,7 @@
 #include "cache.h"
 #include "config.h"
 #include "builtin.h"
+#include "object-store.h"
 #include "commit.h"
 #include "run-command.h"
 #include <signal.h>
diff --git a/bulk-checkin.c b/bulk-checkin.c
index de1f4040c78..918d58a7ebf 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -8,6 +8,7 @@
 #include "pack.h"
 #include "strbuf.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static struct bulk_checkin_state {
 	unsigned plugged:1;
diff --git a/bundle.c b/bundle.c
index 902c9b54485..00ada05f46c 100644
--- a/bundle.c
+++ b/bundle.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "lockfile.h"
 #include "bundle.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "diff.h"
diff --git a/cache-tree.c b/cache-tree.c
index 6a555f4d431..1c338c41f3a 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -3,6 +3,7 @@
 #include "tree.h"
 #include "tree-walk.h"
 #include "cache-tree.h"
+#include "object-store.h"
 
 #ifndef DEBUG
 #define DEBUG 0
diff --git a/cache.h b/cache.h
index c75559b7d38..ab716011b7e 100644
--- a/cache.h
+++ b/cache.h
@@ -1183,32 +1183,6 @@ extern char *xdg_config_home(const char *filename);
  */
 extern char *xdg_cache_home(const char *filename);
 
-extern void *read_object_file_extended(const struct object_id *oid,
-				       enum object_type *type,
-				       unsigned long *size, int lookup_replace);
-static inline void *read_object_file(const struct object_id *oid, enum object_type *type, unsigned long *size)
-{
-	return read_object_file_extended(oid, type, size, 1);
-}
-
-/* Read and unpack an object file into memory, write memory to an object file */
-int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
-
-extern int hash_object_file(const void *buf, unsigned long len,
-			    const char *type, struct object_id *oid);
-
-extern int write_object_file(const void *buf, unsigned long len,
-			     const char *type, struct object_id *oid);
-
-extern int hash_object_file_literally(const void *buf, unsigned long len,
-				      const char *type, struct object_id *oid,
-				      unsigned flags);
-
-extern int pretend_object_file(void *, unsigned long, enum object_type,
-			       struct object_id *oid);
-
-extern int force_object_loose(const struct object_id *oid, time_t mtime);
-
 extern int git_open_cloexec(const char *name, int flags);
 #define git_open(name) git_open_cloexec(name, O_RDONLY)
 extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);
@@ -1218,43 +1192,6 @@ extern int check_object_signature(const struct object_id *oid, void *buf, unsign
 
 extern int finalize_object_file(const char *tmpfile, const char *filename);
 
-/*
- * Open the loose object at path, check its hash, and return the contents,
- * type, and size. If the object is a blob, then "contents" may return NULL,
- * to allow streaming of large blobs.
- *
- * Returns 0 on success, negative on error (details may be written to stderr).
- */
-int read_loose_object(const char *path,
-		      const struct object_id *expected_oid,
-		      enum object_type *type,
-		      unsigned long *size,
-		      void **contents);
-
-/*
- * Convenience for sha1_object_info_extended() with a NULL struct
- * object_info. OBJECT_INFO_SKIP_CACHED is automatically set; pass
- * nonzero flags to also set other flags.
- */
-extern int has_sha1_file_with_flags(const unsigned char *sha1, int flags);
-static inline int has_sha1_file(const unsigned char *sha1)
-{
-	return has_sha1_file_with_flags(sha1, 0);
-}
-
-/* Same as the above, except for struct object_id. */
-extern int has_object_file(const struct object_id *oid);
-extern int has_object_file_with_flags(const struct object_id *oid, int flags);
-
-/*
- * Return true iff an alternate object database has a loose object
- * with the specified name.  This function does not respect replace
- * references.
- */
-extern int has_loose_object_nonlocal(const unsigned char *sha1);
-
-extern void assert_oid_type(const struct object_id *oid, enum object_type expect);
-
 /* Helper to check and "touch" a file */
 extern int check_and_freshen_file(const char *fn, int freshen);
 
@@ -1624,60 +1561,6 @@ int for_each_loose_file_in_objdir_buf(struct strbuf *path,
 #define FOR_EACH_OBJECT_LOCAL_ONLY 0x1
 extern int for_each_loose_object(each_loose_object_fn, void *, unsigned flags);
 
-struct object_info {
-	/* Request */
-	enum object_type *typep;
-	unsigned long *sizep;
-	off_t *disk_sizep;
-	unsigned char *delta_base_sha1;
-	struct strbuf *type_name;
-	void **contentp;
-
-	/* Response */
-	enum {
-		OI_CACHED,
-		OI_LOOSE,
-		OI_PACKED,
-		OI_DBCACHED
-	} whence;
-	union {
-		/*
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } cached;
-		 * struct {
-		 * 	... Nothing to expose in this case
-		 * } loose;
-		 */
-		struct {
-			struct packed_git *pack;
-			off_t offset;
-			unsigned int is_delta;
-		} packed;
-	} u;
-};
-
-/*
- * Initializer for a "struct object_info" that wants no items. You may
- * also memset() the memory to all-zeroes.
- */
-#define OBJECT_INFO_INIT {NULL}
-
-/* Invoke lookup_replace_object() on the given hash */
-#define OBJECT_INFO_LOOKUP_REPLACE 1
-/* Allow reading from a loose object file of unknown/bogus type */
-#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
-/* Do not check cached storage */
-#define OBJECT_INFO_SKIP_CACHED 4
-/* Do not retry packed storage after checking packed and loose storage */
-#define OBJECT_INFO_QUICK 8
-/* Do not check loose object */
-#define OBJECT_INFO_IGNORE_LOOSE 16
-
-int oid_object_info_extended(struct repository *r,
-			     const struct object_id *,
-			     struct object_info *, unsigned flags);
-
 /*
  * Set this to 0 to prevent sha1_object_info_extended() from fetching missing
  * blobs. This has a difference only if extensions.partialClone is set.
diff --git a/combine-diff.c b/combine-diff.c
index 2ef495963fc..de7695e7282 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "blob.h"
 #include "diff.h"
diff --git a/commit.c b/commit.c
index 5eb4d2f08f8..b053f07f305 100644
--- a/commit.c
+++ b/commit.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "tag.h"
 #include "commit.h"
+#include "object-store.h"
 #include "pkt-line.h"
 #include "utf8.h"
 #include "diff.h"
diff --git a/config.c b/config.c
index c698988f5e1..041db3ce7d4 100644
--- a/config.c
+++ b/config.c
@@ -14,6 +14,7 @@
 #include "quote.h"
 #include "hashmap.h"
 #include "string-list.h"
+#include "object-store.h"
 #include "utf8.h"
 #include "dir.h"
 
diff --git a/convert.c b/convert.c
index c480097a2a0..7f5afd6dd66 100644
--- a/convert.c
+++ b/convert.c
@@ -1,6 +1,7 @@
 #define NO_THE_INDEX_COMPATIBILITY_MACROS
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "attr.h"
 #include "run-command.h"
 #include "quote.h"
diff --git a/diff.c b/diff.c
index 4753170fe12..18d478c5190 100644
--- a/diff.c
+++ b/diff.c
@@ -13,6 +13,7 @@
 #include "attr.h"
 #include "run-command.h"
 #include "utf8.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "submodule-config.h"
 #include "submodule.h"
diff --git a/diffcore-rename.c b/diffcore-rename.c
index 0b7e4989a87..d775183c2fd 100644
--- a/diffcore-rename.c
+++ b/diffcore-rename.c
@@ -4,6 +4,7 @@
 #include "cache.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "object-store.h"
 #include "hashmap.h"
 #include "progress.h"
 
diff --git a/dir.c b/dir.c
index 63a917be45d..9e7c90cf50b 100644
--- a/dir.c
+++ b/dir.c
@@ -11,6 +11,7 @@
 #include "cache.h"
 #include "config.h"
 #include "dir.h"
+#include "object-store.h"
 #include "attr.h"
 #include "refs.h"
 #include "wildmatch.h"
diff --git a/entry.c b/entry.c
index 2101201a111..b5d1d3cf231 100644
--- a/entry.c
+++ b/entry.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "blob.h"
+#include "object-store.h"
 #include "dir.h"
 #include "streaming.h"
 #include "submodule.h"
diff --git a/fetch-pack.c b/fetch-pack.c
index adc1b68dd3a..a1535b37b9b 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -19,6 +19,7 @@
 #include "sha1-array.h"
 #include "oidset.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static int transfer_unpack_limit = -1;
 static int fetch_unpack_limit = -1;
diff --git a/fsck.c b/fsck.c
index 9218c2a643b..59b0c7d640e 100644
--- a/fsck.c
+++ b/fsck.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "object.h"
 #include "blob.h"
 #include "tree.h"
diff --git a/grep.c b/grep.c
index 65b90c10a38..438987220ea 100644
--- a/grep.c
+++ b/grep.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "grep.h"
+#include "object-store.h"
 #include "userdiff.h"
 #include "xdiff-interface.h"
 #include "diff.h"
diff --git a/list-objects-filter.c b/list-objects-filter.c
index ea94fe8af28..44b3a49bdb3 100644
--- a/list-objects-filter.c
+++ b/list-objects-filter.c
@@ -11,6 +11,7 @@
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
 #include "oidset.h"
+#include "object-store.h"
 
 /* Remember to update object flag allocation in object.h */
 /*
diff --git a/list-objects.c b/list-objects.c
index 168bef688a8..7ae0eb8ebc7 100644
--- a/list-objects.c
+++ b/list-objects.c
@@ -10,6 +10,7 @@
 #include "list-objects-filter.h"
 #include "list-objects-filter-options.h"
 #include "packfile.h"
+#include "object-store.h"
 
 static void process_blob(struct rev_info *revs,
 			 struct blob *blob,
diff --git a/log-tree.c b/log-tree.c
index d1c0bedf244..5660ee7c785 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "diff.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "graph.h"
diff --git a/mailmap.c b/mailmap.c
index 13f0d2884e2..962fd86d6d7 100644
--- a/mailmap.c
+++ b/mailmap.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "string-list.h"
 #include "mailmap.h"
+#include "object-store.h"
 
 #define DEBUG_MAILMAP 0
 #if DEBUG_MAILMAP
diff --git a/match-trees.c b/match-trees.c
index 72cc2baa3f9..4cdeff53e1e 100644
--- a/match-trees.c
+++ b/match-trees.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "tree.h"
 #include "tree-walk.h"
+#include "object-store.h"
 
 static int score_missing(unsigned mode, const char *path)
 {
diff --git a/merge-blobs.c b/merge-blobs.c
index fa49c17287f..fabb8c19ce9 100644
--- a/merge-blobs.c
+++ b/merge-blobs.c
@@ -4,6 +4,7 @@
 #include "ll-merge.h"
 #include "blob.h"
 #include "merge-blobs.h"
+#include "object-store.h"
 
 static int fill_mmfile_blob(mmfile_t *f, struct blob *obj)
 {
diff --git a/merge-recursive.c b/merge-recursive.c
index cbded673c28..455f59d21ac 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -8,6 +8,7 @@
 #include "advice.h"
 #include "lockfile.h"
 #include "cache-tree.h"
+#include "object-store.h"
 #include "commit.h"
 #include "blob.h"
 #include "builtin.h"
diff --git a/notes-cache.c b/notes-cache.c
index e61988e503b..d5770031776 100644
--- a/notes-cache.c
+++ b/notes-cache.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "notes-cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "refs.h"
 
diff --git a/notes-merge.c b/notes-merge.c
index 8e0726a9418..fd96c5ab3d6 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "commit.h"
 #include "refs.h"
+#include "object-store.h"
 #include "diff.h"
 #include "diffcore.h"
 #include "xdiff-interface.h"
diff --git a/notes.c b/notes.c
index a386d450c4c..32d3dbcc1e7 100644
--- a/notes.c
+++ b/notes.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "config.h"
 #include "notes.h"
+#include "object-store.h"
 #include "blob.h"
 #include "tree.h"
 #include "utf8.h"
diff --git a/object-store.h b/object-store.h
index 1ff862c7f93..695266891bf 100644
--- a/object-store.h
+++ b/object-store.h
@@ -137,4 +137,121 @@ void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned cha
 
 void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);
 
+extern void *read_object_file_extended(const struct object_id *oid,
+				       enum object_type *type,
+				       unsigned long *size, int lookup_replace);
+static inline void *read_object_file(const struct object_id *oid, enum object_type *type, unsigned long *size)
+{
+	return read_object_file_extended(oid, type, size, 1);
+}
+
+/* Read and unpack an object file into memory, write memory to an object file */
+int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);
+
+extern int hash_object_file(const void *buf, unsigned long len,
+			    const char *type, struct object_id *oid);
+
+extern int write_object_file(const void *buf, unsigned long len,
+			     const char *type, struct object_id *oid);
+
+extern int hash_object_file_literally(const void *buf, unsigned long len,
+				      const char *type, struct object_id *oid,
+				      unsigned flags);
+
+extern int pretend_object_file(void *, unsigned long, enum object_type,
+			       struct object_id *oid);
+
+extern int force_object_loose(const struct object_id *oid, time_t mtime);
+
+/*
+ * Open the loose object at path, check its hash, and return the contents,
+ * type, and size. If the object is a blob, then "contents" may return NULL,
+ * to allow streaming of large blobs.
+ *
+ * Returns 0 on success, negative on error (details may be written to stderr).
+ */
+int read_loose_object(const char *path,
+		      const struct object_id *expected_oid,
+		      enum object_type *type,
+		      unsigned long *size,
+		      void **contents);
+
+/*
+ * Convenience for sha1_object_info_extended() with a NULL struct
+ * object_info. OBJECT_INFO_SKIP_CACHED is automatically set; pass
+ * nonzero flags to also set other flags.
+ */
+extern int has_sha1_file_with_flags(const unsigned char *sha1, int flags);
+static inline int has_sha1_file(const unsigned char *sha1)
+{
+	return has_sha1_file_with_flags(sha1, 0);
+}
+
+/* Same as the above, except for struct object_id. */
+extern int has_object_file(const struct object_id *oid);
+extern int has_object_file_with_flags(const struct object_id *oid, int flags);
+
+/*
+ * Return true iff an alternate object database has a loose object
+ * with the specified name.  This function does not respect replace
+ * references.
+ */
+extern int has_loose_object_nonlocal(const unsigned char *sha1);
+
+extern void assert_oid_type(const struct object_id *oid, enum object_type expect);
+
+struct object_info {
+	/* Request */
+	enum object_type *typep;
+	unsigned long *sizep;
+	off_t *disk_sizep;
+	unsigned char *delta_base_sha1;
+	struct strbuf *type_name;
+	void **contentp;
+
+	/* Response */
+	enum {
+		OI_CACHED,
+		OI_LOOSE,
+		OI_PACKED,
+		OI_DBCACHED
+	} whence;
+	union {
+		/*
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } cached;
+		 * struct {
+		 * 	... Nothing to expose in this case
+		 * } loose;
+		 */
+		struct {
+			struct packed_git *pack;
+			off_t offset;
+			unsigned int is_delta;
+		} packed;
+	} u;
+};
+
+/*
+ * Initializer for a "struct object_info" that wants no items. You may
+ * also memset() the memory to all-zeroes.
+ */
+#define OBJECT_INFO_INIT {NULL}
+
+/* Invoke lookup_replace_object() on the given hash */
+#define OBJECT_INFO_LOOKUP_REPLACE 1
+/* Allow reading from a loose object file of unknown/bogus type */
+#define OBJECT_INFO_ALLOW_UNKNOWN_TYPE 2
+/* Do not check cached storage */
+#define OBJECT_INFO_SKIP_CACHED 4
+/* Do not retry packed storage after checking packed and loose storage */
+#define OBJECT_INFO_QUICK 8
+/* Do not check loose object */
+#define OBJECT_INFO_IGNORE_LOOSE 16
+
+int oid_object_info_extended(struct repository *r,
+			     const struct object_id *,
+			     struct object_info *, unsigned flags);
+
 #endif /* OBJECT_STORE_H */
diff --git a/object.c b/object.c
index 8e29f63bf23..0116ed6529a 100644
--- a/object.c
+++ b/object.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "object.h"
 #include "replace-object.h"
+#include "object-store.h"
 #include "blob.h"
 #include "tree.h"
 #include "commit.h"
diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c
index cd1903e717a..f7fac9d0f24 100644
--- a/pack-bitmap-write.c
+++ b/pack-bitmap-write.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "diff.h"
diff --git a/packfile.h b/packfile.h
index fdfddb89b53..1abbc2c07c8 100644
--- a/packfile.h
+++ b/packfile.h
@@ -3,6 +3,11 @@
 
 #include "oidset.h"
 
+/* in object-store.h */
+struct packed_git;
+struct object_info;
+enum object_type;
+
 /*
  * Generate the filename to be used for a pack file with checksum "sha1" and
  * extension "ext". The result is written into the strbuf "buf", overwriting
diff --git a/read-cache.c b/read-cache.c
index 10f1c6bb8a3..7f10d9ffff9 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -11,6 +11,7 @@
 #include "cache-tree.h"
 #include "refs.h"
 #include "dir.h"
+#include "object-store.h"
 #include "tree.h"
 #include "commit.h"
 #include "blob.h"
diff --git a/ref-filter.c b/ref-filter.c
index 9a333e21b51..e2ae1f47fee 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "refs.h"
 #include "wildmatch.h"
+#include "object-store.h"
 #include "commit.h"
 #include "remote.h"
 #include "color.h"
diff --git a/refs.c b/refs.c
index 27c88ba7689..2361ba9c6ee 100644
--- a/refs.c
+++ b/refs.c
@@ -9,6 +9,7 @@
 #include "iterator.h"
 #include "refs.h"
 #include "refs/refs-internal.h"
+#include "object-store.h"
 #include "object.h"
 #include "tag.h"
 #include "submodule.h"
diff --git a/remote-testsvn.c b/remote-testsvn.c
index c4bb9a8ba92..515b922b503 100644
--- a/remote-testsvn.c
+++ b/remote-testsvn.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "refs.h"
 #include "remote.h"
+#include "object-store.h"
 #include "strbuf.h"
 #include "url.h"
 #include "exec_cmd.h"
diff --git a/remote.c b/remote.c
index 481bf933f39..db438cfbc56 100644
--- a/remote.c
+++ b/remote.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "remote.h"
 #include "refs.h"
+#include "object-store.h"
 #include "commit.h"
 #include "diff.h"
 #include "revision.h"
diff --git a/rerere.c b/rerere.c
index 18cae2d11c9..fcb99cc9542 100644
--- a/rerere.c
+++ b/rerere.c
@@ -9,6 +9,7 @@
 #include "ll-merge.h"
 #include "attr.h"
 #include "pathspec.h"
+#include "object-store.h"
 #include "sha1-lookup.h"
 
 #define RESOLVED 0
diff --git a/revision.c b/revision.c
index 1cff11833e7..260b9c276c7 100644
--- a/revision.c
+++ b/revision.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "object-store.h"
 #include "tag.h"
 #include "blob.h"
 #include "tree.h"
diff --git a/send-pack.c b/send-pack.c
index 19025a7aca8..71600028cdd 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "commit.h"
 #include "refs.h"
+#include "object-store.h"
 #include "pkt-line.h"
 #include "sideband.h"
 #include "run-command.h"
diff --git a/sequencer.c b/sequencer.c
index 44f0518b9c4..cf6456ab75d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "lockfile.h"
 #include "dir.h"
+#include "object-store.h"
 #include "object.h"
 #include "commit.h"
 #include "sequencer.h"
diff --git a/shallow.c b/shallow.c
index df4d44ea7a3..c2f81a5a5a8 100644
--- a/shallow.c
+++ b/shallow.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "tempfile.h"
 #include "lockfile.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tag.h"
 #include "pkt-line.h"
diff --git a/submodule-config.c b/submodule-config.c
index 3f2075764fe..a1e57c33339 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -4,6 +4,7 @@
 #include "submodule-config.h"
 #include "submodule.h"
 #include "strbuf.h"
+#include "object-store.h"
 #include "parse-options.h"
 
 /*
diff --git a/tag.c b/tag.c
index 7c12426b4ea..3be7206e920 100644
--- a/tag.c
+++ b/tag.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "tag.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "blob.h"
diff --git a/tree-walk.c b/tree-walk.c
index e11b3063afa..cd428a1ee53 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -2,6 +2,7 @@
 #include "tree-walk.h"
 #include "unpack-trees.h"
 #include "dir.h"
+#include "object-store.h"
 #include "tree.h"
 #include "pathspec.h"
 
diff --git a/tree.c b/tree.c
index 8f8ef3189af..dc20a2693fd 100644
--- a/tree.c
+++ b/tree.c
@@ -2,6 +2,7 @@
 #include "cache.h"
 #include "cache-tree.h"
 #include "tree.h"
+#include "object-store.h"
 #include "blob.h"
 #include "commit.h"
 #include "tag.h"
diff --git a/unpack-trees.c b/unpack-trees.c
index e73745051e5..e3f95cc6f95 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -15,6 +15,7 @@
 #include "submodule.h"
 #include "submodule-config.h"
 #include "fsmonitor.h"
+#include "object-store.h"
 #include "fetch-object.h"
 
 /*
diff --git a/upload-pack.c b/upload-pack.c
index 4a82602be5d..a11c6d192ce 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -3,6 +3,7 @@
 #include "refs.h"
 #include "pkt-line.h"
 #include "sideband.h"
+#include "object-store.h"
 #include "tag.h"
 #include "object.h"
 #include "commit.h"
diff --git a/walker.c b/walker.c
index dffb9c8e37c..3fb5fb2a0be 100644
--- a/walker.c
+++ b/walker.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "walker.h"
+#include "object-store.h"
 #include "commit.h"
 #include "tree.h"
 #include "tree-walk.h"
diff --git a/xdiff-interface.c b/xdiff-interface.c
index 9315bc0ede1..ec6e574e4aa 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "config.h"
+#include "object-store.h"
 #include "xdiff-interface.h"
 #include "xdiff/xtypes.h"
 #include "xdiff/xdiffi.h"
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 5%]

* [PATCH v2 0/1] rebased: inform about auto submodule ff
  2018-05-17 18:40   ` [PATCH] merge-recursive: give notice when submodule commit gets fast-forwarded Stefan Beller
@ 2018-05-18 19:43     ` Leif Middelschulte
  2018-05-18 19:48     ` [PATCH v3 " Leif Middelschulte
  1 sibling, 0 replies; 200+ results
From: Leif Middelschulte @ 2018-05-18 19:43 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, leif.middelschulte, Leif Middelschulte

From: Leif Middelschulte <Leif.Middelschulte@gmail.com>

This is a follow-up on Junio C Hamano's comments [0] and Stefan Beller's request
[1] for a more explainatory/elaborate commit message.

[0] https://public-inbox.org/git/xmqqk1s474vx.fsf@gitster-ct.c.googlers.com/
[1] https://public-inbox.org/git/20180517184008.25445-1-sbeller@google.com/

Leif Middelschulte (1):
  Inform about fast-forwarding of submodules during merge

 merge-recursive.c | 4 ++++
 1 file changed, 4 insertions(+)

-- 
2.15.1 (Apple Git-101)


^ permalink raw reply	[relevance 7%]

* [PATCH v3 0/1] rebased: inform about auto submodule ff
  2018-05-17 18:40   ` [PATCH] merge-recursive: give notice when submodule commit gets fast-forwarded Stefan Beller
  2018-05-18 19:43     ` [PATCH v2 0/1] rebased: inform about auto submodule ff Leif Middelschulte
@ 2018-05-18 19:48     ` " Leif Middelschulte
  1 sibling, 0 replies; 200+ results
From: Leif Middelschulte @ 2018-05-18 19:48 UTC (permalink / raw)
  To: sbeller; +Cc: git, gitster, leif.middelschulte, Leif Middelschulte

From: Leif Middelschulte <Leif.Middelschulte@gmail.com>

This is a follow-up on Junio C Hamano's comments [0] and Stefan Beller's request
[1] for a more explainatory/elaborate commit message.

[0] https://public-inbox.org/git/xmqqk1s474vx.fsf@gitster-ct.c.googlers.com/
[1] https://public-inbox.org/git/20180517184008.25445-1-sbeller@google.com/

Leif Middelschulte (1):
  Inform about fast-forwarding of submodules during merge

 merge-recursive.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

-- 
2.15.1 (Apple Git-101)


^ permalink raw reply	[relevance 7%]

* Re: [PATCH 02/11] repository: introduce repo_read_index_or_die
  2018-05-16 22:21         ` [PATCH 02/11] repository: introduce repo_read_index_or_die Stefan Beller
@ 2018-05-19  6:37           ` Duy Nguyen
  2018-05-21 18:38             ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Duy Nguyen @ 2018-05-19  6:37 UTC (permalink / raw)
  To: Stefan Beller; +Cc: ao2, bmwill, git, gitster

On Wed, May 16, 2018 at 03:21:09PM -0700, Stefan Beller wrote:
> A common pattern with the repo_read_index function is to die if the return
> of repo_read_index is negative.  Move this pattern into a function.
> 
> This patch replaces the calls of repo_read_index with its _or_die version,
> if the error message is exactly the same.
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  builtin/add.c               | 3 +--
>  builtin/check-ignore.c      | 7 ++++---
>  builtin/clean.c             | 4 ++--
>  builtin/mv.c                | 3 +--
>  builtin/reset.c             | 3 +--
>  builtin/rm.c                | 3 +--
>  builtin/submodule--helper.c | 3 +--

'git grep -w -A3 read_cache' tells me you're missing (*)

builtin/diff-tree.c:    if (read_cache() < 0)
builtin/diff-tree.c-            die(_("index file corrupt"));

builtin/merge-ours.c:   if (read_cache() < 0)
builtin/merge-ours.c:           die_errno("read_cache failed");

builtin/update-index.c: entries = read_cache();
builtin/update-index.c- if (entries < 0)
builtin/update-index.c-         die("cache corrupted");

merge-ours.c is interesting because it uses die_errno() version
instead. I think that might give inaccurate diagnostics because we do
not only fail when a libc/system call fails in read_cache(), so it
should be safe to just use die() here.

update-index.c is also interesting because it actually saves the
return value. I don't think it's used anywhere, so you can just drop
that 'entries' variable. But perhaps it's good to make
repo_read_index_or_die() return the number of entries too.

(*) yes yes you did mention "if the error message is exactly the
same". But these look like good candicates to convert anyway

--
Duy

^ permalink raw reply	[relevance 2%]

* Re: [PATCH 02/11] repository: introduce repo_read_index_or_die
  2018-05-19  6:37           ` Duy Nguyen
@ 2018-05-21 18:38             ` Stefan Beller
  2018-05-21 18:50               ` Brandon Williams
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-21 18:38 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: Antonio Ospite, Brandon Williams, git, Junio C Hamano

On Fri, May 18, 2018 at 11:37 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Wed, May 16, 2018 at 03:21:09PM -0700, Stefan Beller wrote:
>> A common pattern with the repo_read_index function is to die if the return
>> of repo_read_index is negative.  Move this pattern into a function.
>>
>> This patch replaces the calls of repo_read_index with its _or_die version,
>> if the error message is exactly the same.
>>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>  builtin/add.c               | 3 +--
>>  builtin/check-ignore.c      | 7 ++++---
>>  builtin/clean.c             | 4 ++--
>>  builtin/mv.c                | 3 +--
>>  builtin/reset.c             | 3 +--
>>  builtin/rm.c                | 3 +--
>>  builtin/submodule--helper.c | 3 +--
>
> 'git grep -w -A3 read_cache' tells me you're missing (*)

> (*) yes yes you did mention "if the error message is exactly the
> same". But these look like good candicates to convert anyway
>
> builtin/diff-tree.c:    if (read_cache() < 0)
> builtin/diff-tree.c-            die(_("index file corrupt"));
>

I would expect this to be covered in a follow up patch as I did look
for (read_cache() < 0) specifically.

> builtin/merge-ours.c:   if (read_cache() < 0)
> builtin/merge-ours.c:           die_errno("read_cache failed");
>
> builtin/update-index.c: entries = read_cache();
> builtin/update-index.c- if (entries < 0)
> builtin/update-index.c-         die("cache corrupted");
>
> merge-ours.c is interesting because it uses die_errno() version
> instead. I think that might give inaccurate diagnostics because we do
> not only fail when a libc/system call fails in read_cache(), so it
> should be safe to just use die() here.
>
> update-index.c is also interesting because it actually saves the
> return value. I don't think it's used anywhere, so you can just drop
> that 'entries' variable. But perhaps it's good to make
> repo_read_index_or_die() return the number of entries too.

Yeah this series left out all the interesting cases, as I just sent it out
without much thought.

Returning the number of entries makes sense.

One of the reviewers of the series questioned the overall goal of the
series as we want to move away from _die() in top level code but this
series moves more towards it.

I don't know.

Stefan

^ permalink raw reply	[relevance 2%]

* Re: [PATCH 02/11] repository: introduce repo_read_index_or_die
  2018-05-21 18:38             ` Stefan Beller
@ 2018-05-21 18:50               ` Brandon Williams
  0 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2018-05-21 18:50 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Duy Nguyen, Antonio Ospite, git, Junio C Hamano

On 05/21, Stefan Beller wrote:
> On Fri, May 18, 2018 at 11:37 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> > On Wed, May 16, 2018 at 03:21:09PM -0700, Stefan Beller wrote:
> >> A common pattern with the repo_read_index function is to die if the return
> >> of repo_read_index is negative.  Move this pattern into a function.
> >>
> >> This patch replaces the calls of repo_read_index with its _or_die version,
> >> if the error message is exactly the same.
> >>
> >> Signed-off-by: Stefan Beller <sbeller@google.com>
> >> ---
> >>  builtin/add.c               | 3 +--
> >>  builtin/check-ignore.c      | 7 ++++---
> >>  builtin/clean.c             | 4 ++--
> >>  builtin/mv.c                | 3 +--
> >>  builtin/reset.c             | 3 +--
> >>  builtin/rm.c                | 3 +--
> >>  builtin/submodule--helper.c | 3 +--
> >
> > 'git grep -w -A3 read_cache' tells me you're missing (*)
> 
> > (*) yes yes you did mention "if the error message is exactly the
> > same". But these look like good candicates to convert anyway
> >
> > builtin/diff-tree.c:    if (read_cache() < 0)
> > builtin/diff-tree.c-            die(_("index file corrupt"));
> >
> 
> I would expect this to be covered in a follow up patch as I did look
> for (read_cache() < 0) specifically.
> 
> > builtin/merge-ours.c:   if (read_cache() < 0)
> > builtin/merge-ours.c:           die_errno("read_cache failed");
> >
> > builtin/update-index.c: entries = read_cache();
> > builtin/update-index.c- if (entries < 0)
> > builtin/update-index.c-         die("cache corrupted");
> >
> > merge-ours.c is interesting because it uses die_errno() version
> > instead. I think that might give inaccurate diagnostics because we do
> > not only fail when a libc/system call fails in read_cache(), so it
> > should be safe to just use die() here.
> >
> > update-index.c is also interesting because it actually saves the
> > return value. I don't think it's used anywhere, so you can just drop
> > that 'entries' variable. But perhaps it's good to make
> > repo_read_index_or_die() return the number of entries too.
> 
> Yeah this series left out all the interesting cases, as I just sent it out
> without much thought.
> 
> Returning the number of entries makes sense.
> 
> One of the reviewers of the series questioned the overall goal of the
> series as we want to move away from _die() in top level code but this
> series moves more towards it.

I've heard every once in a while that we want to move toward this but I
don't believe there is an actual effort along those lines just yet.  For
that to be the case we would need a clearly defined error handling
methodology (aside from the existing "die"ing behavior), which we don't
currently have.

> 
> I don't know.
> 
> Stefan

-- 
Brandon Williams

^ permalink raw reply	[relevance 2%]

* Re: [PATCH v3 01/20] commit-graph: UNLEAK before die()
      [irrelevant]   ` <20180524162504.158394-2-dstolee@microsoft.com>
@ 2018-05-24 22:47     ` Stefan Beller
  2018-05-25  0:08       ` Derrick Stolee
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-24 22:47 UTC (permalink / raw)
  To: Derrick Stolee; +Cc: git, gitster, jnareb, stolee, avarab, marten.agren, peff

On Thu, May 24, 2018 at 9:25 AM, Derrick Stolee <dstolee@microsoft.com> wrote:
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  builtin/commit-graph.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
> index 37420ae0fd..f0875b8bf3 100644
> --- a/builtin/commit-graph.c
> +++ b/builtin/commit-graph.c
> @@ -51,8 +51,11 @@ static int graph_read(int argc, const char **argv)
>         graph_name = get_commit_graph_filename(opts.obj_dir);
>         graph = load_commit_graph_one(graph_name);
>
> -       if (!graph)
> +       if (!graph) {
> +               UNLEAK(graph_name);
>                 die("graph file %s does not exist", graph_name);

Unrelated to this patch: Is the command that ends up die()ing here
a plumbing or porcelain, or: Do we want to translate the message here?

In a lot of commands that show paths we single quote them '%s',
(speaking from experience with a lot of submodule path code)

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] submodule: do not pass null OID to setup_revisions
      [irrelevant] <20180524204729.19896-1-jonathantanmy@google.com>
@ 2018-05-24 23:07 ` Stefan Beller
  2018-05-24 23:30   ` Jonathan Tan
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-24 23:07 UTC (permalink / raw)
  To: Jonathan Tan; +Cc: git

Hi Jonathan,

On Thu, May 24, 2018 at 1:47 PM, Jonathan Tan <jonathantanmy@google.com> wrote:
> If "git pull --recurse-submodules --rebase" is invoked when the current
> branch and its corresponding remote-tracking branch have no merge base,
> a "bad object" fatal error occurs. This issue was introduced with commit
> a6d7eb2c7a ("pull: optionally rebase submodules (remote submodule
> changes only)", 2017-06-23), which also introduced this feature.

Ok, what should happen instead?

> This is because cmd_pull() in builtin/pull.c thus invokes
> submodule_touches_in_range() with a null OID as the first parameter.
> Ensure that this case works, and document what happens in this case.

By documenting you mean adding a test, i.e. documenting it for the
developers, not the users.

>
> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  submodule.c               |  6 ++++--
>  submodule.h               |  5 ++++-
>  t/t5572-pull-submodule.sh | 21 +++++++++++++++++++++
>  3 files changed, 29 insertions(+), 3 deletions(-)
>
> diff --git a/submodule.c b/submodule.c
> index 74d35b2577..49def93dd9 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -1169,8 +1169,10 @@ int submodule_touches_in_range(struct object_id *excl_oid,
>
>         argv_array_push(&args, "--"); /* args[0] program name */
>         argv_array_push(&args, oid_to_hex(incl_oid));
> -       argv_array_push(&args, "--not");
> -       argv_array_push(&args, oid_to_hex(excl_oid));
> +       if (!is_null_oid(excl_oid)) {
> +               argv_array_push(&args, "--not");
> +               argv_array_push(&args, oid_to_hex(excl_oid));
> +       }
>
>         collect_changed_submodules(&subs, &args);
>         ret = subs.nr;
> diff --git a/submodule.h b/submodule.h
> index e5526f6aaa..1fd7111f60 100644
> --- a/submodule.h
> +++ b/submodule.h
> @@ -94,7 +94,10 @@ extern int merge_submodule(struct object_id *result, const char *path,
>                            const struct object_id *a,
>                            const struct object_id *b, int search);
>
> -/* Checks if there are submodule changes in a..b. */
> +/*
> + * Checks if there are submodule changes in a..b. If a is the null OID,
> + * checks b and all its ancestors instead.
> + */
>  extern int submodule_touches_in_range(struct object_id *a,
>                                       struct object_id *b);
>  extern int find_unpushed_submodules(struct oid_array *commits,
> diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh
> index 321bd37deb..f916729a12 100755
> --- a/t/t5572-pull-submodule.sh
> +++ b/t/t5572-pull-submodule.sh
> @@ -132,4 +132,25 @@ test_expect_success 'pull rebase recursing fails with conflicts' '
>         test_i18ngrep "locally recorded submodule modifications" err
>  '
>
> +test_expect_success 'branch has no merge base with remote-tracking counterpart' '
> +       rm -rf parent child &&
> +
> +       test_create_repo a-submodule &&
> +       test_commit -C a-submodule foo &&
> +
> +       test_create_repo parent &&
> +       git -C parent submodule add "$(pwd)/a-submodule" &&
> +       git -C parent commit -m foo &&
> +
> +       git clone parent child &&
> +
> +       # Reset master so that it has no merge base with
> +       # refs/remotes/origin/master.
> +       OTHER=$(git -C child commit-tree -m bar \
> +               $(git -C child rev-parse HEAD^{tree})) &&
> +       git -C child reset --hard "$OTHER" &&

I inserted a test_pause here and inspect child:
* the submodule is the same as in parent, so this patch is
  just testing it works with submodules the same?
* No, the submodule is not cloned into the child
  at all. So we do not know what do do with the submodule.

However this patch is about making sure the superproject
works out well, without this patch we'd have
$ git -C child pull --recurse-submodules --rebase
fatal: bad object 0000000000000000000000000000000000000000
which is to be avoided.

Yes I think this is the best way to fix the issue, I thought for some time that
could first check if submodules are initialzed or active, but these
are checks are done afterwards, so this is ok.

Reviewed-by: Stefan Beller <sbeller@google.com>

Thanks!
Stefan

^ permalink raw reply	[relevance 10%]

* Re: [PATCH] submodule: do not pass null OID to setup_revisions
  2018-05-24 23:07 ` [PATCH] submodule: do not pass null OID to setup_revisions Stefan Beller
@ 2018-05-24 23:30   ` Jonathan Tan
  0 siblings, 0 replies; 200+ results
From: Jonathan Tan @ 2018-05-24 23:30 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git

On Thu, 24 May 2018 16:07:49 -0700
Stefan Beller <sbeller@google.com> wrote:

> Hi Jonathan,
> 
> On Thu, May 24, 2018 at 1:47 PM, Jonathan Tan <jonathantanmy@google.com> wrote:
> > If "git pull --recurse-submodules --rebase" is invoked when the current
> > branch and its corresponding remote-tracking branch have no merge base,
> > a "bad object" fatal error occurs. This issue was introduced with commit
> > a6d7eb2c7a ("pull: optionally rebase submodules (remote submodule
> > changes only)", 2017-06-23), which also introduced this feature.
> 
> Ok, what should happen instead?

Just for there not to be this "bad object" error.

> > This is because cmd_pull() in builtin/pull.c thus invokes
> > submodule_touches_in_range() with a null OID as the first parameter.
> > Ensure that this case works, and document what happens in this case.
> 
> By documenting you mean adding a test, i.e. documenting it for the
> developers, not the users.

I also updated the submodule.h file, but yes, it is for the developers.
I'll change the commit message to make this more clear if I need a
reroll.

> I inserted a test_pause here and inspect child:
> * the submodule is the same as in parent, so this patch is
>   just testing it works with submodules the same?
> * No, the submodule is not cloned into the child
>   at all. So we do not know what do do with the submodule.

Yes, this test doesn't do much. I just wanted to make sure that
submodule_touches_in_range() could be called without encountering this
unrelated error.

(Incidentally, we might want to add tests for the "cannot rebase with
locally recorded submodule modifications", but I haven't looked into
that.)

> However this patch is about making sure the superproject
> works out well, without this patch we'd have
> $ git -C child pull --recurse-submodules --rebase
> fatal: bad object 0000000000000000000000000000000000000000
> which is to be avoided.
> 
> Yes I think this is the best way to fix the issue, I thought for some time that
> could first check if submodules are initialzed or active, but these
> are checks are done afterwards, so this is ok.
> 
> Reviewed-by: Stefan Beller <sbeller@google.com>

Thanks!

^ permalink raw reply	[relevance 8%]

* Re: [PATCH v3 01/20] commit-graph: UNLEAK before die()
  2018-05-24 22:47     ` [PATCH v3 01/20] commit-graph: UNLEAK before die() Stefan Beller
@ 2018-05-25  0:08       ` Derrick Stolee
  0 siblings, 0 replies; 200+ results
From: Derrick Stolee @ 2018-05-25  0:08 UTC (permalink / raw)
  To: Stefan Beller, Derrick Stolee
  Cc: git, gitster, jnareb, avarab, marten.agren, peff

On 5/24/2018 6:47 PM, Stefan Beller wrote:
> On Thu, May 24, 2018 at 9:25 AM, Derrick Stolee <dstolee@microsoft.com> wrote:
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>>   builtin/commit-graph.c | 5 ++++-
>>   1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
>> index 37420ae0fd..f0875b8bf3 100644
>> --- a/builtin/commit-graph.c
>> +++ b/builtin/commit-graph.c
>> @@ -51,8 +51,11 @@ static int graph_read(int argc, const char **argv)
>>          graph_name = get_commit_graph_filename(opts.obj_dir);
>>          graph = load_commit_graph_one(graph_name);
>>
>> -       if (!graph)
>> +       if (!graph) {
>> +               UNLEAK(graph_name);
>>                  die("graph file %s does not exist", graph_name);
> Unrelated to this patch: Is the command that ends up die()ing here
> a plumbing or porcelain, or: Do we want to translate the message here?
>
> In a lot of commands that show paths we single quote them '%s',
> (speaking from experience with a lot of submodule path code)

This is for the 'git commit-graph read' command, which is plumbing (and 
'read' is really only for testing). I don't think this message requires 
translation.

I'll keep the quotes in mind for the future.

Thanks,

-Stolee


^ permalink raw reply	[relevance 2%]

* Re: sb/submodule-move-nested breaks t7411 under GIT_FSMONITOR_TEST
      [irrelevant] ` <87lgc77wc7.fsf@evledraar.gmail.com>
@ 2018-05-25 17:27   ` Stefan Beller
  2018-05-25 19:49   ` Stefan Beller
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-25 17:27 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Junio C Hamano, git, Ben Peart

Hi Ævar,

On Fri, May 25, 2018 at 5:28 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Thu, May 17 2018, Junio C Hamano wrote:
>
>> * sb/submodule-move-nested (2018-03-29) 6 commits
>>   (merged to 'next' on 2018-04-25 at 86b177433a)
>>  + submodule: fixup nested submodules after moving the submodule
>>  + submodule-config: remove submodule_from_cache
>>  + submodule-config: add repository argument to submodule_from_{name, path}
>>  + submodule-config: allow submodule_free to handle arbitrary repositories
>>  + grep: remove "repo" arg from non-supporting funcs
>>  + submodule.h: drop declaration of connect_work_tree_and_git_dir
>>
>>  Moving a submodule that itself has submodule in it with "git mv"
>>  forgot to make necessary adjustment to the nested sub-submodules;
>>  now the codepath learned to recurse into the submodules.
>
> I didn't spot this earlier because I don't test this a lot, but I've
> bisected the following breakage down to da62f786d2 ("submodule: fixup
> nested submodules after moving the submodule", 2018-03-28) (and manually
> confirmed by reverting). On Linux both Debian & CentOS I get tests 3 and
> 4 failing with:
>
>      GIT_FSMONITOR_TEST=$PWD/t7519/fsmonitor-all ./t7411-submodule-config.sh

I can reproduce this. I'll look into it.

^ permalink raw reply	[relevance 5%]

* Re: sb/submodule-move-nested breaks t7411 under GIT_FSMONITOR_TEST
      [irrelevant] ` <87lgc77wc7.fsf@evledraar.gmail.com>
  2018-05-25 17:27   ` sb/submodule-move-nested breaks t7411 under GIT_FSMONITOR_TEST Stefan Beller
@ 2018-05-25 19:49   ` Stefan Beller
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-25 19:49 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason; +Cc: Junio C Hamano, git, Ben Peart

On Fri, May 25, 2018 at 5:28 AM, Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Thu, May 17 2018, Junio C Hamano wrote:
>
>> * sb/submodule-move-nested (2018-03-29) 6 commits
>>   (merged to 'next' on 2018-04-25 at 86b177433a)
>>  + submodule: fixup nested submodules after moving the submodule
>>  + submodule-config: remove submodule_from_cache
>>  + submodule-config: add repository argument to submodule_from_{name, path}
>>  + submodule-config: allow submodule_free to handle arbitrary repositories
>>  + grep: remove "repo" arg from non-supporting funcs
>>  + submodule.h: drop declaration of connect_work_tree_and_git_dir
>>
>>  Moving a submodule that itself has submodule in it with "git mv"
>>  forgot to make necessary adjustment to the nested sub-submodules;
>>  now the codepath learned to recurse into the submodules.
>
> I didn't spot this earlier because I don't test this a lot, but I've
> bisected the following breakage down to da62f786d2 ("submodule: fixup
> nested submodules after moving the submodule", 2018-03-28) (and manually
> confirmed by reverting). On Linux both Debian & CentOS I get tests 3 and
> 4 failing with:
>
>      GIT_FSMONITOR_TEST=$PWD/t7519/fsmonitor-all ./t7411-submodule-config.sh
>
> -v -x output follows:
>
> expecting success:
>         mkdir submodule &&
>         (cd submodule &&
>                 git init &&
>                 echo a >a &&
>                 git add . &&
>                 git commit -ma
>         ) &&
>         mkdir super &&
>         (cd super &&
>                 git init &&
>                 git submodule add ../submodule &&
>                 git submodule add ../submodule a &&
>                 git commit -m "add as submodule and as a" &&
>                 git mv a b &&
>                 git commit -m "move a to b"
>         )

when you add a test_pause here and dump the
state of the setup, then it can be observed that when the fsmonitor is active
the last commit is different; without fsmonitor the moved gitlink and the change
to the .gitmodules file is part of the commit, i.e.

$ git -C super show
        commit d3d90b70a01bd17d026f75a803c8b65f5903a7c0 (HEAD -> master)
        Author: A U Thor <author@example.com>
        Date:   Fri May 25 19:21:58 2018 +0000

            move a to b

        diff --git a/.gitmodules b/.gitmodules
        index 3f4d474..6149210 100644
        --- a/.gitmodules
        +++ b/.gitmodules
        @@ -2,5 +2,5 @@
          path = submodule
          url = ../submodule
         [submodule "a"]
        - path = a
        + path = b
          url = ../submodule
        diff --git a/a b/b
        similarity index 100%
        rename from a
        rename to b
When running with the fsmonitor:

$ git -C super show
        commit 57022a92acf46f303498c045440ec099cbc35a2d (HEAD -> master)
        Author: A U Thor <author@example.com>
        Date:   Fri May 25 19:22:52 2018 +0000

            move a to b

        diff --git a/a b/b
        similarity index 100%
        rename from a
        rename to b
$ git -C super diff
        diff --git a/.gitmodules b/.gitmodules
        index 3f4d474..6149210 100644
        --- a/.gitmodules
        +++ b/.gitmodules
        @@ -2,5 +2,5 @@
          path = submodule
          url = ../submodule
         [submodule "a"]
        - path = a
        + path = b
          url = ../submodule

This hints at a problem with git commit;

I tried adding test_tick, to unconfuse the fsmonitor, but that doesn't help,
digging further, the problem is in the git mv command, which fails to
add the change in
.gitmodules to the index.

Adding the verbose flag to stage_updated_gitmodules() that is called by
git-mv very late in the game, such that

void stage_updated_gitmodules(struct index_state *istate)
{
    trace_printf("staging .gitmodules files");
    if (add_file_to_index(istate, GITMODULES_FILE, ADD_CACHE_VERBOSE))
        die(_("staging updated .gitmodules failed"));
}

We would get a message if the .gitmodules file is staged correctly, as
add_file_to_index() that calls add_to_index that would print

    if (verbose && !was_same)
        printf("add '%s'\n", path);

I could not see that message, so I suspect, that there is something
racy.

Will debug further.

^ permalink raw reply	[relevance 7%]

* Re: RFC: Merge-related plans
      [irrelevant] <CABPp-BFQJZHfCJZ1qvhvVcMd-_sOfi0Fkm5PexEwzzN+Zw2akw@mail.gmail.com>
@ 2018-05-29 18:19 ` Stefan Beller
  2018-05-29 21:03   ` Elijah Newren
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-29 18:19 UTC (permalink / raw)
  To: Elijah Newren; +Cc: Git Mailing List

On Mon, May 28, 2018 at 1:48 PM, Elijah Newren <newren@gmail.com> wrote:
> Hi everyone,
>
> I have some merge-related plans (and work in progress) that I'd like
> to get some feedback on in order to find what order would be best to
> address things in, if there are special steps I should take while
> approaching some of the bigger items, and even if folks disagree with
> any of the plans.
>
>
> Currently, I would like to:
>
> A) Fix cases where directory rename detection does not work with
>    rebase/am due to how they call merge-recursive.
>
>    Notes: Could just wait for D & E to land before fixing.
>    Alternatively, email RFC to list now explaining issues and how the
>    fix has performance implications; poll for opinions on whether to
>    fix before or after D.
>
> B) Implement a remerge-diff ability (compare a merge commit to what an
>    "automatic merge" would look like)[1].
>
>    Notes: Possibly for cherry-picks and reverts too.  Depends on C &
>    E.
>
> C) Modify/extend how path-based and mode-based conflicts are
>    communicated to the user.
>
>    Notes: Particularly important as a mechanism for handling
>    challenges[2] with implementing the remerge-diff ability.  Need to
>    send RFC to list with ideas to get feedback.
>
> D) Improve merge performance.
>
>    Notes: Includes 4-5 specific optimizations[5], some of which I
>    expect to be somewhat invasive and thus may make more sense to just
>    make part of the new merge strategy implemented in E.  Biggest
>    optimization depends on F.
>
> E) Write a new merge strategy meant to replace merge-recursive.
>
>    Notes: Suggested by Junio[3][4].  Depends on F & G.
>
> F) Make file collision conflict types more consistent.
>
>    Notes: Specifically, make rename/rename(2to1) & rename/add
>    conflicts behave more like add/add[6][7].  Depends on part of G.
>    Would prefer H to be accepted first.
>
> G) Improve merge-related portion of testsuite.
>
>    Notes: Intended to help test new merge strategy with more
>    confidence.  Will include approximately a dozen edge and corner
>    cases where merge-recursive currently falls short.  Started at [8];
>    see also [9].

Most items forward-reference "Depends on (<later letter>) up to here;
(H) seems independent, but might be a good first start.
(G) [8] is queued as origin/en/merge-recursive-tests, or do you mean
to expand (G) into a mini-framework of merge-testing? i.e. run the
mini test framework multiple times, each using a different
merge strategy, similar to submodule tests, e.g. see
t/lib-submodule-update.sh and one of its users, t1013.


> H) Miscellaneous code cleanups irritating me while working on other
>    changes[10].
>
>
> My current plan was to work roughly in reverse, from H to A.  Some questions:
>
>   * Does any of this look objectionable?

Going in order A-H seems slightly out-of-order to me, I'd think (H) and (G)
would go first;

(B) sounds like an independent feature, which could go in parallel?

>   * Should I post RFC questions on A and C earlier?

I would think so, it is easier to give feedback on code, I would think.

>   * Should I split D and G?  (Current plan was to keep D together, but
>     split G into five short slightly inter-dependent topics)

I would have expected to have tests (G) as a companion of (A) or (C)
rather than (D), as performance improvements would keep the test suite
unchanged?

>   * E is kind of big.  Are there any specific things folks would like to see
>     with how that is handled?

How much abstraction can be done ahead of time such that there is an
interface/API where you just plug in a new merge strategy and do not
need to duplicate a lot of code/tests?

Stefan

^ permalink raw reply	[relevance 5%]

* Re: [PATCH v2 3/3] completion: collapse extra --no-.. options
      [irrelevant]   ` <20180527083828.6919-4-pclouds@gmail.com>
@ 2018-05-29 18:48     ` Stefan Beller
  2018-05-29 19:04       ` Duy Nguyen
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-29 18:48 UTC (permalink / raw)
  To: Nguyễn Thái Ngọc Duy; +Cc: git, Junio C Hamano, Eric Sunshine

On Sun, May 27, 2018 at 1:38 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> The commands that make use of --git-completion-helper feature could
> now produce a lot of --no-xxx options that a command can take. This in
> many case could nearly double the amount of completable options, using
> more screen estate and also harder to search for the wanted option.
>
> This patch attempts to mitigate that by collapsing extra --no-
> options, the ones that are added by --git-completion-helper and not in
> original struct option arrays. The "--no-..." option will be displayed
> in this case to hint about more options, e.g.
>
>     > ~/w/git $ git clone --
>     --bare                 --origin=
>     --branch=              --progress
>     --checkout             --quiet
>     --config=              --recurse-submodules
>     --depth=               --reference=
>     --dissociate           --reference-if-able=
>     --filter=              --separate-git-dir=
>     --hardlinks            --shallow-exclude=
>     --ipv4                 --shallow-since=
>     --ipv6                 --shallow-submodules
>     --jobs=                --shared
>     --local                --single-branch
>     --mirror               --tags
>     --no-...               --template=
>     --no-checkout          --upload-pack=
>     --no-hardlinks         --verbose
>     --no-tags

https://public-inbox.org/git/20180527083828.6919-1-pclouds@gmail.com/
" There's no magic numbers (previously we keep 3 --no- options)"

Here I see 3 no- options, is the number how many no's to show configurable now?

^ permalink raw reply	[relevance 2%]

* Re: [PATCH v2 3/3] completion: collapse extra --no-.. options
  2018-05-29 18:48     ` [PATCH v2 3/3] completion: collapse extra --no-.. options Stefan Beller
@ 2018-05-29 19:04       ` Duy Nguyen
  0 siblings, 0 replies; 200+ results
From: Duy Nguyen @ 2018-05-29 19:04 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, Junio C Hamano, Eric Sunshine

On Tue, May 29, 2018 at 8:48 PM, Stefan Beller <sbeller@google.com> wrote:
> On Sun, May 27, 2018 at 1:38 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
>> The commands that make use of --git-completion-helper feature could
>> now produce a lot of --no-xxx options that a command can take. This in
>> many case could nearly double the amount of completable options, using
>> more screen estate and also harder to search for the wanted option.
>>
>> This patch attempts to mitigate that by collapsing extra --no-
>> options, the ones that are added by --git-completion-helper and not in
>> original struct option arrays. The "--no-..." option will be displayed
>> in this case to hint about more options, e.g.
>>
>>     > ~/w/git $ git clone --
>>     --bare                 --origin=
>>     --branch=              --progress
>>     --checkout             --quiet
>>     --config=              --recurse-submodules
>>     --depth=               --reference=
>>     --dissociate           --reference-if-able=
>>     --filter=              --separate-git-dir=
>>     --hardlinks            --shallow-exclude=
>>     --ipv4                 --shallow-since=
>>     --ipv6                 --shallow-submodules
>>     --jobs=                --shared
>>     --local                --single-branch
>>     --mirror               --tags
>>     --no-...               --template=
>>     --no-checkout          --upload-pack=
>>     --no-hardlinks         --verbose
>>     --no-tags
>
> https://public-inbox.org/git/20180527083828.6919-1-pclouds@gmail.com/
> " There's no magic numbers (previously we keep 3 --no- options)"
>
> Here I see 3 no- options, is the number how many no's to show configurable now?

In a sense, yes. If you write OPT_BOOL(0, "no-foo",...) then that
--no-foo _always_ shows. "git clone" just happens to have three of
them.
-- 
Duy

^ permalink raw reply	[relevance 2%]

* Re: RFC: Merge-related plans
  2018-05-29 18:19 ` RFC: Merge-related plans Stefan Beller
@ 2018-05-29 21:03   ` Elijah Newren
  2018-05-29 22:12     ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Elijah Newren @ 2018-05-29 21:03 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Git Mailing List

Hi Stefan,

On Tue, May 29, 2018 at 11:19 AM, Stefan Beller <sbeller@google.com> wrote:
> On Mon, May 28, 2018 at 1:48 PM, Elijah Newren <newren@gmail.com> wrote:
>> Currently, I would like to:
>>
>> A) Fix cases where directory rename detection does not work with
>>    rebase/am due to how they call merge-recursive.
>>
>>    Notes: Could just wait for D & E to land before fixing.
>>    Alternatively, email RFC to list now explaining issues and how the
>>    fix has performance implications; poll for opinions on whether to
>>    fix before or after D.
>>
>> B) Implement a remerge-diff ability (compare a merge commit to what an
>>    "automatic merge" would look like)[1].
>>
>>    Notes: Possibly for cherry-picks and reverts too.  Depends on C &
>>    E.
>>
>> C) Modify/extend how path-based and mode-based conflicts are
>>    communicated to the user.
>>
>>    Notes: Particularly important as a mechanism for handling
>>    challenges[2] with implementing the remerge-diff ability.  Need to
>>    send RFC to list with ideas to get feedback.
>>
>> D) Improve merge performance.
>>
>>    Notes: Includes 4-5 specific optimizations[5], some of which I
>>    expect to be somewhat invasive and thus may make more sense to just
>>    make part of the new merge strategy implemented in E.  Biggest
>>    optimization depends on F.
>>
>> E) Write a new merge strategy meant to replace merge-recursive.
>>
>>    Notes: Suggested by Junio[3][4].  Depends on F & G.
>>
>> F) Make file collision conflict types more consistent.
>>
>>    Notes: Specifically, make rename/rename(2to1) & rename/add
>>    conflicts behave more like add/add[6][7].  Depends on part of G.
>>    Would prefer H to be accepted first.
>>
>> G) Improve merge-related portion of testsuite.
>>
>>    Notes: Intended to help test new merge strategy with more
>>    confidence.  Will include approximately a dozen edge and corner
>>    cases where merge-recursive currently falls short.  Started at [8];
>>    see also [9].
>
> Most items forward-reference "Depends on (<later letter>) up to here;
> (H) seems independent, but might be a good first start.
> (G) [8] is queued as origin/en/merge-recursive-tests, or do you mean
> to expand (G) into a mini-framework of merge-testing? i.e. run the
> mini test framework multiple times, each using a different
> merge strategy, similar to submodule tests, e.g. see
> t/lib-submodule-update.sh and one of its users, t1013.

Sorry, I should have been more clear about G.
en/merge-recursive-tests is not G, it's just a preparation topic for
it (or, alternatively, the first of a few topics for G).  In short,
the idea for G was missing coverage of existing functionality for
merge-recursive.

In more detail...

The idea for G was that, if possible, I wanted to avoid a repeat of
the problems directory rename detection caused Junio where he had to
revert it from master and worried about mis-merges.  That could have
been avoided if we had additional tests in the testsuite covering the
already-up-to-date check, making me wonder if there were other
important cases where merge-recursive lacked coverage.  Rather than
finding untested cases which merge-recursive handles correctly, my
investigation mostly yielded additional testcases where
merge-recursive falls down, including:
  * about 3 different rename cases (non-recursive), one of which was
reported previously but isn't in the testsuite
  * about 4 different issues with submodules (non-recursive) that I
don't think are represented in the testsuite (a couple of which I
mentioned in an email to you on the list last year but never created
testcases for)
  * about 6 different issues with special types and/or mode conflicts
(recursive only)
  * a few additional tests, commentary, and an alternate idea for
improving directory/file conflict handling

>> H) Miscellaneous code cleanups irritating me while working on other
>>    changes[10].
>>
>>
>> My current plan was to work roughly in reverse, from H to A.  Some questions:
>>
>>   * Does any of this look objectionable?
>
> Going in order A-H seems slightly out-of-order to me, I'd think (H) and (G)
> would go first;
>
> (B) sounds like an independent feature, which could go in parallel?

B may sound like an independent feature, but it needs a merge
algorithm that doesn't mess with the working tree so it depends pretty
strongly on E.

>>   * Should I post RFC questions on A and C earlier?
>
> I would think so, it is easier to give feedback on code, I would think.

If the idea is to give feedback on *code* rather than just
ideas/tradeoffs/pinpointing-buggy-lines, then it sounds like you're
actually suggesting posting the RFC later rather than earlier?

Also, the bigger question for me wasn't so much "should I ask the list
about these changes?" before making them, but rather: Do folks want me
to bring these things up before I work on D & E -- even if I end up
not getting back to incorporating their answers for months until D & E
are completed and merged?

>>   * Should I split D and G?  (Current plan was to keep D together, but
>>     split G into five short slightly inter-dependent topics)
>
> I would have expected to have tests (G) as a companion of (A) or (C)
> rather than (D), as performance improvements would keep the test suite
> unchanged?

Let me re-phrase: D and G are completely independent series, both of
which happen to be divisible.  Should either of them be split?

More background: D is only a handful of commits, so far; the main
reason to split it is to allow some of it to go first (maybe even
before G or H).  The downside is introducing extra churn and risk in
merge-recursive, when I'm planning to rewrite it soon anyway.  I was
trying to minimize merge-recursive changes, other than trying to make
sure that the new merge strategy and merge-recursive will give
identical results (modulo maybe fixing a few extra corner cases and
running faster).  Basically, I wanted it to be really easy to compare
old and new strategies, but otherwise wanted to leave merge-recursive
mostly alone.  It's not entirely clear how quickly I'll find time to
work on all of this, though, so maybe just-wait-for-the-rewrite is not
the right prioritization.

(And yes, A & C deserve tests, but they're not of the
missing-coverage-for-merge-recursive variety that I was thinking of
for G.)

>>   * E is kind of big.  Are there any specific things folks would like to see
>>     with how that is handled?
>
> How much abstraction can be done ahead of time such that there is an
> interface/API where you just plug in a new merge strategy and do not
> need to duplicate a lot of code/tests?

For avoiding duplicate tests, my plan was to use a variable (like
GIT_TEST_SPLIT_INDEX or GIT_FSMONITOR_TEST), which, when set, would
change the default merge strategy from "recursive" to the new one, and
also replace explicit requests for the "recursive" merge strategy with
the new one.

For avoiding duplicate code...well, Junio's suggestion was "[to
rewrite] without using much from the existing code in
merge-recursive.c at all" because he has "written off that code as
mostly unsalvageable"[4].

There are some functions that I think are worth leaving intact (e.g.
helpers like output(), the directory rename detection stuff, or the
merge_submodule code that you recently moved).  That stuff I was
planning to put in a "/* COPIED FROM merge-recursive.c VERBATIM */"
section.  And eventually, the idea would be to delete the old
merge-recursive and just use the replacement, making the duplication
concerns eventually go away.

...or is your question more about how to abstract things so that
others can also write new merge strategies in the future and allow as
much code and test re-use as possible?


>> [4] https://public-inbox.org/git/xmqqk1ydkbx0.fsf@gitster.mtv.corp.google.com/
>>    [New strategy]


Thanks for taking a look!

Elijah

^ permalink raw reply	[relevance 2%]

* Re: RFC: Merge-related plans
  2018-05-29 21:03   ` Elijah Newren
@ 2018-05-29 22:12     ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-29 22:12 UTC (permalink / raw)
  To: Elijah Newren, Thomas Rast; +Cc: Git Mailing List

context: https://public-inbox.org/git/CABPp-BFQJZHfCJZ1qvhvVcMd-_sOfi0Fkm5PexEwzzN+Zw2akw@mail.gmail.com/

Hi Elijah,

>> Most items forward-reference "Depends on (<later letter>) up to here;
>> (H) seems independent, but might be a good first start.
>> (G) [8] is queued as origin/en/merge-recursive-tests, or do you mean
>> to expand (G) into a mini-framework of merge-testing? i.e. run the
>> mini test framework multiple times, each using a different
>> merge strategy, similar to submodule tests, e.g. see
>> t/lib-submodule-update.sh and one of its users, t1013.
>
> Sorry, I should have been more clear about G.
>
> In more detail...
>
> [...]

Yeah that is what I had in mind when talking about a mini test frame work
above, such that you can run the same test on any merge strategy easily.
Sorry for being confusing there.

>> (B) sounds like an independent feature, which could go in parallel?
>
> B may sound like an independent feature, but it needs a merge
> algorithm that doesn't mess with the working tree

I agree on that,

> so it depends pretty strongly on E.

.. but not quite on the conclusion:
You could also make the current merge algorithm working
tree independent. Thomas Rast (cc'd) did so IIUC in
https://public-inbox.org/git/cover.1409860234.git.tr@thomasrast.ch/
(search the archive for "--remerge" if interested in this prior work)
which did not land upstream.

>>>   * Should I post RFC questions on A and C earlier?
>>
>> I would think so, it is easier to give feedback on code, I would think.
>
> If the idea is to give feedback on *code* rather than just
> ideas/tradeoffs/pinpointing-buggy-lines, then it sounds like you're
> actually suggesting posting the RFC later rather than earlier?

Some people have complained that they don't get feedback on
ideas/tradeoffs/pinpointing-buggy-lines, but did so after sending
RFC code. So I'd think a sure way to get feedback is to send actual
code as an RFC even if it misses some parts.

> Also, the bigger question for me wasn't so much "should I ask the list
> about these changes?" before making them, but rather: Do folks want me
> to bring these things up before I work on D & E -- even if I end up
> not getting back to incorporating their answers for months until D & E
> are completed and merged?

I'd think A + C are worth asking early nevertheless, even if D & E are
away for some month; having these niche cases covered (in code and
tests) as well as a new UI/UX for user communication sound exciting
(The latter could result in some bike shedding, and then having enough
time before you spend time to do it one way or another in D&E sounds
favorable)

>>>   * Should I split D and G?  (Current plan was to keep D together, but
>>>     split G into five short slightly inter-dependent topics)
>>
>> I would have expected to have tests (G) as a companion of (A) or (C)
>> rather than (D), as performance improvements would keep the test suite
>> unchanged?
>
> Let me re-phrase: D and G are completely independent series, both of
> which happen to be divisible.  Should either of them be split?

Sorry for my confusion. This clears it up

>
> More background: D is only a handful of commits, so far; the main
> reason to split it is to allow some of it to go first (maybe even
> before G or H).  The downside is introducing extra churn and risk in
> merge-recursive, when I'm planning to rewrite it soon anyway.  I was
> trying to minimize merge-recursive changes, other than trying to make
> sure that the new merge strategy and merge-recursive will give
> identical results (modulo maybe fixing a few extra corner cases and
> running faster).  Basically, I wanted it to be really easy to compare
> old and new strategies, but otherwise wanted to leave merge-recursive
> mostly alone.  It's not entirely clear how quickly I'll find time to
> work on all of this, though, so maybe just-wait-for-the-rewrite is not
> the right prioritization.

My experience is that smaller patch series are reviewed faster,
so if the cost of splitting them up is not prohibitive, I'd strongly
consider doing that.

>>>   * E is kind of big.  Are there any specific things folks would like to see
>>>     with how that is handled?
>>
>> How much abstraction can be done ahead of time such that there is an
>> interface/API where you just plug in a new merge strategy and do not
>> need to duplicate a lot of code/tests?
>
> For avoiding duplicate tests, my plan was to use a variable (like
> GIT_TEST_SPLIT_INDEX or GIT_FSMONITOR_TEST), which, when set, would
> change the default merge strategy from "recursive" to the new one, and
> also replace explicit requests for the "recursive" merge strategy with
> the new one.

So your long term plan is to *replace* the whole merge recursive strategy
giving the same results [analogous to implementing refstable as a new refs
back end, that behaves the same except faster and easier code for corner
cases] and not offer a *new* strategy [analog here: different (merge/)diff
strategies, such as Myers vs minimal vs patience] that yields user visible
difference and it is unclear whether the user prefers one or the other?
That wasn't clear to me, thanks for clarifying.

My original reply was assuming a new "mode" of merge strategy which
would be a user visible knob to turn. The environment variable you mentioned
sounds like that is an implementation detail, that is just for a
smooth transition
period, but the current code would be deprecated and deleted eventually.

> For avoiding duplicate code...well, Junio's suggestion was "[to
> rewrite] without using much from the existing code in
> merge-recursive.c at all" because he has "written off that code as
> mostly unsalvageable"[4].
>
> There are some functions that I think are worth leaving intact (e.g.
> helpers like output(), the directory rename detection stuff, or the
> merge_submodule code that you recently moved).  That stuff I was
> planning to put in a "/* COPIED FROM merge-recursive.c VERBATIM */"
> section.  And eventually, the idea would be to delete the old
> merge-recursive and just use the replacement, making the duplication
> concerns eventually go away.

This clears up even more, thanks for clarifying!

> ...or is your question more about how to abstract things so that
> others can also write new merge strategies in the future and allow as
> much code and test re-use as possible?

That was my original question as I thought an API would help with
doing that as it would help to not fiddle around with the rest of the
code base.

>>> [4] https://public-inbox.org/git/xmqqk1ydkbx0.fsf@gitster.mtv.corp.google.com/

Did you mean:
https://public-inbox.org/git/xmqqd147kpdm.fsf@gitster.mtv.corp.google.com/

So in that case my reply above ("..so it depends pretty strongly on E.."
I have a different conclusion) is void, as that *is* the new strategy?

Thanks,
Stefan

^ permalink raw reply	[relevance 2%]

* [RFC PATCH 00/35] object-store: lookup_commit
@ 2018-05-30  0:47 Stefan Beller
  2018-05-30  0:47 ` [PATCH 01/35] object: add repository argument to parse_object Stefan Beller
                   ` (4 more replies)
  0 siblings, 5 replies; 200+ results
From: Stefan Beller @ 2018-05-30  0:47 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

This applies on the merge of nd/commit-util-to-slab and sb/object-store-grafts,
and is available at http://github.com/stefanbeller/ as branch object-store-lookup-commit
as the merge has some merge conflicts as well as syntactical conflicts (upload-pack.c
and fetch-pack.c introduce new calls of functions that would want to take a repository struct
in the object-store-grafts series)

As layed out in https://public-inbox.org/git/20180517225154.9200-1-sbeller@google.com/
this is getting close to finishing the set of object store series though the last
unfinished part of this RFC hints at new work on the plate:
* To give this series a nice polish, we'd want to convert parse_commit, too.
  But that requires the conversion of the new commit graph. Maybe we need
  to split this series into 2. 
* Once this is in good shape we can talk about converting parts of the revision
  walking code,
* which then can be used by the submodule code as the end goal for the
  object store series.

Thanks,
Stefan

Jonathan Nieder (1):
  object: add repository argument to lookup_unknown_object

Stefan Beller (34):
  object: add repository argument to parse_object
  object: add repository argument to lookup_object
  object: add repository argument to parse_object_buffer
  object: add repository argument to object_as_type
  blob: add repository argument to lookup_blob
  tree: add repository argument to lookup_tree
  commit: add repository argument to lookup_commit_reference_gently
  commit: add repository argument to lookup_commit_reference
  commit: add repository argument to lookup_commit
  commit: add repository argument to parse_commit_buffer
  commit: add repository argument to set_commit_buffer
  commit: add repository argument to get_cached_commit_buffer
  tag: add repository argument to lookup_tag
  tag: add repository argument to parse_tag_buffer
  tag: add repository argument to deref_tag
  object: add repository argument to parse_commit_gently
  commit: add repository argument to parse_commit
  object: allow object_as_type to handle arbitrary repositories
  object: allow lookup_object to handle arbitrary repositories
  blob: allow lookup_blob to handle arbitrary repositories
  tree: allow lookup_tree to handle arbitrary repositories
  commit: allow lookup_commit to handle arbitrary repositories
  tag: allow lookup_tag to handle arbitrary repositories
  tag: allow parse_tag_buffer to handle arbitrary repositories
  commit.c: allow parse_commit_buffer to handle arbitrary repositories
  commit-slabs: remove realloc counter outside of slab struct
  commit.c: migrate the commit buffer to the parsed object store
  commit.c: allow set_commit_buffer to handle arbitrary repositories
  commit.c: allow get_cached_commit_buffer to handle arbitrary
    repositories
  object.c: allow parse_object_buffer to handle arbitrary repositories
  object.c: allow parse_object to handle arbitrary repositories
  tag.c: allow deref_tag to handle arbitrary repositories
  commit.c: allow lookup_commit_reference_gently to handle arbitrary
    repositories
  commit.c: allow lookup_commit_reference to handle arbitrary
    repositories

 archive.c                        |   2 +-
 bisect.c                         |   2 +-
 blame.c                          |  17 +++---
 blob.c                           |  10 ++--
 blob.h                           |   2 +-
 branch.c                         |   2 +-
 builtin/am.c                     |   9 ++-
 builtin/branch.c                 |   7 ++-
 builtin/checkout.c               |  10 ++--
 builtin/clone.c                  |   3 +-
 builtin/commit-tree.c            |   4 +-
 builtin/commit.c                 |   2 +-
 builtin/describe.c               |  17 +++---
 builtin/diff-tree.c              |   9 +--
 builtin/diff.c                   |   7 ++-
 builtin/fast-export.c            |  14 +++--
 builtin/fetch.c                  |   9 ++-
 builtin/fmt-merge-msg.c          |   9 ++-
 builtin/fsck.c                   |  22 ++++---
 builtin/grep.c                   |   3 +-
 builtin/index-pack.c             |   5 +-
 builtin/log.c                    |  10 ++--
 builtin/merge-base.c             |   9 +--
 builtin/merge-tree.c             |   3 +-
 builtin/name-rev.c               |  15 +++--
 builtin/notes.c                  |   5 +-
 builtin/pack-objects.c           |   4 +-
 builtin/prune.c                  |   2 +-
 builtin/pull.c                   |  15 +++--
 builtin/receive-pack.c           |   6 +-
 builtin/reflog.c                 |  17 +++---
 builtin/replace.c                |   4 +-
 builtin/reset.c                  |   4 +-
 builtin/rev-list.c               |   2 +-
 builtin/rev-parse.c              |   6 +-
 builtin/show-branch.c            |   9 +--
 builtin/tag.c                    |   2 +-
 builtin/unpack-objects.c         |   7 ++-
 builtin/verify-commit.c          |   4 +-
 bundle.c                         |  10 ++--
 cache-tree.c                     |   3 +-
 commit-graph.c                   |  20 +++----
 commit-slab-impl.h               |   3 -
 commit.c                         | 100 ++++++++++++++++++-------------
 commit.h                         |  24 +++++---
 fast-import.c                    |   6 +-
 fetch-pack.c                     |  45 ++++++++------
 fsck.c                           |   9 +--
 http-backend.c                   |   4 +-
 http-push.c                      |  18 +++---
 line-log.c                       |   2 +-
 list-objects.c                   |   4 +-
 log-tree.c                       |  13 ++--
 merge-recursive.c                |  15 +++--
 notes-cache.c                    |   3 +-
 notes-merge.c                    |   5 +-
 notes-utils.c                    |   6 +-
 object.c                         |  63 ++++++++++---------
 object.h                         |  13 ++--
 packfile.c                       |   2 +-
 parse-options-cb.c               |   2 +-
 pretty.c                         |   4 +-
 reachable.c                      |   8 +--
 ref-filter.c                     |  12 ++--
 reflog-walk.c                    |   3 +-
 refs.c                           |   4 +-
 refs/files-backend.c             |   2 +-
 remote.c                         |  21 ++++---
 revision.c                       |  37 ++++++------
 sequencer.c                      |  30 +++++-----
 server-info.c                    |   4 +-
 sha1-file.c                      |   4 +-
 sha1-name.c                      |  37 ++++++------
 shallow.c                        |  34 +++++++----
 submodule.c                      |  10 ++--
 t/helper/test-example-decorate.c |   6 +-
 tag.c                            |  28 ++++-----
 tag.h                            |   7 +--
 tree.c                           |  21 +++----
 tree.h                           |   2 +-
 upload-pack.c                    |  19 +++---
 walker.c                         |  16 +++--
 wt-status.c                      |   2 +-
 83 files changed, 547 insertions(+), 423 deletions(-)

-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 5%]

* [PATCH 01/35] object: add repository argument to parse_object
  2018-05-30  0:47 [RFC PATCH 00/35] object-store: lookup_commit Stefan Beller
@ 2018-05-30  0:47 ` Stefan Beller
  2018-05-30  0:47 ` [PATCH 09/35] commit: add repository argument to lookup_commit_reference Stefan Beller
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-30  0:47 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller, Jonathan Nieder

Add a repository argument to allow the callers of parse_object
to be more specific about which repository to act on. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/diff-tree.c     |  3 ++-
 builtin/diff.c          |  2 +-
 builtin/fast-export.c   |  2 +-
 builtin/fmt-merge-msg.c |  6 ++++--
 builtin/fsck.c          |  4 ++--
 builtin/log.c           |  3 ++-
 builtin/name-rev.c      |  7 ++++---
 builtin/receive-pack.c  |  6 +++---
 builtin/reflog.c        |  3 ++-
 builtin/rev-list.c      |  2 +-
 bundle.c                |  5 +++--
 commit.c                |  5 +++--
 fetch-pack.c            | 18 ++++++++++--------
 fsck.c                  |  3 ++-
 http-backend.c          |  2 +-
 http-push.c             |  6 ++++--
 log-tree.c              |  7 ++++---
 merge-recursive.c       |  4 +++-
 object.c                |  4 ++--
 object.h                |  3 ++-
 packfile.c              |  2 +-
 pretty.c                |  2 +-
 ref-filter.c            |  3 ++-
 reflog-walk.c           |  3 ++-
 refs/files-backend.c    |  2 +-
 remote.c                |  4 ++--
 revision.c              | 14 +++++++-------
 server-info.c           |  2 +-
 sha1-name.c             | 14 +++++++-------
 tag.c                   |  5 +++--
 tree.c                  |  5 +++--
 upload-pack.c           | 13 +++++++------
 walker.c                |  3 ++-
 33 files changed, 95 insertions(+), 72 deletions(-)

diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index 473615117e0..d8db8f682f0 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -5,6 +5,7 @@
 #include "log-tree.h"
 #include "builtin.h"
 #include "submodule.h"
+#include "repository.h"
 
 static struct rev_info log_tree_opt;
 
@@ -68,7 +69,7 @@ static int diff_tree_stdin(char *line)
 	line[len-1] = 0;
 	if (parse_oid_hex(line, &oid, &p))
 		return -1;
-	obj = parse_object(&oid);
+	obj = parse_object(the_repository, &oid);
 	if (!obj)
 		return -1;
 	if (obj->type == OBJ_COMMIT)
diff --git a/builtin/diff.c b/builtin/diff.c
index 16bfb22f738..ed6092ef1a3 100644
--- a/builtin/diff.c
+++ b/builtin/diff.c
@@ -393,7 +393,7 @@ int cmd_diff(int argc, const char **argv, const char *prefix)
 		const char *name = entry->name;
 		int flags = (obj->flags & UNINTERESTING);
 		if (!obj->parsed)
-			obj = parse_object(&obj->oid);
+			obj = parse_object(the_repository, &obj->oid);
 		obj = deref_tag(obj, NULL, 0);
 		if (!obj)
 			die(_("invalid object '%s' given."), name);
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index cb4b1c7f815..0da875b58c9 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -802,7 +802,7 @@ static struct commit *get_commit(struct rev_cmdline_entry *e, char *full_name)
 
 		/* handle nested tags */
 		while (tag && tag->object.type == OBJ_TAG) {
-			parse_object(&tag->object.oid);
+			parse_object(the_repository, &tag->object.oid);
 			string_list_append(&extra_refs, full_name)->util = tag;
 			tag = (struct tag *)tag->tagged;
 		}
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index 1b526adb3a9..5e44589b545 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -11,6 +11,7 @@
 #include "branch.h"
 #include "fmt-merge-msg.h"
 #include "gpg-interface.h"
+#include "repository.h"
 
 static const char * const fmt_merge_msg_usage[] = {
 	N_("git fmt-merge-msg [-m <message>] [--log[=<n>] | --no-log] [--file <file>]"),
@@ -343,7 +344,8 @@ static void shortlog(const char *name,
 	const struct object_id *oid = &origin_data->oid;
 	int limit = opts->shortlog_len;
 
-	branch = deref_tag(parse_object(oid), oid_to_hex(oid), GIT_SHA1_HEXSZ);
+	branch = deref_tag(parse_object(the_repository, oid), oid_to_hex(oid),
+			   GIT_SHA1_HEXSZ);
 	if (!branch || branch->type != OBJ_COMMIT)
 		return;
 
@@ -563,7 +565,7 @@ static void find_merge_parents(struct merge_parents *result,
 		 * "name" here and we do not want to contaminate its
 		 * util field yet.
 		 */
-		obj = parse_object(&oid);
+		obj = parse_object(the_repository, &oid);
 		parent = (struct commit *)peel_to_type(NULL, 0, obj, OBJ_COMMIT);
 		if (!parent)
 			continue;
diff --git a/builtin/fsck.c b/builtin/fsck.c
index 9d59d7d5a21..4b410cba54e 100644
--- a/builtin/fsck.c
+++ b/builtin/fsck.c
@@ -452,7 +452,7 @@ static int fsck_handle_ref(const char *refname, const struct object_id *oid,
 {
 	struct object *obj;
 
-	obj = parse_object(oid);
+	obj = parse_object(the_repository, oid);
 	if (!obj) {
 		if (is_promisor_object(oid)) {
 			/*
@@ -616,7 +616,7 @@ static int fsck_cache_tree(struct cache_tree *it)
 		fprintf(stderr, "Checking cache tree\n");
 
 	if (0 <= it->entry_count) {
-		struct object *obj = parse_object(&it->oid);
+		struct object *obj = parse_object(the_repository, &it->oid);
 		if (!obj) {
 			error("%s: invalid sha1 pointer in cache-tree",
 			      oid_to_hex(&it->oid));
diff --git a/builtin/log.c b/builtin/log.c
index cff0d58d698..063057e45de 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -30,6 +30,7 @@
 #include "gpg-interface.h"
 #include "progress.h"
 #include "commit-slab.h"
+#include "repository.h"
 
 #define MAIL_DEFAULT_WRAP 72
 
@@ -619,7 +620,7 @@ int cmd_show(int argc, const char **argv, const char *prefix)
 			rev.shown_one = 1;
 			if (ret)
 				break;
-			o = parse_object(&t->tagged->oid);
+			o = parse_object(the_repository, &t->tagged->oid);
 			if (!o)
 				ret = error(_("Could not read object %s"),
 					    oid_to_hex(&t->tagged->oid));
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 0eb440359dd..de54fa93e4f 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -1,5 +1,6 @@
 #include "builtin.h"
 #include "cache.h"
+#include "repository.h"
 #include "config.h"
 #include "commit.h"
 #include "tag.h"
@@ -203,7 +204,7 @@ static int tipcmp(const void *a_, const void *b_)
 
 static int name_ref(const char *path, const struct object_id *oid, int flags, void *cb_data)
 {
-	struct object *o = parse_object(oid);
+	struct object *o = parse_object(the_repository, oid);
 	struct name_ref_data *data = cb_data;
 	int can_abbreviate_output = data->tags_only && data->name_only;
 	int deref = 0;
@@ -261,7 +262,7 @@ static int name_ref(const char *path, const struct object_id *oid, int flags, vo
 		struct tag *t = (struct tag *) o;
 		if (!t->tagged)
 			break; /* broken repository */
-		o = parse_object(&t->tagged->oid);
+		o = parse_object(the_repository, &t->tagged->oid);
 		deref = 1;
 		taggerdate = t->date;
 	}
@@ -451,7 +452,7 @@ int cmd_name_rev(int argc, const char **argv, const char *prefix)
 		}
 
 		commit = NULL;
-		object = parse_object(&oid);
+		object = parse_object(the_repository, &oid);
 		if (object) {
 			struct object *peeled = deref_tag(object, *argv, 0);
 			if (peeled && peeled->type == OBJ_COMMIT)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 7adb61a37f2..0a74fcc7b0a 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -1108,8 +1108,8 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 		struct object *old_object, *new_object;
 		struct commit *old_commit, *new_commit;
 
-		old_object = parse_object(old_oid);
-		new_object = parse_object(new_oid);
+		old_object = parse_object(the_repository, old_oid);
+		new_object = parse_object(the_repository, new_oid);
 
 		if (!old_object || !new_object ||
 		    old_object->type != OBJ_COMMIT ||
@@ -1132,7 +1132,7 @@ static const char *update(struct command *cmd, struct shallow_info *si)
 
 	if (is_null_oid(new_oid)) {
 		struct strbuf err = STRBUF_INIT;
-		if (!parse_object(old_oid)) {
+		if (!parse_object(the_repository, old_oid)) {
 			old_oid = NULL;
 			if (ref_exists(name)) {
 				rp_warning("Allowing deletion of corrupt ref.");
diff --git a/builtin/reflog.c b/builtin/reflog.c
index bee4795a936..3a751fbaa60 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "lockfile.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "refs.h"
 #include "dir.h"
@@ -129,7 +130,7 @@ static int commit_is_complete(struct commit *commit)
 		struct commit_list *parent;
 
 		c = (struct commit *)object_array_pop(&study);
-		if (!c->object.parsed && !parse_object(&c->object.oid))
+		if (!c->object.parsed && !parse_object(the_repository, &c->object.oid))
 			c->object.flags |= INCOMPLETE;
 
 		if (c->object.flags & INCOMPLETE) {
diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index e9bd4e378ad..cbaaae83eae 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -239,7 +239,7 @@ static int finish_object(struct object *obj, const char *name, void *cb_data)
 		return 1;
 	}
 	if (info->revs->verify_objects && !obj->parsed && obj->type != OBJ_COMMIT)
-		parse_object(&obj->oid);
+		parse_object(the_repository, &obj->oid);
 	return 0;
 }
 
diff --git a/bundle.c b/bundle.c
index 00ada05f46c..9ef55f6bdb0 100644
--- a/bundle.c
+++ b/bundle.c
@@ -2,6 +2,7 @@
 #include "lockfile.h"
 #include "bundle.h"
 #include "object-store.h"
+#include "repository.h"
 #include "object.h"
 #include "commit.h"
 #include "diff.h"
@@ -142,7 +143,7 @@ int verify_bundle(struct bundle_header *header, int verbose)
 	init_revisions(&revs, NULL);
 	for (i = 0; i < p->nr; i++) {
 		struct ref_list_entry *e = p->list + i;
-		struct object *o = parse_object(&e->oid);
+		struct object *o = parse_object(the_repository, &e->oid);
 		if (o) {
 			o->flags |= PREREQ_MARK;
 			add_pending_object(&revs, o, e->name);
@@ -167,7 +168,7 @@ int verify_bundle(struct bundle_header *header, int verbose)
 
 	for (i = 0; i < p->nr; i++) {
 		struct ref_list_entry *e = p->list + i;
-		struct object *o = parse_object(&e->oid);
+		struct object *o = parse_object(the_repository, &e->oid);
 		assert(o); /* otherwise we'd have returned early */
 		if (o->flags & SHOWN)
 			continue;
diff --git a/commit.c b/commit.c
index ddb6c04738b..11db72e9db2 100644
--- a/commit.c
+++ b/commit.c
@@ -26,7 +26,8 @@ const char *commit_type = "commit";
 struct commit *lookup_commit_reference_gently(const struct object_id *oid,
 					      int quiet)
 {
-	struct object *obj = deref_tag(parse_object(oid), NULL, 0);
+	struct object *obj = deref_tag(parse_object(the_repository, oid),
+				       NULL, 0);
 
 	if (!obj)
 		return NULL;
@@ -1616,7 +1617,7 @@ struct commit *get_merge_parent(const char *name)
 	struct object_id oid;
 	if (get_oid(name, &oid))
 		return NULL;
-	obj = parse_object(&oid);
+	obj = parse_object(the_repository, &oid);
 	commit = (struct commit *)peel_to_type(name, 0, obj, OBJ_COMMIT);
 	if (commit && !merge_remote_util(commit))
 		set_merge_remote_desc(commit, name, obj);
diff --git a/fetch-pack.c b/fetch-pack.c
index 44e50b57d98..fa39ada77ee 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -84,7 +84,7 @@ static void cache_one_alternate(const char *refname,
 				void *vcache)
 {
 	struct alternate_object_cache *cache = vcache;
-	struct object *obj = parse_object(oid);
+	struct object *obj = parse_object(the_repository, oid);
 
 	if (!obj || (obj->flags & ALTERNATE))
 		return;
@@ -126,7 +126,8 @@ static void rev_list_push(struct commit *commit, int mark)
 
 static int rev_list_insert_ref(const char *refname, const struct object_id *oid)
 {
-	struct object *o = deref_tag(parse_object(oid), refname, 0);
+	struct object *o = deref_tag(parse_object(the_repository, oid),
+				     refname, 0);
 
 	if (o && o->type == OBJ_COMMIT)
 		rev_list_push((struct commit *)o, SEEN);
@@ -143,7 +144,8 @@ static int rev_list_insert_ref_oid(const char *refname, const struct object_id *
 static int clear_marks(const char *refname, const struct object_id *oid,
 		       int flag, void *cb_data)
 {
-	struct object *o = deref_tag(parse_object(oid), refname, 0);
+	struct object *o = deref_tag(parse_object(the_repository, oid),
+				     refname, 0);
 
 	if (o && o->type == OBJ_COMMIT)
 		clear_commit_marks((struct commit *)o,
@@ -437,7 +439,7 @@ static int find_common(struct fetch_pack_args *args,
 				if (!lookup_object(oid.hash))
 					die(_("object not found: %s"), line);
 				/* make sure that it is parsed as shallow */
-				if (!parse_object(&oid))
+				if (!parse_object(the_repository, &oid))
 					die(_("error in object: %s"), line);
 				if (unregister_shallow(&oid))
 					die(_("no shallow found: %s"), line);
@@ -570,14 +572,14 @@ static struct commit_list *complete;
 
 static int mark_complete(const struct object_id *oid)
 {
-	struct object *o = parse_object(oid);
+	struct object *o = parse_object(the_repository, oid);
 
 	while (o && o->type == OBJ_TAG) {
 		struct tag *t = (struct tag *) o;
 		if (!t->tagged)
 			break; /* broken repository */
 		o->flags |= COMPLETE;
-		o = parse_object(&t->tagged->oid);
+		o = parse_object(the_repository, &t->tagged->oid);
 	}
 	if (o && o->type == OBJ_COMMIT) {
 		struct commit *commit = (struct commit *)o;
@@ -768,7 +770,7 @@ static int everything_local(struct fetch_pack_args *args,
 
 		if (!has_object_file_with_flags(&ref->old_oid, flags))
 			continue;
-		o = parse_object(&ref->old_oid);
+		o = parse_object(the_repository, &ref->old_oid);
 		if (!o)
 			continue;
 
@@ -1296,7 +1298,7 @@ static void receive_shallow_info(struct fetch_pack_args *args,
 			if (!lookup_object(oid.hash))
 				die(_("object not found: %s"), reader->line);
 			/* make sure that it is parsed as shallow */
-			if (!parse_object(&oid))
+			if (!parse_object(the_repository, &oid))
 				die(_("error in object: %s"), reader->line);
 			if (unregister_shallow(&oid))
 				die(_("no shallow found: %s"), reader->line);
diff --git a/fsck.c b/fsck.c
index 104c3c0a434..f9476f56e93 100644
--- a/fsck.c
+++ b/fsck.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "object-store.h"
+#include "repository.h"
 #include "object.h"
 #include "blob.h"
 #include "tree.h"
@@ -462,7 +463,7 @@ int fsck_walk(struct object *obj, void *data, struct fsck_options *options)
 		return -1;
 
 	if (obj->type == OBJ_NONE)
-		parse_object(&obj->oid);
+		parse_object(the_repository, &obj->oid);
 
 	switch (obj->type) {
 	case OBJ_BLOB:
diff --git a/http-backend.c b/http-backend.c
index adaef16fadf..50ba4d53d54 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -436,7 +436,7 @@ static int show_text_ref(const char *name, const struct object_id *oid,
 {
 	const char *name_nons = strip_namespace(name);
 	struct strbuf *buf = cb_data;
-	struct object *o = parse_object(oid);
+	struct object *o = parse_object(the_repository, oid);
 	if (!o)
 		return 0;
 
diff --git a/http-push.c b/http-push.c
index f308ce0195a..311b6295359 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1,4 +1,5 @@
 #include "cache.h"
+#include "repository.h"
 #include "commit.h"
 #include "tag.h"
 #include "blob.h"
@@ -14,6 +15,7 @@
 #include "packfile.h"
 #include "object-store.h"
 
+
 #ifdef EXPAT_NEEDS_XMLPARSE_H
 #include <xmlparse.h>
 #else
@@ -722,7 +724,7 @@ static void one_remote_object(const struct object_id *oid)
 
 	obj = lookup_object(oid->hash);
 	if (!obj)
-		obj = parse_object(oid);
+		obj = parse_object(the_repository, oid);
 
 	/* Ignore remote objects that don't exist locally */
 	if (!obj)
@@ -1459,7 +1461,7 @@ static void add_remote_info_ref(struct remote_ls_ctx *ls)
 		return;
 	}
 
-	o = parse_object(&ref->old_oid);
+	o = parse_object(the_repository, &ref->old_oid);
 	if (!o) {
 		fprintf(stderr,
 			"Unable to parse object %s for remote ref %s\n",
diff --git a/log-tree.c b/log-tree.c
index 9c3053600a4..e8d2857bf06 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "diff.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "tag.h"
 #include "graph.h"
@@ -111,13 +112,13 @@ static int add_ref_decoration(const char *refname, const struct object_id *oid,
 			warning("invalid replace ref %s", refname);
 			return 0;
 		}
-		obj = parse_object(&original_oid);
+		obj = parse_object(the_repository, &original_oid);
 		if (obj)
 			add_name_decoration(DECORATION_GRAFTED, "replaced", obj);
 		return 0;
 	}
 
-	obj = parse_object(oid);
+	obj = parse_object(the_repository, oid);
 	if (!obj)
 		return 0;
 
@@ -138,7 +139,7 @@ static int add_ref_decoration(const char *refname, const struct object_id *oid,
 		if (!obj)
 			break;
 		if (!obj->parsed)
-			parse_object(&obj->oid);
+			parse_object(the_repository, &obj->oid);
 		add_name_decoration(DECORATION_REF_TAG, refname, obj);
 	}
 	return 0;
diff --git a/merge-recursive.c b/merge-recursive.c
index 41cf9512d63..8ddedf79c0b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -9,6 +9,7 @@
 #include "lockfile.h"
 #include "cache-tree.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "blob.h"
 #include "builtin.h"
@@ -2183,7 +2184,8 @@ static struct commit *get_ref(const struct object_id *oid, const char *name)
 {
 	struct object *object;
 
-	object = deref_tag(parse_object(oid), name, strlen(name));
+	object = deref_tag(parse_object(the_repository, oid), name,
+			   strlen(name));
 	if (!object)
 		return NULL;
 	if (object->type == OBJ_TREE)
diff --git a/object.c b/object.c
index 30b8a721cf6..469e6126286 100644
--- a/object.c
+++ b/object.c
@@ -239,14 +239,14 @@ struct object *parse_object_buffer(const struct object_id *oid, enum object_type
 struct object *parse_object_or_die(const struct object_id *oid,
 				   const char *name)
 {
-	struct object *o = parse_object(oid);
+	struct object *o = parse_object(the_repository, oid);
 	if (o)
 		return o;
 
 	die(_("unable to parse object: %s"), name ? name : oid_to_hex(oid));
 }
 
-struct object *parse_object(const struct object_id *oid)
+struct object *parse_object_the_repository(const struct object_id *oid)
 {
 	unsigned long size;
 	enum object_type type;
diff --git a/object.h b/object.h
index 998145de46a..f69095d3dde 100644
--- a/object.h
+++ b/object.h
@@ -121,7 +121,8 @@ void *object_as_type(struct object *obj, enum object_type type, int quiet);
  *
  * Returns NULL if the object is missing or corrupt.
  */
-struct object *parse_object(const struct object_id *oid);
+#define parse_object(r, oid) parse_object_##r(oid)
+struct object *parse_object_the_repository(const struct object_id *oid);
 
 /*
  * Like parse_object, but will die() instead of returning NULL. If the
diff --git a/packfile.c b/packfile.c
index 1ec86930209..85ef0909087 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1924,7 +1924,7 @@ static int add_promisor_object(const struct object_id *oid,
 			       void *set_)
 {
 	struct oidset *set = set_;
-	struct object *obj = parse_object(oid);
+	struct object *obj = parse_object(the_repository, oid);
 	if (!obj)
 		return 1;
 
diff --git a/pretty.c b/pretty.c
index 34fe891fc03..c99f8243faf 100644
--- a/pretty.c
+++ b/pretty.c
@@ -1146,7 +1146,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
 
 	/* these depend on the commit */
 	if (!commit->object.parsed)
-		parse_object(&commit->object.oid);
+		parse_object(the_repository, &commit->object.oid);
 
 	switch (placeholder[0]) {
 	case 'H':		/* commit hash */
diff --git a/ref-filter.c b/ref-filter.c
index 7b9e44dcaba..6ebb4630f9c 100644
--- a/ref-filter.c
+++ b/ref-filter.c
@@ -4,6 +4,7 @@
 #include "refs.h"
 #include "wildmatch.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "remote.h"
 #include "color.h"
@@ -1898,7 +1899,7 @@ static const struct object_id *match_points_at(struct oid_array *points_at,
 
 	if (oid_array_lookup(points_at, oid) >= 0)
 		return oid;
-	obj = parse_object(oid);
+	obj = parse_object(the_repository, oid);
 	if (!obj)
 		die(_("malformed object at '%s'"), refname);
 	if (obj->type == OBJ_TAG)
diff --git a/reflog-walk.c b/reflog-walk.c
index 5008bbf6ada..3561a8b9558 100644
--- a/reflog-walk.c
+++ b/reflog-walk.c
@@ -305,7 +305,8 @@ static struct commit *next_reflog_commit(struct commit_reflog *log)
 {
 	for (; log->recno >= 0; log->recno--) {
 		struct reflog_info *entry = &log->reflogs->items[log->recno];
-		struct object *obj = parse_object(&entry->noid);
+		struct object *obj = parse_object(the_repository,
+						  &entry->noid);
 
 		if (obj && obj->type == OBJ_COMMIT)
 			return (struct commit *)obj;
diff --git a/refs/files-backend.c b/refs/files-backend.c
index 49d8f67bf13..942f6a0c219 100644
--- a/refs/files-backend.c
+++ b/refs/files-backend.c
@@ -1660,7 +1660,7 @@ static int write_ref_to_lockfile(struct ref_lock *lock,
 	struct object *o;
 	int fd;
 
-	o = parse_object(oid);
+	o = parse_object(the_repository, oid);
 	if (!o) {
 		strbuf_addf(err,
 			    "trying to write ref '%s' with nonexistent object %s",
diff --git a/remote.c b/remote.c
index db438cfbc56..92a17828d25 100644
--- a/remote.c
+++ b/remote.c
@@ -1994,12 +1994,12 @@ int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)
 	 * Both new_commit and old_commit must be commit-ish and new_commit is descendant of
 	 * old_commit.  Otherwise we require --force.
 	 */
-	o = deref_tag(parse_object(old_oid), NULL, 0);
+	o = deref_tag(parse_object(the_repository, old_oid), NULL, 0);
 	if (!o || o->type != OBJ_COMMIT)
 		return 0;
 	old_commit = (struct commit *) o;
 
-	o = deref_tag(parse_object(new_oid), NULL, 0);
+	o = deref_tag(parse_object(the_repository, new_oid), NULL, 0);
 	if (!o || o->type != OBJ_COMMIT)
 		return 0;
 	new_commit = (struct commit *) o;
diff --git a/revision.c b/revision.c
index a00c43219fb..f051c6c5806 100644
--- a/revision.c
+++ b/revision.c
@@ -187,7 +187,7 @@ void add_head_to_pending(struct rev_info *revs)
 	struct object *obj;
 	if (get_oid("HEAD", &oid))
 		return;
-	obj = parse_object(&oid);
+	obj = parse_object(the_repository, &oid);
 	if (!obj)
 		return;
 	add_pending_object(revs, obj, "HEAD");
@@ -199,7 +199,7 @@ static struct object *get_reference(struct rev_info *revs, const char *name,
 {
 	struct object *object;
 
-	object = parse_object(oid);
+	object = parse_object(the_repository, oid);
 	if (!object) {
 		if (revs->ignore_missing)
 			return object;
@@ -236,7 +236,7 @@ static struct commit *handle_commit(struct rev_info *revs,
 			add_pending_object(revs, object, tag->tag);
 		if (!tag->tagged)
 			die("bad tag");
-		object = parse_object(&tag->tagged->oid);
+		object = parse_object(the_repository, &tag->tagged->oid);
 		if (!object) {
 			if (revs->ignore_missing_links || (flags & UNINTERESTING))
 				return NULL;
@@ -1239,7 +1239,7 @@ static void handle_one_reflog_commit(struct object_id *oid, void *cb_data)
 {
 	struct all_refs_cb *cb = cb_data;
 	if (!is_null_oid(oid)) {
-		struct object *o = parse_object(oid);
+		struct object *o = parse_object(the_repository, oid);
 		if (o) {
 			o->flags |= cb->all_flags;
 			/* ??? CMDLINEFLAGS ??? */
@@ -1567,8 +1567,8 @@ static int handle_dotdot_1(const char *arg, char *dotdot,
 		*dotdot = '\0';
 	}
 
-	a_obj = parse_object(&a_oid);
-	b_obj = parse_object(&b_oid);
+	a_obj = parse_object(the_repository, &a_oid);
+	b_obj = parse_object(the_repository, &b_oid);
 	if (!a_obj || !b_obj)
 		return dotdot_missing(arg, dotdot, revs, symmetric);
 
@@ -2872,7 +2872,7 @@ static int mark_uninteresting(const struct object_id *oid,
 			      uint32_t pos,
 			      void *unused)
 {
-	struct object *o = parse_object(oid);
+	struct object *o = parse_object(the_repository, oid);
 	o->flags |= UNINTERESTING | SEEN;
 	return 0;
 }
diff --git a/server-info.c b/server-info.c
index 83460ec0d6f..368fb547f4b 100644
--- a/server-info.c
+++ b/server-info.c
@@ -56,7 +56,7 @@ static int add_info_ref(const char *path, const struct object_id *oid,
 			int flag, void *cb_data)
 {
 	FILE *fp = cb_data;
-	struct object *o = parse_object(oid);
+	struct object *o = parse_object(the_repository, oid);
 	if (!o)
 		return -1;
 
diff --git a/sha1-name.c b/sha1-name.c
index b5406b6eb21..3e1c57e1539 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -239,7 +239,7 @@ static int disambiguate_committish_only(const struct object_id *oid, void *cb_da
 		return 0;
 
 	/* We need to do this the hard way... */
-	obj = deref_tag(parse_object(oid), NULL, 0);
+	obj = deref_tag(parse_object(the_repository, oid), NULL, 0);
 	if (obj && obj->type == OBJ_COMMIT)
 		return 1;
 	return 0;
@@ -263,7 +263,7 @@ static int disambiguate_treeish_only(const struct object_id *oid, void *cb_data_
 		return 0;
 
 	/* We need to do this the hard way... */
-	obj = deref_tag(parse_object(oid), NULL, 0);
+	obj = deref_tag(parse_object(the_repository, oid), NULL, 0);
 	if (obj && (obj->type == OBJ_TREE || obj->type == OBJ_COMMIT))
 		return 1;
 	return 0;
@@ -857,7 +857,7 @@ struct object *peel_to_type(const char *name, int namelen,
 	if (name && !namelen)
 		namelen = strlen(name);
 	while (1) {
-		if (!o || (!o->parsed && !parse_object(&o->oid)))
+		if (!o || (!o->parsed && !parse_object(the_repository, &o->oid)))
 			return NULL;
 		if (expected_type == OBJ_ANY || o->type == expected_type)
 			return o;
@@ -930,12 +930,12 @@ static int peel_onion(const char *name, int len, struct object_id *oid,
 	if (get_oid_1(name, sp - name - 2, &outer, lookup_flags))
 		return -1;
 
-	o = parse_object(&outer);
+	o = parse_object(the_repository, &outer);
 	if (!o)
 		return -1;
 	if (!expected_type) {
 		o = deref_tag(o, name, sp - name - 2);
-		if (!o || (!o->parsed && !parse_object(&o->oid)))
+		if (!o || (!o->parsed && !parse_object(the_repository, &o->oid)))
 			return -1;
 		oidcpy(oid, &o->oid);
 		return 0;
@@ -1062,7 +1062,7 @@ static int handle_one_ref(const char *path, const struct object_id *oid,
 			  int flag, void *cb_data)
 {
 	struct commit_list **list = cb_data;
-	struct object *object = parse_object(oid);
+	struct object *object = parse_object(the_repository, oid);
 	if (!object)
 		return 0;
 	if (object->type == OBJ_TAG) {
@@ -1108,7 +1108,7 @@ static int get_oid_oneline(const char *prefix, struct object_id *oid,
 		int matches;
 
 		commit = pop_most_recent_commit(&list, ONELINE_SEEN);
-		if (!parse_object(&commit->object.oid))
+		if (!parse_object(the_repository, &commit->object.oid))
 			continue;
 		buf = get_commit_buffer(commit, NULL);
 		p = strstr(buf, "\n\n");
diff --git a/tag.c b/tag.c
index 3be7206e920..7d282df0697 100644
--- a/tag.c
+++ b/tag.c
@@ -68,7 +68,8 @@ struct object *deref_tag(struct object *o, const char *warn, int warnlen)
 {
 	while (o && o->type == OBJ_TAG)
 		if (((struct tag *)o)->tagged)
-			o = parse_object(&((struct tag *)o)->tagged->oid);
+			o = parse_object(the_repository,
+					 &((struct tag *)o)->tagged->oid);
 		else
 			o = NULL;
 	if (!o && warn) {
@@ -82,7 +83,7 @@ struct object *deref_tag(struct object *o, const char *warn, int warnlen)
 struct object *deref_tag_noverify(struct object *o)
 {
 	while (o && o->type == OBJ_TAG) {
-		o = parse_object(&o->oid);
+		o = parse_object(the_repository, &o->oid);
 		if (o && o->type == OBJ_TAG && ((struct tag *)o)->tagged)
 			o = ((struct tag *)o)->tagged;
 		else
diff --git a/tree.c b/tree.c
index dc20a2693fd..1033e69b590 100644
--- a/tree.c
+++ b/tree.c
@@ -8,6 +8,7 @@
 #include "tag.h"
 #include "alloc.h"
 #include "tree-walk.h"
+#include "repository.h"
 
 const char *tree_type = "tree";
 
@@ -244,7 +245,7 @@ void free_tree_buffer(struct tree *tree)
 
 struct tree *parse_tree_indirect(const struct object_id *oid)
 {
-	struct object *obj = parse_object(oid);
+	struct object *obj = parse_object(the_repository, oid);
 	do {
 		if (!obj)
 			return NULL;
@@ -257,6 +258,6 @@ struct tree *parse_tree_indirect(const struct object_id *oid)
 		else
 			return NULL;
 		if (!obj->parsed)
-			parse_object(&obj->oid);
+			parse_object(the_repository, &obj->oid);
 	} while (1);
 }
diff --git a/upload-pack.c b/upload-pack.c
index 3627e8fafa5..afe997c89ec 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -3,6 +3,7 @@
 #include "refs.h"
 #include "pkt-line.h"
 #include "sideband.h"
+#include "repository.h"
 #include "object-store.h"
 #include "tag.h"
 #include "object.h"
@@ -311,7 +312,7 @@ static int got_oid(const char *hex, struct object_id *oid)
 	if (!has_object_file(oid))
 		return -1;
 
-	o = parse_object(oid);
+	o = parse_object(the_repository, oid);
 	if (!o)
 		die("oops (%s)", oid_to_hex(oid));
 	if (o->type == OBJ_COMMIT) {
@@ -349,7 +350,7 @@ static int reachable(struct commit *want)
 			break;
 		}
 		if (!commit->object.parsed)
-			parse_object(&commit->object.oid);
+			parse_object(the_repository, &commit->object.oid);
 		if (commit->object.flags & REACHABLE)
 			continue;
 		commit->object.flags |= REACHABLE;
@@ -798,7 +799,7 @@ static int process_shallow(const char *line, struct object_array *shallows)
 		struct object *object;
 		if (get_oid_hex(arg, &oid))
 			die("invalid shallow line: %s", line);
-		object = parse_object(&oid);
+		object = parse_object(the_repository, &oid);
 		if (!object)
 			return 1;
 		if (object->type != OBJ_COMMIT)
@@ -926,7 +927,7 @@ static void receive_needs(void)
 		if (allow_filter && parse_feature_request(features, "filter"))
 			filter_capability_requested = 1;
 
-		o = parse_object(&oid_buf);
+		o = parse_object(the_repository, &oid_buf);
 		if (!o) {
 			packet_write_fmt(1,
 					 "ERR upload-pack: not our ref %s",
@@ -1168,7 +1169,7 @@ static int parse_want(const char *line)
 			die("git upload-pack: protocol error, "
 			    "expected to get oid, not '%s'", line);
 
-		o = parse_object(&oid);
+		o = parse_object(the_repository, &oid);
 		if (!o) {
 			packet_write_fmt(1,
 					 "ERR upload-pack: not our ref %s",
@@ -1274,7 +1275,7 @@ static int process_haves(struct oid_array *haves, struct oid_array *common)
 
 		oid_array_append(common, oid);
 
-		o = parse_object(oid);
+		o = parse_object(the_repository, oid);
 		if (!o)
 			die("oops (%s)", oid_to_hex(oid));
 		if (o->type == OBJ_COMMIT) {
diff --git a/walker.c b/walker.c
index a80811f6d5e..3678e344312 100644
--- a/walker.c
+++ b/walker.c
@@ -1,5 +1,6 @@
 #include "cache.h"
 #include "walker.h"
+#include "repository.h"
 #include "object-store.h"
 #include "commit.h"
 #include "tree.h"
@@ -178,7 +179,7 @@ static int loop(struct walker *walker)
 			}
 		}
 		if (!obj->type)
-			parse_object(&obj->oid);
+			parse_object(the_repository, &obj->oid);
 		if (process_object(walker, obj))
 			return -1;
 	}
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 2%]

* [PATCH 10/35] commit: add repository argument to lookup_commit
  2018-05-30  0:47 [RFC PATCH 00/35] object-store: lookup_commit Stefan Beller
  2018-05-30  0:47 ` [PATCH 01/35] object: add repository argument to parse_object Stefan Beller
  2018-05-30  0:47 ` [PATCH 09/35] commit: add repository argument to lookup_commit_reference Stefan Beller
@ 2018-05-30  0:47 ` Stefan Beller
  2018-05-30  0:47 ` [PATCH 18/35] commit: add repository argument to parse_commit Stefan Beller
  2018-06-13 23:04 ` [PATCH v2 00/31] object-store: lookup_commit Stefan Beller
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-30  0:47 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

Add a repository argument to allow callers of lookup_commit to be more
specific about which repository to handle. This is a small mechanical
change; it doesn't change the implementation to handle repositories
other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/am.c            |  3 ++-
 builtin/commit-tree.c   |  4 +++-
 builtin/diff-tree.c     |  2 +-
 builtin/fast-export.c   |  2 +-
 builtin/fmt-merge-msg.c |  2 +-
 builtin/merge-base.c    |  2 +-
 builtin/verify-commit.c |  4 +++-
 commit-graph.c          | 10 +++++-----
 commit.c                |  7 ++++---
 commit.h                |  3 ++-
 fetch-pack.c            |  5 +++--
 log-tree.c              |  2 +-
 notes-utils.c           |  4 +++-
 object.c                |  2 +-
 sequencer.c             |  4 ++--
 sha1-name.c             |  2 +-
 shallow.c               | 17 ++++++++++-------
 tag.c                   |  2 +-
 tree.c                  |  2 +-
 19 files changed, 46 insertions(+), 33 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index f4b510bcc5f..fb7c21f7103 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1633,7 +1633,8 @@ static void do_commit(const struct am_state *state)
 
 	if (!get_oid_commit("HEAD", &parent)) {
 		old_oid = &parent;
-		commit_list_insert(lookup_commit(&parent), &parents);
+		commit_list_insert(lookup_commit(the_repository, &parent),
+				   &parents);
 	} else {
 		old_oid = NULL;
 		say(state, stderr, _("applying to an empty history"));
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index 9fbd3529fb1..9ec36a82b63 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -6,6 +6,7 @@
 #include "cache.h"
 #include "config.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "tree.h"
 #include "builtin.h"
@@ -60,7 +61,8 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)
 			if (get_oid_commit(argv[i], &oid))
 				die("Not a valid object name %s", argv[i]);
 			assert_oid_type(&oid, OBJ_COMMIT);
-			new_parent(lookup_commit(&oid), &parents);
+			new_parent(lookup_commit(the_repository, &oid),
+						 &parents);
 			continue;
 		}
 
diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index a5718d96ee2..91ba67070e2 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -25,7 +25,7 @@ static int stdin_diff_commit(struct commit *commit, const char *p)
 
 	/* Graft the fake parents locally to the commit */
 	while (isspace(*p++) && !parse_oid_hex(p, &oid, &p)) {
-		struct commit *parent = lookup_commit(&oid);
+		struct commit *parent = lookup_commit(the_repository, &oid);
 		if (!pptr) {
 			/* Free the real parent list */
 			free_commit_list(commit->parents);
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 23ca46e6008..bc837d2593c 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -964,7 +964,7 @@ static void import_marks(char *input_file)
 			/* only commits */
 			continue;
 
-		commit = lookup_commit(&oid);
+		commit = lookup_commit(the_repository, &oid);
 		if (!commit)
 			die("not a commit? can't happen: %s", oid_to_hex(&oid));
 
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index 5e44589b545..36318ef46e7 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -572,7 +572,7 @@ static void find_merge_parents(struct merge_parents *result,
 		commit_list_insert(parent, &parents);
 		add_merge_parent(result, &obj->oid, &parent->object.oid);
 	}
-	head_commit = lookup_commit(head);
+	head_commit = lookup_commit(the_repository, head);
 	if (head_commit)
 		commit_list_insert(head_commit, &parents);
 	reduce_heads_replace(&parents);
diff --git a/builtin/merge-base.c b/builtin/merge-base.c
index bbead6f33e5..08d91b1f0c0 100644
--- a/builtin/merge-base.c
+++ b/builtin/merge-base.c
@@ -124,7 +124,7 @@ static void add_one_commit(struct object_id *oid, struct rev_collect *revs)
 	if (is_null_oid(oid))
 		return;
 
-	commit = lookup_commit(oid);
+	commit = lookup_commit(the_repository, oid);
 	if (!commit ||
 	    (commit->object.flags & TMP_MARK) ||
 	    parse_commit(commit))
diff --git a/builtin/verify-commit.c b/builtin/verify-commit.c
index f6922da16d6..7772c07ed7a 100644
--- a/builtin/verify-commit.c
+++ b/builtin/verify-commit.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "builtin.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "run-command.h"
 #include <signal.h>
@@ -27,7 +28,8 @@ static int run_gpg_verify(const struct object_id *oid, const char *buf, unsigned
 
 	memset(&signature_check, 0, sizeof(signature_check));
 
-	ret = check_commit_signature(lookup_commit(oid), &signature_check);
+	ret = check_commit_signature(lookup_commit(the_repository, oid),
+				     &signature_check);
 	print_signature_buffer(&signature_check, flags);
 
 	signature_check_clear(&signature_check);
diff --git a/commit-graph.c b/commit-graph.c
index 5345cc90ed1..6c08fc0a9ba 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -241,7 +241,7 @@ static struct commit_list **insert_parent_or_die(struct commit_graph *g,
 	struct commit *c;
 	struct object_id oid;
 	hashcpy(oid.hash, g->chunk_oid_lookup + g->hash_len * pos);
-	c = lookup_commit(&oid);
+	c = lookup_commit(the_repository, &oid);
 	if (!c)
 		die("could not find commit %s", oid_to_hex(&oid));
 	c->graph_pos = pos;
@@ -527,7 +527,7 @@ static void close_reachable(struct packed_oid_list *oids)
 	struct commit *commit;
 
 	for (i = 0; i < oids->nr; i++) {
-		commit = lookup_commit(&oids->list[i]);
+		commit = lookup_commit(the_repository, &oids->list[i]);
 		if (commit)
 			commit->object.flags |= UNINTERESTING;
 	}
@@ -538,14 +538,14 @@ static void close_reachable(struct packed_oid_list *oids)
 	 * closure.
 	 */
 	for (i = 0; i < oids->nr; i++) {
-		commit = lookup_commit(&oids->list[i]);
+		commit = lookup_commit(the_repository, &oids->list[i]);
 
 		if (commit && !parse_commit(commit))
 			add_missing_parents(oids, commit);
 	}
 
 	for (i = 0; i < oids->nr; i++) {
-		commit = lookup_commit(&oids->list[i]);
+		commit = lookup_commit(the_repository, &oids->list[i]);
 
 		if (commit)
 			commit->object.flags &= ~UNINTERESTING;
@@ -658,7 +658,7 @@ void write_commit_graph(const char *obj_dir,
 		if (i > 0 && !oidcmp(&oids.list[i-1], &oids.list[i]))
 			continue;
 
-		commits.list[commits.nr] = lookup_commit(&oids.list[i]);
+		commits.list[commits.nr] = lookup_commit(the_repository, &oids.list[i]);
 		parse_commit(commits.list[commits.nr]);
 
 		for (parent = commits.list[commits.nr]->parents;
diff --git a/commit.c b/commit.c
index e031a6c3175..5e3f18801a1 100644
--- a/commit.c
+++ b/commit.c
@@ -51,7 +51,7 @@ struct commit *lookup_commit_or_die(const struct object_id *oid, const char *ref
 	return c;
 }
 
-struct commit *lookup_commit(const struct object_id *oid)
+struct commit *lookup_commit_the_repository(const struct object_id *oid)
 {
 	struct object *obj = lookup_object(the_repository, oid->hash);
 	if (!obj)
@@ -373,7 +373,7 @@ int parse_commit_buffer(struct commit *item, const void *buffer, unsigned long s
 		 */
 		if (graft && (graft->nr_parent < 0 || grafts_replace_parents))
 			continue;
-		new_parent = lookup_commit(&parent);
+		new_parent = lookup_commit(the_repository, &parent);
 		if (new_parent)
 			pptr = &commit_list_insert(new_parent, pptr)->next;
 	}
@@ -381,7 +381,8 @@ int parse_commit_buffer(struct commit *item, const void *buffer, unsigned long s
 		int i;
 		struct commit *new_parent;
 		for (i = 0; i < graft->nr_parent; i++) {
-			new_parent = lookup_commit(&graft->parent[i]);
+			new_parent = lookup_commit(the_repository,
+						   &graft->parent[i]);
 			if (!new_parent)
 				continue;
 			pptr = &commit_list_insert(new_parent, pptr)->next;
diff --git a/commit.h b/commit.h
index a5f84466efd..431a7d97a24 100644
--- a/commit.h
+++ b/commit.h
@@ -53,7 +53,8 @@ enum decoration_type {
 void add_name_decoration(enum decoration_type type, const char *name, struct object *obj);
 const struct name_decoration *get_name_decoration(const struct object *obj);
 
-struct commit *lookup_commit(const struct object_id *oid);
+#define lookup_commit(r, o) lookup_commit_##r(o)
+struct commit *lookup_commit_the_repository(const struct object_id *oid);
 #define lookup_commit_reference(r, o) \
 		lookup_commit_reference_##r(o)
 struct commit *lookup_commit_reference_the_repository(const struct object_id *oid);
diff --git a/fetch-pack.c b/fetch-pack.c
index 4523e25ff4e..74ac2977e85 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -498,7 +498,8 @@ static int find_common(struct fetch_pack_args *args,
 				case ACK_ready:
 				case ACK_continue: {
 					struct commit *commit =
-						lookup_commit(result_oid);
+						lookup_commit(the_repository,
+							      result_oid);
 					if (!commit)
 						die(_("invalid commit %s"), oid_to_hex(result_oid));
 					if (args->stateless_rpc
@@ -1256,7 +1257,7 @@ static int process_acks(struct packet_reader *reader, struct oidset *common)
 			if (!get_oid_hex(arg, &oid)) {
 				struct commit *commit;
 				oidset_insert(common, &oid);
-				commit = lookup_commit(&oid);
+				commit = lookup_commit(the_repository, &oid);
 				mark_common(commit, 0, 1);
 			}
 			continue;
diff --git a/log-tree.c b/log-tree.c
index e8d2857bf06..a47283eca64 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -147,7 +147,7 @@ static int add_ref_decoration(const char *refname, const struct object_id *oid,
 
 static int add_graft_decoration(const struct commit_graft *graft, void *cb_data)
 {
-	struct commit *commit = lookup_commit(&graft->oid);
+	struct commit *commit = lookup_commit(the_repository, &graft->oid);
 	if (!commit)
 		return 0;
 	add_name_decoration(DECORATION_GRAFTED, "grafted", &commit->object);
diff --git a/notes-utils.c b/notes-utils.c
index 02407fe2a73..14ea03178e9 100644
--- a/notes-utils.c
+++ b/notes-utils.c
@@ -3,6 +3,7 @@
 #include "commit.h"
 #include "refs.h"
 #include "notes-utils.h"
+#include "repository.h"
 
 void create_notes_commit(struct notes_tree *t, struct commit_list *parents,
 			 const char *msg, size_t msg_len,
@@ -19,7 +20,8 @@ void create_notes_commit(struct notes_tree *t, struct commit_list *parents,
 		/* Deduce parent commit from t->ref */
 		struct object_id parent_oid;
 		if (!read_ref(t->ref, &parent_oid)) {
-			struct commit *parent = lookup_commit(&parent_oid);
+			struct commit *parent = lookup_commit(the_repository,
+							      &parent_oid);
 			if (parse_commit(parent))
 				die("Failed to find/parse commit %s", t->ref);
 			commit_list_insert(parent, &parents);
diff --git a/object.c b/object.c
index 3fa7722d673..f4207d09e9f 100644
--- a/object.c
+++ b/object.c
@@ -212,7 +212,7 @@ struct object *parse_object_buffer_the_repository(const struct object_id *oid, e
 			}
 		}
 	} else if (type == OBJ_COMMIT) {
-		struct commit *commit = lookup_commit(oid);
+		struct commit *commit = lookup_commit(the_repository, oid);
 		if (commit) {
 			if (parse_commit_buffer(commit, buffer, size))
 				return NULL;
diff --git a/sequencer.c b/sequencer.c
index 8931c461f20..da6c9fc5511 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -543,7 +543,7 @@ static int is_index_unchanged(void)
 	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, &head_oid, NULL))
 		return error(_("could not resolve HEAD commit"));
 
-	head_commit = lookup_commit(&head_oid);
+	head_commit = lookup_commit(the_repository, &head_oid);
 
 	/*
 	 * If head_commit is NULL, check_commit, called from
@@ -972,7 +972,7 @@ void print_commit_summary(const char *prefix, const struct object_id *oid,
 	struct strbuf author_ident = STRBUF_INIT;
 	struct strbuf committer_ident = STRBUF_INIT;
 
-	commit = lookup_commit(oid);
+	commit = lookup_commit(the_repository, oid);
 	if (!commit)
 		die(_("couldn't look up newly created commit"));
 	if (parse_commit(commit))
diff --git a/sha1-name.c b/sha1-name.c
index d153d8c692b..5eef8ddd6d6 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -352,7 +352,7 @@ static int show_ambiguous_object(const struct object_id *oid, void *data)
 
 	type = oid_object_info(the_repository, oid, NULL);
 	if (type == OBJ_COMMIT) {
-		struct commit *commit = lookup_commit(oid);
+		struct commit *commit = lookup_commit(the_repository, oid);
 		if (commit) {
 			struct pretty_print_context pp = {0};
 			pp.date_mode.type = DATE_SHORT;
diff --git a/shallow.c b/shallow.c
index 9bb07a56dca..60fe1fe1e58 100644
--- a/shallow.c
+++ b/shallow.c
@@ -31,7 +31,7 @@ int register_shallow(struct repository *r, const struct object_id *oid)
 {
 	struct commit_graft *graft =
 		xmalloc(sizeof(struct commit_graft));
-	struct commit *commit = lookup_commit(oid);
+	struct commit *commit = lookup_commit(the_repository, oid);
 
 	oidcpy(&graft->oid, oid);
 	graft->nr_parent = -1;
@@ -256,7 +256,7 @@ static int write_one_shallow(const struct commit_graft *graft, void *cb_data)
 	if (graft->nr_parent != -1)
 		return 0;
 	if (data->flags & SEEN_ONLY) {
-		struct commit *c = lookup_commit(&graft->oid);
+		struct commit *c = lookup_commit(the_repository, &graft->oid);
 		if (!c || !(c->object.flags & SEEN)) {
 			if (data->flags & VERBOSE)
 				printf("Removing %s from .git/shallow\n",
@@ -621,7 +621,8 @@ void assign_shallow_commits_to_refs(struct shallow_info *info,
 
 	/* Mark potential bottoms so we won't go out of bound */
 	for (i = 0; i < nr_shallow; i++) {
-		struct commit *c = lookup_commit(&oid[shallow[i]]);
+		struct commit *c = lookup_commit(the_repository,
+						 &oid[shallow[i]]);
 		c->object.flags |= BOTTOM;
 	}
 
@@ -632,7 +633,8 @@ void assign_shallow_commits_to_refs(struct shallow_info *info,
 		int bitmap_size = DIV_ROUND_UP(pi.nr_bits, 32) * sizeof(uint32_t);
 		memset(used, 0, sizeof(*used) * info->shallow->nr);
 		for (i = 0; i < nr_shallow; i++) {
-			const struct commit *c = lookup_commit(&oid[shallow[i]]);
+			const struct commit *c = lookup_commit(the_repository,
+							       &oid[shallow[i]]);
 			uint32_t **map = ref_bitmap_at(&pi.ref_bitmap, c);
 			if (*map)
 				used[shallow[i]] = xmemdupz(*map, bitmap_size);
@@ -702,7 +704,7 @@ static void post_assign_shallow(struct shallow_info *info,
 	for (i = dst = 0; i < info->nr_theirs; i++) {
 		if (i != dst)
 			info->theirs[dst] = info->theirs[i];
-		c = lookup_commit(&oid[info->theirs[i]]);
+		c = lookup_commit(the_repository, &oid[info->theirs[i]]);
 		bitmap = ref_bitmap_at(ref_bitmap, c);
 		if (!*bitmap)
 			continue;
@@ -723,7 +725,7 @@ static void post_assign_shallow(struct shallow_info *info,
 	for (i = dst = 0; i < info->nr_ours; i++) {
 		if (i != dst)
 			info->ours[dst] = info->ours[i];
-		c = lookup_commit(&oid[info->ours[i]]);
+		c = lookup_commit(the_repository, &oid[info->ours[i]]);
 		bitmap = ref_bitmap_at(ref_bitmap, c);
 		if (!*bitmap)
 			continue;
@@ -745,7 +747,8 @@ static void post_assign_shallow(struct shallow_info *info,
 int delayed_reachability_test(struct shallow_info *si, int c)
 {
 	if (si->need_reachability_test[c]) {
-		struct commit *commit = lookup_commit(&si->shallow->oid[c]);
+		struct commit *commit = lookup_commit(the_repository,
+						      &si->shallow->oid[c]);
 
 		if (!si->commits) {
 			struct commit_array ca;
diff --git a/tag.c b/tag.c
index 310102150ab..5dcdf7bf6f9 100644
--- a/tag.c
+++ b/tag.c
@@ -158,7 +158,7 @@ int parse_tag_buffer(struct tag *item, const void *data, unsigned long size)
 	} else if (!strcmp(type, tree_type)) {
 		item->tagged = (struct object *)lookup_tree(the_repository, &oid);
 	} else if (!strcmp(type, commit_type)) {
-		item->tagged = (struct object *)lookup_commit(&oid);
+		item->tagged = (struct object *)lookup_commit(the_repository, &oid);
 	} else if (!strcmp(type, tag_type)) {
 		item->tagged = (struct object *)lookup_tag(&oid);
 	} else {
diff --git a/tree.c b/tree.c
index fbc8d4bc653..33063b8dde0 100644
--- a/tree.c
+++ b/tree.c
@@ -101,7 +101,7 @@ static int read_tree_1(struct tree *tree, struct strbuf *base,
 		else if (S_ISGITLINK(entry.mode)) {
 			struct commit *commit;
 
-			commit = lookup_commit(entry.oid);
+			commit = lookup_commit(the_repository, entry.oid);
 			if (!commit)
 				die("Commit %s in submodule path %s%s not found",
 				    oid_to_hex(entry.oid),
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 3%]

* [PATCH 09/35] commit: add repository argument to lookup_commit_reference
  2018-05-30  0:47 [RFC PATCH 00/35] object-store: lookup_commit Stefan Beller
  2018-05-30  0:47 ` [PATCH 01/35] object: add repository argument to parse_object Stefan Beller
@ 2018-05-30  0:47 ` Stefan Beller
  2018-05-30  0:47 ` [PATCH 10/35] commit: add repository argument to lookup_commit Stefan Beller
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-30  0:47 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

Add a repository argument to allow callers of lookup_commit_reference
to be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 bisect.c              |  2 +-
 blame.c               |  2 +-
 branch.c              |  2 +-
 builtin/branch.c      |  7 ++++---
 builtin/clone.c       |  3 ++-
 builtin/describe.c    |  2 +-
 builtin/diff-tree.c   |  2 +-
 builtin/log.c         |  7 ++++---
 builtin/merge-base.c  |  5 +++--
 builtin/notes.c       |  3 ++-
 builtin/pull.c        | 15 ++++++++++-----
 builtin/reset.c       |  4 ++--
 builtin/rev-parse.c   |  6 +++---
 builtin/show-branch.c |  2 +-
 builtin/tag.c         |  2 +-
 bundle.c              |  3 ++-
 commit.c              |  6 +++---
 commit.h              |  4 +++-
 notes-merge.c         |  5 +++--
 parse-options-cb.c    |  2 +-
 remote.c              |  4 ++--
 revision.c            |  4 ++--
 sequencer.c           |  6 +++---
 sha1-name.c           |  4 ++--
 submodule.c           | 10 +++++-----
 25 files changed, 63 insertions(+), 49 deletions(-)

diff --git a/bisect.c b/bisect.c
index 6de1abd407b..e1275ba79e8 100644
--- a/bisect.c
+++ b/bisect.c
@@ -724,7 +724,7 @@ static int bisect_checkout(const struct object_id *bisect_rev, int no_checkout)
 
 static struct commit *get_commit_reference(const struct object_id *oid)
 {
-	struct commit *r = lookup_commit_reference(oid);
+	struct commit *r = lookup_commit_reference(the_repository, oid);
 	if (!r)
 		die(_("Not a valid commit name %s"), oid_to_hex(oid));
 	return r;
diff --git a/blame.c b/blame.c
index 726a7a76f20..86d0dd73338 100644
--- a/blame.c
+++ b/blame.c
@@ -119,7 +119,7 @@ static struct commit_list **append_parent(struct commit_list **tail, const struc
 {
 	struct commit *parent;
 
-	parent = lookup_commit_reference(oid);
+	parent = lookup_commit_reference(the_repository, oid);
 	if (!parent)
 		die("no such commit %s", oid_to_hex(oid));
 	return &commit_list_insert(parent, tail)->next;
diff --git a/branch.c b/branch.c
index 9b2742de32a..08d4efc1be6 100644
--- a/branch.c
+++ b/branch.c
@@ -301,7 +301,7 @@ void create_branch(const char *name, const char *start_name,
 		break;
 	}
 
-	if ((commit = lookup_commit_reference(&oid)) == NULL)
+	if ((commit = lookup_commit_reference(the_repository, &oid)) == NULL)
 		die(_("Not a valid branch point: '%s'."), start_name);
 	oidcpy(&oid, &commit->object.oid);
 
diff --git a/builtin/branch.c b/builtin/branch.c
index efc9ac1922c..fbbe311e270 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -126,7 +126,8 @@ static int branch_merged(int kind, const char *name,
 		    (reference_name = reference_name_to_free =
 		     resolve_refdup(upstream, RESOLVE_REF_READING,
 				    &oid, NULL)) != NULL)
-			reference_rev = lookup_commit_reference(&oid);
+			reference_rev = lookup_commit_reference(the_repository,
+								&oid);
 	}
 	if (!reference_rev)
 		reference_rev = head_rev;
@@ -159,7 +160,7 @@ static int check_branch_commit(const char *branchname, const char *refname,
 			       const struct object_id *oid, struct commit *head_rev,
 			       int kinds, int force)
 {
-	struct commit *rev = lookup_commit_reference(oid);
+	struct commit *rev = lookup_commit_reference(the_repository, oid);
 	if (!rev) {
 		error(_("Couldn't look up commit object for '%s'"), refname);
 		return -1;
@@ -213,7 +214,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 	}
 
 	if (!force) {
-		head_rev = lookup_commit_reference(&head_oid);
+		head_rev = lookup_commit_reference(the_repository, &head_oid);
 		if (!head_rev)
 			die(_("Couldn't look up commit object for HEAD"));
 	}
diff --git a/builtin/clone.c b/builtin/clone.c
index 2f10c468300..5b10ddfc832 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -695,7 +695,8 @@ static void update_head(const struct ref *our, const struct ref *remote,
 			install_branch_config(0, head, option_origin, our->name);
 		}
 	} else if (our) {
-		struct commit *c = lookup_commit_reference(&our->old_oid);
+		struct commit *c = lookup_commit_reference(the_repository,
+							   &our->old_oid);
 		/* --branch specifies a non-branch (i.e. tags), detach HEAD */
 		update_ref(msg, "HEAD", &c->object.oid, NULL, REF_NO_DEREF,
 			   UPDATE_REFS_DIE_ON_ERR);
diff --git a/builtin/describe.c b/builtin/describe.c
index f3dfd0228be..080b5ce082c 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -303,7 +303,7 @@ static void describe_commit(struct object_id *oid, struct strbuf *dst)
 	unsigned long seen_commits = 0;
 	unsigned int unannotated_cnt = 0;
 
-	cmit = lookup_commit_reference(oid);
+	cmit = lookup_commit_reference(the_repository, oid);
 
 	n = find_commit_name(&cmit->object.oid);
 	if (n && (tags || all || n->prio == 2)) {
diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index 29901515a13..a5718d96ee2 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -11,7 +11,7 @@ static struct rev_info log_tree_opt;
 
 static int diff_tree_commit_oid(const struct object_id *oid)
 {
-	struct commit *commit = lookup_commit_reference(oid);
+	struct commit *commit = lookup_commit_reference(the_repository, oid);
 	if (!commit)
 		return -1;
 	return log_tree_commit(&log_tree_opt, commit);
diff --git a/builtin/log.c b/builtin/log.c
index 063057e45de..c962d22707b 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -907,8 +907,8 @@ static void get_patch_ids(struct rev_info *rev, struct patch_ids *ids)
 	o2 = rev->pending.objects[1].item;
 	flags1 = o1->flags;
 	flags2 = o2->flags;
-	c1 = lookup_commit_reference(&o1->oid);
-	c2 = lookup_commit_reference(&o2->oid);
+	c1 = lookup_commit_reference(the_repository, &o1->oid);
+	c2 = lookup_commit_reference(the_repository, &o2->oid);
 
 	if ((flags1 & UNINTERESTING) == (flags2 & UNINTERESTING))
 		die(_("Not a range."));
@@ -1864,7 +1864,8 @@ static int add_pending_commit(const char *arg, struct rev_info *revs, int flags)
 {
 	struct object_id oid;
 	if (get_oid(arg, &oid) == 0) {
-		struct commit *commit = lookup_commit_reference(&oid);
+		struct commit *commit = lookup_commit_reference(the_repository,
+								&oid);
 		if (commit) {
 			commit->object.flags |= flags;
 			add_pending_object(revs, &commit->object, arg);
diff --git a/builtin/merge-base.c b/builtin/merge-base.c
index 3b7600150b6..bbead6f33e5 100644
--- a/builtin/merge-base.c
+++ b/builtin/merge-base.c
@@ -6,6 +6,7 @@
 #include "diff.h"
 #include "revision.h"
 #include "parse-options.h"
+#include "repository.h"
 
 static int show_merge_base(struct commit **rev, int rev_nr, int show_all)
 {
@@ -42,7 +43,7 @@ static struct commit *get_commit_reference(const char *arg)
 
 	if (get_oid(arg, &revkey))
 		die("Not a valid object name %s", arg);
-	r = lookup_commit_reference(&revkey);
+	r = lookup_commit_reference(the_repository, &revkey);
 	if (!r)
 		die("Not a valid commit name %s", arg);
 
@@ -171,7 +172,7 @@ static int handle_fork_point(int argc, const char **argv)
 	if (get_oid(commitname, &oid))
 		die("Not a valid object name: '%s'", commitname);
 
-	derived = lookup_commit_reference(&oid);
+	derived = lookup_commit_reference(the_repository, &oid);
 	memset(&revs, 0, sizeof(revs));
 	revs.initial = 1;
 	for_each_reflog_ent(refname, collect_one_reflog_ent, &revs);
diff --git a/builtin/notes.c b/builtin/notes.c
index 6011cd2a074..50963642075 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -12,6 +12,7 @@
 #include "builtin.h"
 #include "notes.h"
 #include "object-store.h"
+#include "repository.h"
 #include "blob.h"
 #include "pretty.h"
 #include "refs.h"
@@ -711,7 +712,7 @@ static int merge_commit(struct notes_merge_options *o)
 
 	if (get_oid("NOTES_MERGE_PARTIAL", &oid))
 		die(_("failed to read ref NOTES_MERGE_PARTIAL"));
-	else if (!(partial = lookup_commit_reference(&oid)))
+	else if (!(partial = lookup_commit_reference(the_repository, &oid)))
 		die(_("could not find commit from NOTES_MERGE_PARTIAL."));
 	else if (parse_commit(partial))
 		die(_("could not parse commit from NOTES_MERGE_PARTIAL."));
diff --git a/builtin/pull.c b/builtin/pull.c
index 324c4ffa1a7..ec3f90545dd 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -760,10 +760,13 @@ static int get_octopus_merge_base(struct object_id *merge_base,
 {
 	struct commit_list *revs = NULL, *result;
 
-	commit_list_insert(lookup_commit_reference(curr_head), &revs);
-	commit_list_insert(lookup_commit_reference(merge_head), &revs);
+	commit_list_insert(lookup_commit_reference(the_repository, curr_head),
+			   &revs);
+	commit_list_insert(lookup_commit_reference(the_repository, merge_head),
+			   &revs);
 	if (!is_null_oid(fork_point))
-		commit_list_insert(lookup_commit_reference(fork_point), &revs);
+		commit_list_insert(lookup_commit_reference(the_repository, fork_point),
+				   &revs);
 
 	result = get_octopus_merge_bases(revs);
 	free_commit_list(revs);
@@ -937,9 +940,11 @@ int cmd_pull(int argc, const char **argv, const char *prefix)
 			struct commit_list *list = NULL;
 			struct commit *merge_head, *head;
 
-			head = lookup_commit_reference(&orig_head);
+			head = lookup_commit_reference(the_repository,
+						       &orig_head);
 			commit_list_insert(head, &list);
-			merge_head = lookup_commit_reference(&merge_heads.oid[0]);
+			merge_head = lookup_commit_reference(the_repository,
+							     &merge_heads.oid[0]);
 			if (is_descendant_of(merge_head, list)) {
 				/* we can fast-forward this without invoking rebase */
 				opt_ff = "--ff-only";
diff --git a/builtin/reset.c b/builtin/reset.c
index a9136a163c0..aeab0c7aea5 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -319,7 +319,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 		struct commit *commit;
 		if (get_oid_committish(rev, &oid))
 			die(_("Failed to resolve '%s' as a valid revision."), rev);
-		commit = lookup_commit_reference(&oid);
+		commit = lookup_commit_reference(the_repository, &oid);
 		if (!commit)
 			die(_("Could not parse object '%s'."), rev);
 		oidcpy(&oid, &commit->object.oid);
@@ -396,7 +396,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 		update_ref_status = reset_refs(rev, &oid);
 
 		if (reset_type == HARD && !update_ref_status && !quiet)
-			print_new_head_line(lookup_commit_reference(&oid));
+			print_new_head_line(lookup_commit_reference(the_repository, &oid));
 	}
 	if (!pathspec.nr)
 		remove_branch_state();
diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index a8a9b506ff6..cc6a6c9a019 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -280,8 +280,8 @@ static int try_difference(const char *arg)
 		if (symmetric) {
 			struct commit_list *exclude;
 			struct commit *a, *b;
-			a = lookup_commit_reference(&start_oid);
-			b = lookup_commit_reference(&end_oid);
+			a = lookup_commit_reference(the_repository, &start_oid);
+			b = lookup_commit_reference(the_repository, &end_oid);
 			exclude = get_merge_bases(a, b);
 			while (exclude) {
 				struct commit *commit = pop_commit(&exclude);
@@ -333,7 +333,7 @@ static int try_parent_shorthands(const char *arg)
 		return 0;
 	}
 
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (exclude_parent &&
 	    exclude_parent > commit_list_count(commit->parents)) {
 		*dotdot = '^';
diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index 2456b47d344..4b9d3c0059b 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -831,7 +831,7 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 			       MAX_REVS), MAX_REVS);
 		if (get_oid(ref_name[num_rev], &revkey))
 			die(_("'%s' is not a valid ref."), ref_name[num_rev]);
-		commit = lookup_commit_reference(&revkey);
+		commit = lookup_commit_reference(the_repository, &revkey);
 		if (!commit)
 			die(_("cannot find commit %s (%s)"),
 			    ref_name[num_rev], oid_to_hex(&revkey));
diff --git a/builtin/tag.c b/builtin/tag.c
index 9919b03b2d6..9a19ffb49f6 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -313,7 +313,7 @@ static void create_reflog_msg(const struct object_id *oid, struct strbuf *sb)
 		}
 		free(buf);
 
-		if ((c = lookup_commit_reference(oid)) != NULL)
+		if ((c = lookup_commit_reference(the_repository, oid)) != NULL)
 			strbuf_addf(sb, ", %s", show_date(c->date, 0, DATE_MODE(SHORT)));
 		break;
 	case OBJ_TREE:
diff --git a/bundle.c b/bundle.c
index 14a209ff149..904b59e72a5 100644
--- a/bundle.c
+++ b/bundle.c
@@ -375,7 +375,8 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
 			 * in terms of a tag (e.g. v2.0 from the range
 			 * "v1.0..v2.0")?
 			 */
-			struct commit *one = lookup_commit_reference(&oid);
+			struct commit *one = lookup_commit_reference(the_repository,
+								     &oid);
 			struct object *obj;
 
 			if (e->item == &(one->object)) {
diff --git a/commit.c b/commit.c
index b039444aa62..e031a6c3175 100644
--- a/commit.c
+++ b/commit.c
@@ -34,14 +34,14 @@ struct commit *lookup_commit_reference_gently_the_repository(
 	return object_as_type(the_repository, obj, OBJ_COMMIT, quiet);
 }
 
-struct commit *lookup_commit_reference(const struct object_id *oid)
+struct commit *lookup_commit_reference_the_repository(const struct object_id *oid)
 {
 	return lookup_commit_reference_gently(the_repository, oid, 0);
 }
 
 struct commit *lookup_commit_or_die(const struct object_id *oid, const char *ref_name)
 {
-	struct commit *c = lookup_commit_reference(oid);
+	struct commit *c = lookup_commit_reference(the_repository, oid);
 	if (!c)
 		die(_("could not parse %s"), ref_name);
 	if (oidcmp(oid, &c->object.oid)) {
@@ -67,7 +67,7 @@ struct commit *lookup_commit_reference_by_name(const char *name)
 
 	if (get_oid_committish(name, &oid))
 		return NULL;
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (parse_commit(commit))
 		return NULL;
 	return commit;
diff --git a/commit.h b/commit.h
index 73ccdd8005b..a5f84466efd 100644
--- a/commit.h
+++ b/commit.h
@@ -54,7 +54,9 @@ void add_name_decoration(enum decoration_type type, const char *name, struct obj
 const struct name_decoration *get_name_decoration(const struct object *obj);
 
 struct commit *lookup_commit(const struct object_id *oid);
-struct commit *lookup_commit_reference(const struct object_id *oid);
+#define lookup_commit_reference(r, o) \
+		lookup_commit_reference_##r(o)
+struct commit *lookup_commit_reference_the_repository(const struct object_id *oid);
 #define lookup_commit_reference_gently(r, o, q) \
 		lookup_commit_reference_gently_##r(o, q)
 struct commit *lookup_commit_reference_gently_the_repository(
diff --git a/notes-merge.c b/notes-merge.c
index fd96c5ab3d6..2572e435432 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -2,6 +2,7 @@
 #include "commit.h"
 #include "refs.h"
 #include "object-store.h"
+#include "repository.h"
 #include "diff.h"
 #include "diffcore.h"
 #include "xdiff-interface.h"
@@ -553,7 +554,7 @@ int notes_merge(struct notes_merge_options *o,
 	else if (!check_refname_format(o->local_ref, 0) &&
 		is_null_oid(&local_oid))
 		local = NULL; /* local_oid == null_oid indicates unborn ref */
-	else if (!(local = lookup_commit_reference(&local_oid)))
+	else if (!(local = lookup_commit_reference(the_repository, &local_oid)))
 		die("Could not parse local commit %s (%s)",
 		    oid_to_hex(&local_oid), o->local_ref);
 	trace_printf("\tlocal commit: %.7s\n", oid_to_hex(&local_oid));
@@ -571,7 +572,7 @@ int notes_merge(struct notes_merge_options *o,
 			die("Failed to resolve remote notes ref '%s'",
 			    o->remote_ref);
 		}
-	} else if (!(remote = lookup_commit_reference(&remote_oid))) {
+	} else if (!(remote = lookup_commit_reference(the_repository, &remote_oid))) {
 		die("Could not parse remote commit %s (%s)",
 		    oid_to_hex(&remote_oid), o->remote_ref);
 	}
diff --git a/parse-options-cb.c b/parse-options-cb.c
index 0f9f311a7a9..e8236534ac8 100644
--- a/parse-options-cb.c
+++ b/parse-options-cb.c
@@ -91,7 +91,7 @@ int parse_opt_commits(const struct option *opt, const char *arg, int unset)
 		return -1;
 	if (get_oid(arg, &oid))
 		return error("malformed object name %s", arg);
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (!commit)
 		return error("no such commit %s", arg);
 	commit_list_insert(commit, opt->value);
diff --git a/remote.c b/remote.c
index 956fd0fd923..2a6ff7d60c1 100644
--- a/remote.c
+++ b/remote.c
@@ -2058,13 +2058,13 @@ int stat_tracking_info(struct branch *branch, int *num_ours, int *num_theirs,
 	/* Cannot stat if what we used to build on no longer exists */
 	if (read_ref(base, &oid))
 		return -1;
-	theirs = lookup_commit_reference(&oid);
+	theirs = lookup_commit_reference(the_repository, &oid);
 	if (!theirs)
 		return -1;
 
 	if (read_ref(branch->refname, &oid))
 		return -1;
-	ours = lookup_commit_reference(&oid);
+	ours = lookup_commit_reference(the_repository, &oid);
 	if (!ours)
 		return -1;
 
diff --git a/revision.c b/revision.c
index 79f228b1586..25df00cc16c 100644
--- a/revision.c
+++ b/revision.c
@@ -1581,8 +1581,8 @@ static int handle_dotdot_1(const char *arg, char *dotdot,
 		struct commit *a, *b;
 		struct commit_list *exclude;
 
-		a = lookup_commit_reference(&a_obj->oid);
-		b = lookup_commit_reference(&b_obj->oid);
+		a = lookup_commit_reference(the_repository, &a_obj->oid);
+		b = lookup_commit_reference(the_repository, &b_obj->oid);
 		if (!a || !b)
 			return dotdot_missing(arg, dotdot, revs, symmetric);
 
diff --git a/sequencer.c b/sequencer.c
index 4e89e2b1123..8931c461f20 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1047,7 +1047,7 @@ static int parse_head(struct commit **head)
 	if (get_oid("HEAD", &oid)) {
 		current_head = NULL;
 	} else {
-		current_head = lookup_commit_reference(&oid);
+		current_head = lookup_commit_reference(the_repository, &oid);
 		if (!current_head)
 			return error(_("could not parse HEAD"));
 		if (oidcmp(&oid, &current_head->object.oid)) {
@@ -1370,7 +1370,7 @@ static int update_squash_messages(enum todo_command command,
 
 		if (get_oid("HEAD", &head))
 			return error(_("need a HEAD to fixup"));
-		if (!(head_commit = lookup_commit_reference(&head)))
+		if (!(head_commit = lookup_commit_reference(the_repository, &head)))
 			return error(_("could not read HEAD"));
 		if (!(head_message = get_commit_buffer(head_commit, NULL)))
 			return error(_("could not read HEAD's commit message"));
@@ -1824,7 +1824,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (status < 0)
 		return -1;
 
-	item->commit = lookup_commit_reference(&commit_oid);
+	item->commit = lookup_commit_reference(the_repository, &commit_oid);
 	return !item->commit;
 }
 
diff --git a/sha1-name.c b/sha1-name.c
index 412a5257d6f..d153d8c692b 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -810,7 +810,7 @@ static int get_parent(const char *name, int len,
 
 	if (ret)
 		return ret;
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (parse_commit(commit))
 		return -1;
 	if (!idx) {
@@ -838,7 +838,7 @@ static int get_nth_ancestor(const char *name, int len,
 	ret = get_oid_1(name, len, &oid, GET_OID_COMMITTISH);
 	if (ret)
 		return ret;
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (!commit)
 		return -1;
 
diff --git a/submodule.c b/submodule.c
index 8fd8e5d178d..e45150a3f51 100644
--- a/submodule.c
+++ b/submodule.c
@@ -516,8 +516,8 @@ static void show_submodule_header(struct diff_options *o, const char *path,
 	 * Attempt to lookup the commit references, and determine if this is
 	 * a fast forward or fast backwards update.
 	 */
-	*left = lookup_commit_reference(one);
-	*right = lookup_commit_reference(two);
+	*left = lookup_commit_reference(the_repository, one);
+	*right = lookup_commit_reference(the_repository, two);
 
 	/*
 	 * Warn about missing commits in the submodule project, but only if
@@ -1800,9 +1800,9 @@ int merge_submodule(struct object_id *result, const char *path,
 		return 0;
 	}
 
-	if (!(commit_base = lookup_commit_reference(base)) ||
-	    !(commit_a = lookup_commit_reference(a)) ||
-	    !(commit_b = lookup_commit_reference(b))) {
+	if (!(commit_base = lookup_commit_reference(the_repository, base)) ||
+	    !(commit_a = lookup_commit_reference(the_repository, a)) ||
+	    !(commit_b = lookup_commit_reference(the_repository, b))) {
 		MERGE_WARNING(path, "commits not present");
 		return 0;
 	}
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 4%]

* [PATCH 18/35] commit: add repository argument to parse_commit
  2018-05-30  0:47 [RFC PATCH 00/35] object-store: lookup_commit Stefan Beller
                   ` (2 preceding siblings ...)
  2018-05-30  0:47 ` [PATCH 10/35] commit: add repository argument to lookup_commit Stefan Beller
@ 2018-05-30  0:47 ` Stefan Beller
  2018-06-13 23:04 ` [PATCH v2 00/31] object-store: lookup_commit Stefan Beller
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-30  0:47 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

Add a repository argument to allow callers of parse_commit to
be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 blame.c               |  4 ++--
 builtin/checkout.c    |  4 ++--
 builtin/commit.c      |  2 +-
 builtin/describe.c    |  4 ++--
 builtin/merge-base.c  |  2 +-
 builtin/name-rev.c    |  2 +-
 builtin/notes.c       |  2 +-
 builtin/reflog.c      |  2 +-
 builtin/show-branch.c |  4 ++--
 commit-graph.c        |  6 +++---
 commit.c              | 18 +++++++++---------
 commit.h              |  3 ++-
 fetch-pack.c          |  6 +++---
 fsck.c                |  2 +-
 merge-recursive.c     |  4 ++--
 notes-utils.c         |  2 +-
 remote.c              |  2 +-
 revision.c            |  7 +++----
 sequencer.c           | 16 ++++++++--------
 sha1-name.c           |  4 ++--
 shallow.c             |  4 ++--
 tree.c                |  2 +-
 walker.c              |  2 +-
 23 files changed, 52 insertions(+), 52 deletions(-)

diff --git a/blame.c b/blame.c
index aad53c0f904..c82922506cc 100644
--- a/blame.c
+++ b/blame.c
@@ -1451,7 +1451,7 @@ static void pass_blame(struct blame_scoreboard *sb, struct blame_origin *origin,
 
 			if (sg_origin[i])
 				continue;
-			if (parse_commit(p))
+			if (parse_commit(the_repository, p))
 				continue;
 			porigin = find(p, origin);
 			if (!porigin)
@@ -1590,7 +1590,7 @@ void assign_blame(struct blame_scoreboard *sb, int opt)
 		 * so hold onto it in the meantime.
 		 */
 		blame_origin_incref(suspect);
-		parse_commit(commit);
+		parse_commit(the_repository, commit);
 		if (sb->reverse ||
 		    (!(commit->object.flags & UNINTERESTING) &&
 		     !(revs->max_age != -1 && commit->date < revs->max_age)))
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 31a71f75625..cf93a6e913d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -402,7 +402,7 @@ static void describe_detached_head(const char *msg, struct commit *commit)
 {
 	struct strbuf sb = STRBUF_INIT;
 
-	if (!parse_commit(commit))
+	if (!parse_commit(the_repository, commit))
 		pp_commit_easy(CMIT_FMT_ONELINE, commit, &sb);
 	if (print_sha1_ellipsis()) {
 		fprintf(stderr, "%s %s... %s\n", msg,
@@ -723,7 +723,7 @@ static void describe_one_orphan(struct strbuf *sb, struct commit *commit)
 	strbuf_addstr(sb, "  ");
 	strbuf_add_unique_abbrev(sb, &commit->object.oid, DEFAULT_ABBREV);
 	strbuf_addch(sb, ' ');
-	if (!parse_commit(commit))
+	if (!parse_commit(the_repository, commit))
 		pp_commit_easy(CMIT_FMT_ONELINE, commit, sb);
 	strbuf_addch(sb, '\n');
 }
diff --git a/builtin/commit.c b/builtin/commit.c
index b6d130b8ff3..4957099080e 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1482,7 +1482,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 		current_head = NULL;
 	else {
 		current_head = lookup_commit_or_die(&oid, "HEAD");
-		if (parse_commit(current_head))
+		if (parse_commit(the_repository, current_head))
 			die(_("could not parse HEAD commit"));
 	}
 	verbose = -1; /* unspecified */
diff --git a/builtin/describe.c b/builtin/describe.c
index 6fb713b6be1..7e4481947c1 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -254,7 +254,7 @@ static unsigned long finish_depth_computation(
 			best->depth++;
 		while (parents) {
 			struct commit *p = parents->item;
-			parse_commit(p);
+			parse_commit(the_repository, p);
 			if (!(p->object.flags & SEEN))
 				commit_list_insert_by_date(p, list);
 			p->object.flags |= c->object.flags;
@@ -381,7 +381,7 @@ static void describe_commit(struct object_id *oid, struct strbuf *dst)
 		}
 		while (parents) {
 			struct commit *p = parents->item;
-			parse_commit(p);
+			parse_commit(the_repository, p);
 			if (!(p->object.flags & SEEN))
 				commit_list_insert_by_date(p, &list);
 			p->object.flags |= c->object.flags;
diff --git a/builtin/merge-base.c b/builtin/merge-base.c
index 08d91b1f0c0..426882818c8 100644
--- a/builtin/merge-base.c
+++ b/builtin/merge-base.c
@@ -127,7 +127,7 @@ static void add_one_commit(struct object_id *oid, struct rev_collect *revs)
 	commit = lookup_commit(the_repository, oid);
 	if (!commit ||
 	    (commit->object.flags & TMP_MARK) ||
-	    parse_commit(commit))
+	    parse_commit(the_repository, commit))
 		return;
 
 	ALLOC_GROW(revs->commit, revs->nr + 1, revs->alloc);
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index f1cb45c2274..bdeb3027179 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -87,7 +87,7 @@ static void name_rev(struct commit *commit,
 	int parent_number = 1;
 	char *to_free = NULL;
 
-	parse_commit(commit);
+	parse_commit(the_repository, commit);
 
 	if (commit->date < cutoff)
 		return;
diff --git a/builtin/notes.c b/builtin/notes.c
index 50963642075..0c558ab0702 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -714,7 +714,7 @@ static int merge_commit(struct notes_merge_options *o)
 		die(_("failed to read ref NOTES_MERGE_PARTIAL"));
 	else if (!(partial = lookup_commit_reference(the_repository, &oid)))
 		die(_("could not find commit from NOTES_MERGE_PARTIAL."));
-	else if (parse_commit(partial))
+	else if (parse_commit(the_repository, partial))
 		die(_("could not parse commit from NOTES_MERGE_PARTIAL."));
 
 	if (partial->parents)
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 6ff3c5abff0..c4e0ee7caad 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -235,7 +235,7 @@ static void mark_reachable(struct expire_reflog_policy_cb *cb)
 		struct commit *commit = pop_commit(&pending);
 		if (commit->object.flags & REACHABLE)
 			continue;
-		if (parse_commit(commit))
+		if (parse_commit(the_repository, commit))
 			continue;
 		commit->object.flags |= REACHABLE;
 		if (commit->date < expire_limit) {
diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index 4b9d3c0059b..98fb2cc2da3 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -238,7 +238,7 @@ static void join_revs(struct commit_list **list_p,
 			parents = parents->next;
 			if ((this_flag & flags) == flags)
 				continue;
-			parse_commit(p);
+			parse_commit(the_repository, p);
 			if (mark_seen(p, seen_p) && !still_interesting)
 				extra--;
 			p->object.flags |= flags;
@@ -835,7 +835,7 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 		if (!commit)
 			die(_("cannot find commit %s (%s)"),
 			    ref_name[num_rev], oid_to_hex(&revkey));
-		parse_commit(commit);
+		parse_commit(the_repository, commit);
 		mark_seen(commit, &seen);
 
 		/* rev#0 uses bit REV_SHIFT, rev#1 uses bit REV_SHIFT+1,
diff --git a/commit-graph.c b/commit-graph.c
index 6c08fc0a9ba..0e91d8f7139 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -371,7 +371,7 @@ static void write_graph_chunk_data(struct hashfile *f, int hash_len,
 		int edge_value;
 		uint32_t packedDate[2];
 
-		parse_commit(*list);
+		parse_commit(the_repository, *list);
 		hashwrite(f, (*list)->tree->object.oid.hash, hash_len);
 
 		parent = (*list)->parents;
@@ -540,7 +540,7 @@ static void close_reachable(struct packed_oid_list *oids)
 	for (i = 0; i < oids->nr; i++) {
 		commit = lookup_commit(the_repository, &oids->list[i]);
 
-		if (commit && !parse_commit(commit))
+		if (commit && !parse_commit(the_repository, commit))
 			add_missing_parents(oids, commit);
 	}
 
@@ -659,7 +659,7 @@ void write_commit_graph(const char *obj_dir,
 			continue;
 
 		commits.list[commits.nr] = lookup_commit(the_repository, &oids.list[i]);
-		parse_commit(commits.list[commits.nr]);
+		parse_commit(the_repository, commits.list[commits.nr]);
 
 		for (parent = commits.list[commits.nr]->parents;
 		     parent; parent = parent->next)
diff --git a/commit.c b/commit.c
index dcb32561275..8e7dfafe850 100644
--- a/commit.c
+++ b/commit.c
@@ -69,7 +69,7 @@ struct commit *lookup_commit_reference_by_name(const char *name)
 	if (get_oid_committish(name, &oid))
 		return NULL;
 	commit = lookup_commit_reference(the_repository, &oid);
-	if (parse_commit(commit))
+	if (parse_commit(the_repository, commit))
 		return NULL;
 	return commit;
 }
@@ -428,7 +428,7 @@ int parse_commit_gently_the_repository(struct commit *item, int quiet_on_missing
 
 void parse_commit_or_die(struct commit *item)
 {
-	if (parse_commit(item))
+	if (parse_commit(the_repository, item))
 		die("unable to parse commit %s",
 		    item ? oid_to_hex(&item->object.oid) : "(null)");
 }
@@ -533,7 +533,7 @@ struct commit *pop_most_recent_commit(struct commit_list **list,
 
 	while (parents) {
 		struct commit *commit = parents->item;
-		if (!parse_commit(commit) && !(commit->object.flags & mark)) {
+		if (!parse_commit(the_repository, commit) && !(commit->object.flags & mark)) {
 			commit->object.flags |= mark;
 			commit_list_insert_by_date(commit, list);
 		}
@@ -830,7 +830,7 @@ static struct commit_list *paint_down_to_common(struct commit *one, int n, struc
 			parents = parents->next;
 			if ((p->object.flags & flags) == flags)
 				continue;
-			if (parse_commit(p))
+			if (parse_commit(the_repository, p))
 				return NULL;
 			p->object.flags |= flags;
 			prio_queue_put(&queue, p);
@@ -856,10 +856,10 @@ static struct commit_list *merge_bases_many(struct commit *one, int n, struct co
 			return commit_list_insert(one, &result);
 	}
 
-	if (parse_commit(one))
+	if (parse_commit(the_repository, one))
 		return NULL;
 	for (i = 0; i < n; i++) {
-		if (parse_commit(twos[i]))
+		if (parse_commit(the_repository, twos[i]))
 			return NULL;
 	}
 
@@ -918,7 +918,7 @@ static int remove_redundant(struct commit **array, int cnt)
 	ALLOC_ARRAY(filled_index, cnt - 1);
 
 	for (i = 0; i < cnt; i++)
-		parse_commit(array[i]);
+		parse_commit(the_repository, array[i]);
 	for (i = 0; i < cnt; i++) {
 		struct commit_list *common;
 
@@ -1041,10 +1041,10 @@ int in_merge_bases_many(struct commit *commit, int nr_reference, struct commit *
 	struct commit_list *bases;
 	int ret = 0, i;
 
-	if (parse_commit(commit))
+	if (parse_commit(the_repository, commit))
 		return ret;
 	for (i = 0; i < nr_reference; i++)
-		if (parse_commit(reference[i]))
+		if (parse_commit(the_repository, reference[i]))
 			return ret;
 
 	bases = paint_down_to_common(commit, nr_reference, reference);
diff --git a/commit.h b/commit.h
index b1198ec2b77..c4d3033dc12 100644
--- a/commit.h
+++ b/commit.h
@@ -76,7 +76,8 @@ struct commit *lookup_commit_or_die(const struct object_id *oid, const char *ref
 int parse_commit_buffer_the_repository(struct commit *item, const void *buffer, unsigned long size);
 #define parse_commit_gently(r, i, q) parse_commit_gently_##r(i, q)
 int parse_commit_gently_the_repository(struct commit *item, int quiet_on_missing);
-static inline int parse_commit(struct commit *item)
+#define parse_commit(r, i) parse_commit_##r(i)
+static inline int parse_commit_the_repository(struct commit *item)
 {
 	return parse_commit_gently(the_repository, item, 0);
 }
diff --git a/fetch-pack.c b/fetch-pack.c
index 358d8421a8f..b551a7d921b 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -114,7 +114,7 @@ static void rev_list_push(struct commit *commit, int mark)
 	if (!(commit->object.flags & mark)) {
 		commit->object.flags |= mark;
 
-		if (parse_commit(commit))
+		if (parse_commit(the_repository, commit))
 			return;
 
 		prio_queue_put(&rev_list, commit);
@@ -178,7 +178,7 @@ static void mark_common(struct commit *commit,
 			if (!ancestors_only && !(o->flags & POPPED))
 				non_common_revs--;
 			if (!o->parsed && !dont_parse)
-				if (parse_commit(commit))
+				if (parse_commit(the_repository, commit))
 					return;
 
 			for (parents = commit->parents;
@@ -205,7 +205,7 @@ static const struct object_id *get_rev(void)
 			return NULL;
 
 		commit = prio_queue_get(&rev_list);
-		parse_commit(commit);
+		parse_commit(the_repository, commit);
 		parents = commit->parents;
 
 		commit->object.flags |= POPPED;
diff --git a/fsck.c b/fsck.c
index b07abb9796c..e8387e015c8 100644
--- a/fsck.c
+++ b/fsck.c
@@ -393,7 +393,7 @@ static int fsck_walk_commit(struct commit *commit, void *data, struct fsck_optio
 	int result;
 	const char *name;
 
-	if (parse_commit(commit))
+	if (parse_commit(the_repository, commit))
 		return -1;
 
 	name = get_object_name(options, &commit->object);
diff --git a/merge-recursive.c b/merge-recursive.c
index 9013aabed9a..bdbaff45fcf 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -236,7 +236,7 @@ static void output_commit_title(struct merge_options *o, struct commit *commit)
 		strbuf_add_unique_abbrev(&o->obuf, &commit->object.oid,
 					 DEFAULT_ABBREV);
 		strbuf_addch(&o->obuf, ' ');
-		if (parse_commit(commit) != 0)
+		if (parse_commit(the_repository, commit) != 0)
 			strbuf_addstr(&o->obuf, _("(bad commit)\n"));
 		else {
 			const char *title;
@@ -2193,7 +2193,7 @@ static struct commit *get_ref(const struct object_id *oid, const char *name)
 		return make_virtual_commit((struct tree*)object, name);
 	if (object->type != OBJ_COMMIT)
 		return NULL;
-	if (parse_commit((struct commit *)object))
+	if (parse_commit(the_repository, (struct commit *)object))
 		return NULL;
 	return (struct commit *)object;
 }
diff --git a/notes-utils.c b/notes-utils.c
index 14ea03178e9..8441ec46199 100644
--- a/notes-utils.c
+++ b/notes-utils.c
@@ -22,7 +22,7 @@ void create_notes_commit(struct notes_tree *t, struct commit_list *parents,
 		if (!read_ref(t->ref, &parent_oid)) {
 			struct commit *parent = lookup_commit(the_repository,
 							      &parent_oid);
-			if (parse_commit(parent))
+			if (parse_commit(the_repository, parent))
 				die("Failed to find/parse commit %s", t->ref);
 			commit_list_insert(parent, &parents);
 		}
diff --git a/remote.c b/remote.c
index 0bf5ce3e7ba..83a3d3ef649 100644
--- a/remote.c
+++ b/remote.c
@@ -2007,7 +2007,7 @@ int ref_newer(const struct object_id *new_oid, const struct object_id *old_oid)
 		return 0;
 	new_commit = (struct commit *) o;
 
-	if (parse_commit(new_commit) < 0)
+	if (parse_commit(the_repository, new_commit) < 0)
 		return 0;
 
 	used = list = NULL;
diff --git a/revision.c b/revision.c
index 33506faabda..6a3a73cc4ce 100644
--- a/revision.c
+++ b/revision.c
@@ -258,8 +258,7 @@ static struct commit *handle_commit(struct rev_info *revs,
 	 */
 	if (object->type == OBJ_COMMIT) {
 		struct commit *commit = (struct commit *)object;
-
-		if (parse_commit(commit) < 0)
+		if (parse_commit(the_repository, commit) < 0)
 			die("unable to parse commit %s", name);
 		if (flags & UNINTERESTING) {
 			mark_parents_uninteresting(commit);
@@ -673,7 +672,7 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 					ts->treesame[0] = 1;
 			}
 		}
-		if (parse_commit(p) < 0)
+		if (parse_commit(the_repository, p) < 0)
 			die("cannot simplify commit %s (because of %s)",
 			    oid_to_hex(&commit->object.oid),
 			    oid_to_hex(&p->object.oid));
@@ -706,7 +705,7 @@ static void try_to_simplify_commit(struct rev_info *revs, struct commit *commit)
 				 * IOW, we pretend this parent is a
 				 * "root" commit.
 				 */
-				if (parse_commit(p) < 0)
+				if (parse_commit(the_repository, p) < 0)
 					die("cannot simplify commit %s (invalid %s)",
 					    oid_to_hex(&commit->object.oid),
 					    oid_to_hex(&p->object.oid));
diff --git a/sequencer.c b/sequencer.c
index da6c9fc5511..d8ba41dc0c8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -553,7 +553,7 @@ static int is_index_unchanged(void)
 	 * the commit is invalid, parse_commit() will complain.  So
 	 * there is nothing for us to say here.  Just return failure.
 	 */
-	if (parse_commit(head_commit))
+	if (parse_commit(the_repository, head_commit))
 		return -1;
 
 	if (!active_cache_tree)
@@ -975,7 +975,7 @@ void print_commit_summary(const char *prefix, const struct object_id *oid,
 	commit = lookup_commit(the_repository, oid);
 	if (!commit)
 		die(_("couldn't look up newly created commit"));
-	if (parse_commit(commit))
+	if (parse_commit(the_repository, commit))
 		die(_("could not parse newly created commit"));
 
 	strbuf_addstr(&format, "format:%h] %s");
@@ -1054,7 +1054,7 @@ static int parse_head(struct commit **head)
 			warning(_("HEAD %s is not a commit!"),
 				oid_to_hex(&oid));
 		}
-		if (parse_commit(current_head))
+		if (parse_commit(the_repository, current_head))
 			return error(_("could not parse HEAD commit"));
 	}
 	*head = current_head;
@@ -1213,12 +1213,12 @@ static int is_original_commit_empty(struct commit *commit)
 {
 	const struct object_id *ptree_oid;
 
-	if (parse_commit(commit))
+	if (parse_commit(the_repository, commit))
 		return error(_("could not parse commit %s"),
 			     oid_to_hex(&commit->object.oid));
 	if (commit->parents) {
 		struct commit *parent = commit->parents->item;
-		if (parse_commit(parent))
+		if (parse_commit(the_repository, parent))
 			return error(_("could not parse parent commit %s"),
 				oid_to_hex(&parent->object.oid));
 		ptree_oid = &parent->tree->object.oid;
@@ -1533,7 +1533,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		msg_file = NULL;
 		goto fast_forward_edit;
 	}
-	if (parent && parse_commit(parent) < 0)
+	if (parent && parse_commit(the_repository, parent) < 0)
 		/* TRANSLATORS: The first %s will be a "todo" command like
 		   "revert" or "pick", the second %s a SHA1. */
 		return error(_("%s: cannot parse parent commit %s"),
@@ -3301,7 +3301,7 @@ int skip_unnecessary_picks(void)
 			continue;
 		if (item->command != TODO_PICK)
 			break;
-		if (parse_commit(item->commit)) {
+		if (parse_commit(the_repository, item->commit)) {
 			todo_list_release(&todo_list);
 			return error(_("could not parse commit '%s'"),
 				oid_to_hex(&item->commit->object.oid));
@@ -3429,7 +3429,7 @@ int rearrange_squash(void)
 
 		*commit_todo_item_at(&commit_todo, item->commit) = item;
 
-		parse_commit(item->commit);
+		parse_commit(the_repository, item->commit);
 		commit_buffer = get_commit_buffer(item->commit, NULL);
 		find_commit_subject(commit_buffer, &subject);
 		format_subject(&buf, subject, " ");
diff --git a/sha1-name.c b/sha1-name.c
index 27cf9cd916e..618fd1569f1 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -813,7 +813,7 @@ static int get_parent(const char *name, int len,
 	if (ret)
 		return ret;
 	commit = lookup_commit_reference(the_repository, &oid);
-	if (parse_commit(commit))
+	if (parse_commit(the_repository, commit))
 		return -1;
 	if (!idx) {
 		oidcpy(result, &commit->object.oid);
@@ -845,7 +845,7 @@ static int get_nth_ancestor(const char *name, int len,
 		return -1;
 
 	while (generation--) {
-		if (parse_commit(commit) || !commit->parents)
+		if (parse_commit(the_repository, commit) || !commit->parents)
 			return -1;
 		commit = commit->parents->item;
 	}
diff --git a/shallow.c b/shallow.c
index c65cb17195e..91d8da1917d 100644
--- a/shallow.c
+++ b/shallow.c
@@ -207,7 +207,7 @@ struct commit_list *get_shallow_commits_by_rev_list(int ac, const char **av,
 		struct commit *c = p->item;
 		struct commit_list *parent;
 
-		if (parse_commit(c))
+		if (parse_commit(the_repository, c))
 			die("unable to parse commit %s",
 			    oid_to_hex(&c->object.oid));
 
@@ -530,7 +530,7 @@ static void paint_down(struct paint_info *info, const struct object_id *oid,
 		if (c->object.flags & BOTTOM)
 			continue;
 
-		if (parse_commit(c))
+		if (parse_commit(the_repository, c))
 			die("unable to parse commit %s",
 			    oid_to_hex(&c->object.oid));
 
diff --git a/tree.c b/tree.c
index 33063b8dde0..6b83942e444 100644
--- a/tree.c
+++ b/tree.c
@@ -107,7 +107,7 @@ static int read_tree_1(struct tree *tree, struct strbuf *base,
 				    oid_to_hex(entry.oid),
 				    base->buf, entry.path);
 
-			if (parse_commit(commit))
+			if (parse_commit(the_repository, commit))
 				die("Invalid commit %s in submodule path %s%s",
 				    oid_to_hex(entry.oid),
 				    base->buf, entry.path);
diff --git a/walker.c b/walker.c
index c7c9db83df7..314f56e61ce 100644
--- a/walker.c
+++ b/walker.c
@@ -78,7 +78,7 @@ static int process_commit(struct walker *walker, struct commit *commit)
 {
 	struct commit_list *parents;
 
-	if (parse_commit(commit))
+	if (parse_commit(the_repository, commit))
 		return -1;
 
 	while (complete && complete->item->date >= commit->date) {
-- 
2.17.0.582.gccdcbd54c44.dirty


^ permalink raw reply	[relevance 2%]

* [PATCH 3/3] submodule--helper: plug mem leak in print_default_remote
      [irrelevant] <20180530170302.191176-1-sbeller@google.com>
@ 2018-05-30 17:03 ` Stefan Beller
  2018-05-31 12:07   ` Johannes Schindelin
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-30 17:03 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index 7c3cd9dbeba..96024fee1b1 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -63,6 +63,7 @@ static int print_default_remote(int argc, const char **argv, const char *prefix)
 	if (remote)
 		printf("%s\n", remote);
 
+	free(remote);
 	return 0;
 }
 
-- 
2.17.1.1185.g55be947832-goog


^ permalink raw reply	[relevance 13%]

* Re: git push requires a worktree?
      [irrelevant] <1527700562.4289.87.camel@novalis.org>
@ 2018-05-30 17:32 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-05-30 17:32 UTC (permalink / raw)
  To: David Turner; +Cc: git

On Wed, May 30, 2018 at 10:16 AM, David Turner <novalis@novalis.org> wrote:
> I am doing a funny thing where I do git -C .git/modules/morx push
> fleem:fleem.

I can do that, too. :)

> This is failing in the case where I have a sparse
> checkout and the worktree directory "morx" (which is where
> .git/modules/morx/config's core.worktree points) doesn't exist.

I could reproduce that without sparseness by removing that directory.

Before removing the dir in the worktree, the push worked fine; after
removing I got a

    fatal: cannot chdir to '../../../entropy': No such file or directory

> I don't know why git push cares about the worktree -- it'll happily
> work in a bare repo with no worktree at all, or if the worktree is an
> unrelated git repo or whatever.

I think we should unset the worktree config for submodules that are
not checked out. I confirm that running
  git config --unset -f .git/modules/<name>/config core.worktree
will have the push working as expected.

So I think it is not so much a bug in push, but there is
* a bug in the submodule data design: Why do we need to store
  core.worktree if there is no worktree? (I'll research if there are
  historic artefacts that would suggest we need that)
* the setup code for commands that do not have the
  NEED_WORK_TREE flag in git.c

> I can work around it, but if there's a bug, I think we should fix it.

I think so, too.

^ permalink raw reply	[relevance 5%]

* [ANNOUNCE] Git v2.18.0-rc0
@ 2018-05-30 22:47 Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-05-30 22:47 UTC (permalink / raw)
  To: git; +Cc: Linux Kernel, git-packagers

An early preview release Git v2.18.0-rc0 is now available for
testing at the usual places.  It is comprised of 802 non-merge
commits since v2.17.0, contributed by 60 people, 19 of which are
new faces.

As the above numbers show, especially the non-merge commit count, it
turned out that 2.18 is relatively sizeable cycle.  Let's plan to
start cooling down and giving it a bit of extra polish to make sure
there is no/little regression.  I am tagging the preview now as I
want to do the -rc1 before I go offline early next week for a few
days for travelling.  I may go offline for the rest of the day as I
seem to have caught something X-<, but plan to spend the next 48
hours re-reviewing what is in (or close to be in) 'next' to see which
ones should be in -rc1.

The tarballs are found at:

    https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the
'v2.18.0-rc0' tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.17.0 are as follows.
Welcome to the Git development community!

  Bill Ritcher, Birger Skogeng Pedersen, Casey Fitzpatrick,
  Dan Jacques, Drew DeVault, Eckhard S. Maaß, Erik E Brady,
  Florian Gamböck, Harald Nordgren, Leif Middelschulte, Loganaden
  Velvindron, Luis Marsano, Paul-Sebastian Ungureanu, Pratik
  Karki, Ryan Dammrose, Takuto Ikuta, Tao Qingyun, Wink Saville,
  and Yuki Kokubun.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Ævar Arnfjörð Bjarmason, Anders Kaseorg, Andreas Heiduk,
  Antonio Ospite, Beat Bolli, Ben Peart, Brandon Williams, brian
  m. carlson, Christian Couder, Christian Hesse, Clemens Buchacher,
  Derrick Stolee, Elijah Newren, Eric Sunshine, Jameson Miller,
  Jeff King, Johannes Schindelin, Johannes Sixt, Jonathan Nieder,
  Jonathan Tan, Junio C Hamano, Kaartic Sivaraam, Lars Schneider,
  Lucas Werkmeister, Martin Ågren, Michal Nazarewicz, Michele
  Locati, Nguyễn Thái Ngọc Duy, Olga Telezhnaya, Philip
  Oakley, Phillip Wood, Ramsay Jones, René Scharfe, Sergey
  Organov, Stefan Agner, Stefan Beller, SZEDER Gábor, Taylor Blau,
  Thomas Gummerer, Todd Zullinger, and Torsten Bögershausen.

----------------------------------------------------------------

Git 2.18 Release Notes (draft)
==============================

Updates since v2.17
-------------------

UI, Workflows & Features

 * Rename detection logic in "diff" family that is used in "merge" has
   learned to guess when all of x/a, x/b and x/c have moved to z/a,
   z/b and z/c, it is likely that x/d added in the meantime would also
   want to move to z/d by taking the hint that the entire directory
   'x' moved to 'z'.  A bug causing dirty files involved in a rename
   to be overwritten during merge has also been fixed as part of this
   work.

 * "git filter-branch" learned to use a different exit code to allow
   the callers to tell the case where there was no new commits to
   rewrite from other error cases.

 * When built with more recent cURL, GIT_SSL_VERSION can now specify
   "tlsv1.3" as its value.

 * "git gui" learned that "~/.ssh/id_ecdsa.pub" and
   "~/.ssh/id_ed25519.pub" are also possible SSH key files.
   (merge 2e2f0288ef bb/git-gui-ssh-key-files later to maint).

 * "git gui" performs commit upon CTRL/CMD+ENTER but the
   CTRL/CMD+KP_ENTER (i.e. enter key on the numpad) did not have the
   same key binding.  It now does.
   (merge 28a1d94a06 bp/git-gui-bind-kp-enter later to maint).

 * "git gui" has been taught to work with old versions of tk (like
   8.5.7) that do not support "ttk::style theme use" as a way to query
   the current theme.
   (merge 4891961105 cb/git-gui-ttk-style later to maint).

 * "git rebase" has learned to honor "--signoff" option when using
   backends other than "am" (but not "--preserve-merges").

 * "git branch --list" during an interrupted "rebase -i" now lets
   users distinguish the case where a detached HEAD is being rebased
   and a normal branch is being rebased.

 * "git mergetools" learned talking to guiffy.

 * The scripts in contrib/emacs/ have outlived their usefulness and
   have been replaced with a stub that errors out and tells the user
   there are replacements.

 * The new "checkout-encoding" attribute can ask Git to convert the
   contents to the specified encoding when checking out to the working
   tree (and the other way around when checking in).

 * The "git config" command uses separate options e.g. "--int",
   "--bool", etc. to specify what type the caller wants the value to
   be interpreted as.  A new "--type=<typename>" option has been
   introduced, which would make it cleaner to define new types.

 * "git config --get" learned the "--default" option, to help the
   calling script.  Building on top of the above changes, the
   "git config" learns "--type=color" type.  Taken together, you can
   do things like "git config --get foo.color --default blue" and get
   the ANSI color sequence for the color given to foo.color variable,
   or "blue" if the variable does not exist.

 * "git ls-remote" learned an option to allow sorting its output based
   on the refnames being shown.

 * The command line completion (in contrib/) has been taught that "git
   stash save" has been deprecated ("git stash push" is the preferred
   spelling in the new world) and does not offer it as a possible
   completion candidate when "git stash push" can be.

 * "git gc --prune=nonsense" spent long time repacking and then
   silently failed when underlying "git prune --expire=nonsense"
   failed to parse its command line.  This has been corrected.

 * Error messages from "git push" can be painted for more visibility.

 * "git http-fetch" (deprecated) had an optional and experimental
   "feature" to fetch only commits and/or trees, which nobody used.
   This has been removed.

 * The functionality of "$GIT_DIR/info/grafts" has been superseded by
   the "refs/replace/" mechanism for some time now, but the internal
   code had support for it in many places, which has been cleaned up
   in order to drop support of the "grafts" mechanism.

 * "git worktree add" learned to check out an existing branch.

 * "git --no-pager cmd" did not have short-and-sweet single letter
   option. Now it does as "-P".
   (merge 7213c28818 js/no-pager-shorthand later to maint).

 * "git rebase" learned "--rebase-merges" to transplant the whole
   topology of commit graph elsewhere.

 * "git status" learned to pay attention to UI related diff
   configuration variables such as diff.renames.

 * The command line completion mechanism (in contrib/) learned to load
   custom completion file for "git $command" where $command is a
   custom "git-$command" that the end user has on the $PATH when using
   newer version of bash.

 * "git send-email" can sometimes offer confirmation dialog "Send this
   email?" with choices 'Yes', 'No', 'Quit', and 'All'.  A new action
   'Edit' has been added to this dialog's choice.

 * With merge.renames configuration set to false, the recursive merge
   strategy can be told not to spend cycles trying to find renamed
   paths and merge them accordingly.

 * "git status" learned to honor a new status.renames configuration to
   skip rename detection, which could be useful for those who want to
   do so without disabling the default rename detection done by the
   "git diff" command.

 * Command line completion (in contrib/) learned to complete pathnames
   for various commands better.

 * "git blame" learns to unhighlight uninteresting metadata from the
   originating commit on lines that are the same as the previous one,
   and also paint lines in different colors depending on the age of
   the commit.

 * Transfer protocol v2 learned to support the partial clone.

 * When a short hexadecimal string is used to name an object but there
   are multiple objects that share the string as the prefix of their
   names, the code lists these ambiguous candidates in a help message.
   These object names are now sorted according to their types for
   easier eyeballing.

 * "git fetch $there $refspec" that talks over protocol v2 can take
   advantage of server-side ref filtering; the code has been extended
   so that this mechanism triggers also when fetching with configured
   refspec.

 * Our HTTP client code used to advertise that we accept gzip encoding
   from the other side; instead, just let cURL library to advertise
   and negotiate the best one.


Performance, Internal Implementation, Development Support etc.

 * A "git fetch" from a repository with insane number of refs into a
   repository that is already up-to-date still wasted too many cycles
   making many lstat(2) calls to see if these objects at the tips
   exist as loose objects locally.  These lstat(2) calls are optimized
   away by enumerating all loose objects beforehand.
   It is unknown if the new strategy negatively affects existing use
   cases, fetching into a repository with many loose objects from a
   repository with small number of refs.

 * Git can be built to use either v1 or v2 of the PCRE library, and so
   far, the build-time configuration USE_LIBPCRE=YesPlease instructed
   the build procedure to use v1, but now it means v2.  USE_LIBPCRE1
   and USE_LIBPCRE2 can be used to explicitly choose which version to
   use, as before.

 * The build procedure learned to optionally use symbolic links
   (instead of hardlinks and copies) to install "git-foo" for built-in
   commands, whose binaries are all identical.

 * Conversion from uchar[20] to struct object_id continues.

 * The way "git worktree prune" worked internally has been simplified,
   by assuming how "git worktree move" moves an existing worktree to a
   different place.

 * Code clean-up for the "repository" abstraction.
   (merge 00a3da2a13 nd/remove-ignore-env-field later to maint).

 * Code to find the length to uniquely abbreviate object names based
   on packfile content, which is a relatively recent addtion, has been
   optimized to use the same fan-out table.

 * The mechanism to use parse-options API to automate the command line
   completion continues to get extended and polished.

 * Copies of old scripted Porcelain commands in contrib/examples/ have
   been removed.

 * Some tests that rely on the exact hardcoded values of object names
   have been updated in preparation for hash function migration.

 * Perf-test update.

 * Test helper update.

 * The effort continues to refactor the internal global data structure
   to make it possible to open multiple repositories, work with and
   then close them,

 * Small test-helper programs have been consolidated into a single
   binary.

 * API clean-up around ref-filter code.

 * Shell completion (in contrib) that gives list of paths have been
   optimized somewhat.

 * The index file is updated to record the fsmonitor section after a
   full scan was made, to avoid wasting the effort that has already
   spent.

 * Performance measuring framework in t/perf learned to help bisecting
   performance regressions.

 * Some multi-word source filenames are being renamed to separate
   words with dashes instead of underscores.

 * An reusable "memory pool" implementation has been extracted from
   fast-import.c, which in turn has become the first user of the
   mem-pool API.

 * A build-time option has been added to allow Git to be told to refer
   to its associated files relative to the main binary, in the same
   way that has been possible on Windows for quite some time, for
   Linux, BSDs and Darwin.

 * Precompute and store information necessary for ancestry traversal
   in a separate file to optimize graph walking.

 * The effort to pass the repository in-core structure throughout the
   API continues.  This round deals with the code that implements the
   refs/replace/ mechanism.

 * The build procedure "make DEVELOPER=YesPlease" learned to enable a
   bit more warning options depending on the compiler used to help
   developers more.  There also is "make DEVOPTS=tokens" knob
   available now, for those who want to help fixing warnings we
   usually ignore, for example.

 * A new version of the transport protocol is being worked on.

 * The code to interface to GPG has been restructured somewhat to make
   it cleaner to integrate with other types of signature systems later.

 * The code has been taught to use the duplicated information stored
   in the commit-graph file to learn the tree object name for a commit
   to avoid opening and parsing the commit object when it makes sense
   to do so.

 * "git gc" in a large repository takes a lot of time as it considers
   to repack all objects into one pack by default.  The command has
   been taught to pretend as if the largest existing packfile is
   marked with ".keep" so that it is left untouched while objects in
   other packs and loose ones are repacked.

 * The transport protocol v2 is getting updated further.

 * The codepath around object-info API has been taught to take the
   repository object (which in turn tells the API which object store
   the objects are to be located).

 * Rename detection logic in "diff" family that is used in "merge" has
   learned to guess when all of x/a, x/b and x/c have moved to z/a,
   z/b and z/c, it is likely that x/d added in the meantime would also
   want to move to z/d by taking the hint that the entire directory
   'x' moved to 'z'.  A bug causing dirty files involved in a rename
   to be overwritten during merge has also been fixed as part of this
   work.  Incidentally, this also avoids updating a file in the
   working tree after a (non-trivial) merge whose result matches what
   our side originally had.

 * "git pack-objects" needs to allocate tons of "struct object_entry"
   while doing its work, and shrinking its size helps the performance
   quite a bit.

 * The implementation of "git rebase -i --root" has been updated to use
   the sequencer machinery more.

 * Developer support update, by using BUG() macro instead of die() to
   mark codepaths that should not happen more clearly.

 * Developer support.  Use newer GCC on one of the builds done at
   TravisCI.org to get more warnings and errors diagnosed.

 * Conversion from uchar[20] to struct object_id continues.

 * By code restructuring of submodule merge in merge-recursive,
   informational messages from the codepath are now given using the
   same mechanism as other output, and honor the merge.verbosity
   configuration.  The code also learned to give a few new messages
   when a submodule three-way merge resolves cleanly when one side
   records a descendant of the commit chosen by the other side.

 * Avoid unchecked snprintf() to make future code auditing easier.
   (merge ac4896f007 jk/snprintf-truncation later to maint).

 * Many tests hardcode the raw object names, which would change once
   we migrate away from SHA-1.  While some of them must test against
   exact object names, most of them do not have to use hardcoded
   constants in the test.  The latter kind of tests have been updated
   to test the moral equivalent of the original without hardcoding the
   actual object names.


Also contains various documentation updates and code clean-ups.


Fixes since v2.17
-----------------

 * "git shortlog cruft" aborted with a BUG message when run outside a
   Git repository.  The command has been taught to complain about
   extra and unwanted arguments on its command line instead in such a
   case.
   (merge 4aa0161e83 ma/shortlog-revparse later to maint).

 * "git stash push -u -- <pathspec>" gave an unnecessary and confusing
   error message when there was no tracked files that match the
   <pathspec>, which has been fixed.
   (merge 353278687e tg/stash-untracked-with-pathspec-fix later to maint).

 * "git tag --contains no-such-commit" gave a full list of options
   after giving an error message.
   (merge 3bb0923f06 ps/contains-id-error-message later to maint).

 * "diff-highlight" filter (in contrib/) learned to undertand "git log
   --graph" output better.
   (merge 4551fbba14 jk/diff-highlight-graph-fix later to maint).

 * when refs that do not point at committish are given, "git
   filter-branch" gave a misleading error messages.  This has been
   corrected.
   (merge f78ab355e7 yk/filter-branch-non-committish-refs later to maint).

 * "git submodule status" misbehaved on a submodule that has been
   removed from the working tree.
   (merge 74b6bda32f rs/status-with-removed-submodule later to maint).

 * When credential helper exits very quickly without reading its
   input, it used to cause Git to die with SIGPIPE, which has been
   fixed.
   (merge a0d51e8d0e eb/cred-helper-ignore-sigpipe later to maint).

 * "git rebase --keep-empty" still removed an empty commit if the
   other side contained an empty commit (due to the "does an
   equivalent patch exist already?" check), which has been corrected.
   (merge 3d946165e1 pw/rebase-keep-empty-fixes later to maint).

 * Some codepaths, including the refs API, get and keep relative
   paths, that go out of sync when the process does chdir(2).  The
   chdir-notify API is introduced to let these codepaths adjust these
   cached paths to the new current directory.
   (merge fb9c2d2703 jk/relative-directory-fix later to maint).

 * "cd sub/dir && git commit ../path" ought to record the changes to
   the file "sub/path", but this regressed long time ago.
   (merge 86238e07ef bw/commit-partial-from-subdirectory-fix later to maint).

 * Recent introduction of "--log-destination" option to "git daemon"
   did not work well when the daemon was run under "--inetd" mode.
   (merge e67d906d73 lw/daemon-log-destination later to maint).

 * Small fix to the autoconf build procedure.
   (merge 249482daf0 es/fread-reads-dir-autoconf-fix later to maint).

 * Fix an unexploitable (because the oversized contents are not under
   attacker's control) buffer overflow.
   (merge d8579accfa bp/fsmonitor-bufsize-fix later to maint).

 * Recent simplification of build procedure forgot a bit of tweak to
   the build procedure of contrib/mw-to-git/
   (merge d8698987f3 ab/simplify-perl-makefile later to maint).

 * Moving a submodule that itself has submodule in it with "git mv"
   forgot to make necessary adjustment to the nested sub-submodules;
   now the codepath learned to recurse into the submodules.

 * "git config --unset a.b", when "a.b" is the last variable in an
   otherwise empty section "a", left an empty section "a" behind, and
   worse yet, a subsequent "git config a.c value" did not reuse that
   empty shell and instead created a new one.  These have been
   (partially) corrected.
   (merge c71d8bb38a js/empty-config-section-fix later to maint).

 * "git worktree remove" learned that "-f" is a shorthand for
   "--force" option, just like for "git worktree add".
   (merge d228eea514 sb/worktree-remove-opt-force later to maint).

 * The completion script (in contrib/) learned to clear cached list of
   command line options upon dot-sourcing it again in a more efficient
   way.
   (merge 94408dc71c sg/completion-clear-cached later to maint).

 * "git svn" had a minor thinko/typo which has been fixed.
   (merge 51db271587 ab/git-svn-get-record-typofix later to maint).

 * During a "rebase -i" session, the code could give older timestamp
   to commits created by later "pick" than an earlier "reword", which
   has been corrected.
   (merge 12f7babd6b js/ident-date-fix later to maint).

 * "git submodule status" did not check the symbolic revision name it
   computed for the submodule HEAD is not the NULL, and threw it at
   printf routines, which has been corrected.
   (merge 0b5e2ea7cf nd/submodule-status-fix later to maint).

 * When fed input that already has In-Reply-To: and/or References:
   headers and told to add the same information, "git send-email"
   added these headers separately, instead of appending to an existing
   one, which is a violation of the RFC.  This has been corrected.
   (merge 256be1d3f0 sa/send-email-dedup-some-headers later to maint).

 * "git fast-export" had a regression in v2.15.0 era where it skipped
   some merge commits in certain cases, which has been corrected.
   (merge be011bbe00 ma/fast-export-skip-merge-fix later to maint).

 * The code did not propagate the terminal width to subprocesses via
   COLUMNS environment variable, which it now does.  This caused
   trouble to "git column" helper subprocess when "git tag --column=row"
   tried to list the existing tags on a display with non-default width.
   (merge b5d5a567fb nd/term-columns later to maint).

 * We learned that our source files with ".pl" and ".py" extensions
   are Perl and Python files respectively and changes to them are
   better viewed as such with appropriate diff drivers.
   (merge 7818b619e2 ab/perl-python-attrs later to maint).

 * "git rebase -i" sometimes left intermediate "# This is a
   combination of N commits" message meant for the human consumption
   inside an editor in the final result in certain corner cases, which
   has been fixed.
   (merge 15ef69314d js/rebase-i-clean-msg-after-fixup-continue later to maint).

 * A test to see if the filesystem normalizes UTF-8 filename has been
   updated to check what we need to know in a more direct way, i.e. a
   path created in NFC form can be accessed with NFD form (or vice
   versa) to cope with APFS as well as HFS.
   (merge 742ae10e35 tb/test-apfs-utf8-normalization later to maint).

 * "git format-patch --cover --attach" created a broken MIME multipart
   message for the cover letter, which has been fixed by keeping the
   cover letter as plain text file.
   (merge 50cd54ef4e bc/format-patch-cover-no-attach later to maint).

 * The split-index feature had a long-standing and dormant bug in
   certain use of the in-core merge machinery, which has been fixed.
   (merge 7db118303a en/unpack-trees-split-index-fix later to maint).

 * Asciidoctor gives a reasonable imitation for AsciiDoc, but does not
   render illustration in a literal block correctly when indented with
   HT by default. The problem is fixed by forcing 8-space tabs.
   (merge 379805051d bc/asciidoctor-tab-width later to maint).

 * Code clean-up to adjust to a more recent lockfile API convention that
   allows lockfile instances kept on the stack.
   (merge 0fa5a2ed8d ma/lockfile-cleanup later to maint).

 * the_repository->index is not a allocated piece of memory but
   repo_clear() indiscriminately attempted to free(3) it, which has
   been corrected.
   (merge 74373b5f10 nd/repo-clear-keep-the-index later to maint).

 * Code clean-up to avoid non-standard-conformant pointer arithmetic.
   (merge c112084af9 rs/no-null-ptr-arith-in-fast-export later to maint).

 * Code clean-up to turn history traversal more robust in a
   semi-corrupt repository.
   (merge 8702b30fd7 jk/unavailable-can-be-missing later to maint).

 * "git update-ref A B" is supposed to ensure that ref A does not yet
   exist when B is a NULL OID, but this check was not done correctly
   for pseudo-refs outside refs/ hierarchy, e.g. MERGE_HEAD.

 * "git submodule update" and "git submodule add" supported the
   "--reference" option to borrow objects from a neighbouring local
   repository like "git clone" does, but lacked the more recent
   invention "--dissociate".  Also "git submodule add" has been taught
   to take the "--progress" option.
   (merge a0ef29341a cf/submodule-progress-dissociate later to maint).

 * Update credential-netrc helper (in contrib/) to allow customizing
   the GPG used to decrypt the encrypted .netrc file.
   (merge 786ef50a23 lm/credential-netrc later to maint).

 * "git submodule update" attempts two different kinds of "git fetch"
   against the upstream repository to grab a commit bound at the
   submodule's path, but it incorrectly gave up if the first kind
   (i.e. a normal fetch) failed, making the second "last resort" one
   (i.e. fetching an exact commit object by object name) ineffective.
   This has been corrected.
   (merge e30d833671 sb/submodule-update-try-harder later to maint).

 * Error behaviour of "git grep" when it cannot read the index was
   inconsistent with other commands that uses the index, which has
   been corrected to error out early.
   (merge b2aa84c789 sb/grep-die-on-unreadable-index later to maint).

 * We used to call regfree() after regcomp() failed in some codepaths,
   which have been corrected.
   (merge 17154b1576 ma/regex-no-regfree-after-comp-fail later to maint).

 * Other minor doc, test and build updates and code cleanups.
   (merge 248f66ed8e nd/trace-with-env later to maint).
   (merge 14ced5562c ys/bisect-object-id-missing-conversion-fix later to maint).
   (merge 5988eb631a ab/doc-hash-brokenness later to maint).
   (merge a4d4e32a70 pk/test-avoid-pipe-hiding-exit-status later to maint).
   (merge 05e293c1ac jk/flockfile-stdio later to maint).
   (merge e9184b0789 jk/t5561-missing-curl later to maint).
   (merge b1801b85a3 nd/worktree-move later to maint).
   (merge bbd374dd20 ak/bisect-doc-typofix later to maint).
   (merge 4855f06fb3 mn/send-email-credential-doc later to maint).
   (merge 8523b1e355 en/doc-typoes later to maint).
   (merge 43b44ccfe7 js/t5404-path-fix later to maint).
   (merge decf711fc1 ps/test-chmtime-get later to maint).
   (merge 22d11a6e8e es/worktree-docs later to maint).
   (merge 92a5dbbc22 tg/use-git-contacts later to maint).
   (merge adc887221f tq/t1510 later to maint).
   (merge bed21a8ad6 sg/doc-gc-quote-mismatch-fix later to maint).
   (merge 73364e4f10 tz/doc-git-urls-reference later to maint).
   (merge cd1e606bad bc/mailmap-self later to maint).
   (merge f7997e3682 ao/config-api-doc later to maint).
   (merge ee930754d8 jk/apply-p-doc later to maint).
   (merge 011b648646 nd/pack-format-doc later to maint).
   (merge 87a6bb701a sg/t5310-jgit-bitmap-test later to maint).
   (merge f6b82970aa sg/t5516-fixes later to maint).
   (merge 4362da078e sg/t7005-spaces-in-filenames-cleanup later to maint).
   (merge 7d0ee47c11 js/test-unset-prereq later to maint).
   (merge 5356a3c354 ah/misc-doc-updates later to maint).
   (merge 92c4a7a129 nd/completion-aliasfiletype-typofix later to maint).
   (merge 58bd77b66a nd/pack-unreachable-objects-doc later to maint).
   (merge 4ed79d5203 sg/t6500-no-redirect-of-stdin later to maint).
   (merge 17b8a2d6cd jk/config-blob-sans-repo later to maint).

----------------------------------------------------------------

Changes since v2.17.0 are as follows:

Anders Kaseorg (1):
      Documentation/git-bisect.txt: git bisect term → git bisect terms

Andreas Heiduk (9):
      git-svn: search --authors-prog in PATH too
      git-svn: allow empty email-address using authors-prog and authors-file
      doc: improve formatting in githooks.txt
      doc: align 'diff --no-index' in text and synopsis
      doc: clarify ignore rules for git ls-files
      doc: add '-d' and '-o' for 'git push'
      git-svn: remove ''--add-author-from' for 'commit-diff'
      doc: add note about shell quoting to revision.txt
      doc: normalize [--options] to [options] in git-diff

Antonio Ospite (1):
      doc: fix config API documentation about config_with_options

Beat Bolli (1):
      git-gui: search for all current SSH key types

Ben Peart (7):
      fsmonitor: fix incorrect buffer size when printing version number
      fsmonitor: force index write after full scan
      test-drop-caches: simplify delay loading of NtSetSystemInformation
      merge: update documentation for {merge,diff}.renameLimit
      merge: add merge.renames config setting
      merge: pass aggressive when rename detection is turned off
      add status config and command line options for rename detection

Bill Ritcher (1):
      mergetools: add support for guiffy

Birger Skogeng Pedersen (1):
      git-gui: bind CTRL/CMD+numpad ENTER to do_commit

Brandon Williams (79):
      pkt-line: introduce packet_read_with_status
      pkt-line: allow peeking a packet line without consuming it
      pkt-line: add delim packet support
      upload-pack: convert to a builtin
      upload-pack: factor out processing lines
      transport: use get_refs_via_connect to get refs
      connect: convert get_remote_heads to use struct packet_reader
      connect: discover protocol version outside of get_remote_heads
      transport: store protocol version
      protocol: introduce enum protocol_version value protocol_v2
      test-pkt-line: introduce a packet-line test helper
      serve: introduce git-serve
      ls-refs: introduce ls-refs server command
      connect: request remote refs using v2
      transport: convert get_refs_list to take a list of ref prefixes
      transport: convert transport_get_remote_refs to take a list of ref prefixes
      ls-remote: pass ref prefixes when requesting a remote's refs
      fetch: pass ref prefixes when fetching
      push: pass ref prefixes when pushing
      upload-pack: introduce fetch server command
      fetch-pack: perform a fetch using v2
      fetch-pack: support shallow requests
      connect: refactor git_connect to only get the protocol version once
      connect: don't request v2 when pushing
      transport-helper: remove name parameter
      transport-helper: refactor process_connect_service
      transport-helper: introduce stateless-connect
      pkt-line: add packet_buf_write_len function
      remote-curl: create copy of the service name
      remote-curl: store the protocol version the server responded with
      http: allow providing extra headers for http requests
      http: don't always add Git-Protocol header
      http: eliminate "# service" line when using protocol v2
      remote-curl: implement stateless-connect command
      remote-curl: don't request v2 when pushing
      commit: allow partial commits with relative paths
      serve: introduce the server-option capability
      ls-remote: send server options when using protocol v2
      fetch: send server options when using protocol v2
      refspec: move refspec parsing logic into its own file
      refspec: rename struct refspec to struct refspec_item
      refspec: factor out parsing a single refspec
      refspec: introduce struct refspec
      refspec: convert valid_fetch_refspec to use parse_refspec
      submodule--helper: convert push_check to use struct refspec
      pull: convert get_tracking_branch to use refspec_item_init
      transport: convert transport_push to use struct refspec
      remote: convert check_push_refs to use struct refspec
      remote: convert match_push_refs to use struct refspec
      clone: convert cmd_clone to use refspec_item_init
      fast-export: convert to use struct refspec
      remote: convert push refspecs to struct refspec
      remote: convert fetch refspecs to struct refspec
      remote: remove add_prune_tags_to_fetch_refspec
      transport-helper: convert to use struct refspec
      fetch: convert fetch_one to use struct refspec
      fetch: convert refmap to use struct refspec
      refspec: remove the deprecated functions
      fetch: convert do_fetch to take a struct refspec
      fetch: convert get_ref_map to take a struct refspec
      fetch: convert prune_refs to take a struct refspec
      remote: convert get_stale_heads to take a struct refspec
      remote: convert apply_refspecs to take a struct refspec
      remote: convert query_refspecs to take a struct refspec
      remote: convert get_ref_match to take a struct refspec
      remote: convert match_explicit_refs to take a struct refspec
      push: check for errors earlier
      push: convert to use struct refspec
      transport: convert transport_push to take a struct refspec
      send-pack: store refspecs in a struct refspec
      transport: remove transport_verify_remote_names
      http-push: store refspecs in a struct refspec
      remote: convert match_push_refs to take a struct refspec
      remote: convert check_push_refs to take a struct refspec
      submodule: convert push_unpushed_submodules to take a struct refspec
      refspec: consolidate ref-prefix generation logic
      fetch: generate ref-prefixes when using a configured refspec
      remote-curl: accept all encodings supported by curl
      remote-curl: accept compressed responses with protocol v2

Casey Fitzpatrick (3):
      submodule: clean up substitutions in script
      submodule: add --progress option to add command
      submodule: add --dissociate option to add/update commands

Christian Couder (6):
      perf/aggregate: add display_dir()
      perf/aggregate: add --sort-by=regression option
      perf/run: add --subsection option
      t/perf: add scripts to bisect performance regressions
      perf/aggregate: use Getopt::Long for option parsing
      perf/bisect_run_script: disable codespeed

Christian Hesse (2):
      perl: fix installing modules from contrib
      Makefile: mark perllibdir as a .PHONY target

Clemens Buchacher (2):
      git-gui: workaround ttk:style theme use
      completion: improve ls-files filter performance

Dan Jacques (3):
      Makefile: generate Perl header from template file
      Makefile: add Perl runtime prefix support
      exec_cmd: RUNTIME_PREFIX on some POSIX systems

Derrick Stolee (20):
      packfile: define and use bsearch_pack()
      sha1_name: use bsearch_pack() for abbreviations
      csum-file: rename hashclose() to finalize_hashfile()
      csum-file: refactor finalize_hashfile() method
      commit-graph: add format document
      graph: add commit graph design document
      commit-graph: create git-commit-graph builtin
      commit-graph: implement write_commit_graph()
      commit-graph: implement git-commit-graph write
      commit-graph: implement git commit-graph read
      commit-graph: add core.commitGraph setting
      commit-graph: close under reachability
      commit: integrate commit graph with commit parsing
      commit-graph: read only from specific pack-indexes
      commit-graph: build graph from starting commits
      commit-graph: implement "--append" option
      treewide: rename tree to maybe_tree
      commit: create get_commit_tree() method
      treewide: replace maybe_tree with accessor methods
      commit-graph: lazy-load trees for commits

Drew DeVault (1):
      git-send-email: allow re-editing of message

Eckhard S. Maaß (1):
      wt-status: use settings from git_diff_ui_config

Elijah Newren (70):
      directory rename detection: basic testcases
      directory rename detection: directory splitting testcases
      directory rename detection: testcases to avoid taking detection too far
      directory rename detection: partially renamed directory testcase/discussion
      directory rename detection: files/directories in the way of some renames
      directory rename detection: testcases checking which side did the rename
      directory rename detection: more involved edge/corner testcases
      directory rename detection: testcases exploring possibly suboptimal merges
      directory rename detection: miscellaneous testcases to complete coverage
      directory rename detection: tests for handling overwriting untracked files
      directory rename detection: tests for handling overwriting dirty files
      merge-recursive: move the get_renames() function
      merge-recursive: introduce new functions to handle rename logic
      merge-recursive: fix leaks of allocated renames and diff_filepairs
      merge-recursive: make !o->detect_rename codepath more obvious
      merge-recursive: split out code for determining diff_filepairs
      merge-recursive: make a helper function for cleanup for handle_renames
      merge-recursive: add get_directory_renames()
      merge-recursive: check for directory level conflicts
      merge-recursive: add computation of collisions due to dir rename & merging
      merge-recursive: check for file level conflicts then get new name
      merge-recursive: when comparing files, don't include trees
      merge-recursive: apply necessary modifications for directory renames
      merge-recursive: avoid clobbering untracked files with directory renames
      merge-recursive: fix overwriting dirty files involved in renames
      merge-recursive: fix remaining directory rename + dirty overwrite cases
      directory rename detection: new testcases showcasing a pair of bugs
      merge-recursive: avoid spurious rename/rename conflict from dir renames
      merge-recursive: ensure we write updates for directory-renamed file
      Documentation: fix several one-character-off spelling errors
      Documentation: normalize spelling of 'normalised'
      directory rename detection: basic testcases
      directory rename detection: directory splitting testcases
      directory rename detection: testcases to avoid taking detection too far
      directory rename detection: partially renamed directory testcase/discussion
      directory rename detection: files/directories in the way of some renames
      directory rename detection: testcases checking which side did the rename
      directory rename detection: more involved edge/corner testcases
      directory rename detection: testcases exploring possibly suboptimal merges
      directory rename detection: miscellaneous testcases to complete coverage
      directory rename detection: tests for handling overwriting untracked files
      directory rename detection: tests for handling overwriting dirty files
      merge-recursive: move the get_renames() function
      merge-recursive: introduce new functions to handle rename logic
      merge-recursive: fix leaks of allocated renames and diff_filepairs
      merge-recursive: make !o->detect_rename codepath more obvious
      merge-recursive: split out code for determining diff_filepairs
      merge-recursive: make a helper function for cleanup for handle_renames
      Make running git under other debugger-like programs easy
      unpack_trees: fix breakage when o->src_index != o->dst_index
      merge-recursive: add get_directory_renames()
      merge-recursive: check for directory level conflicts
      merge-recursive: add computation of collisions due to dir rename & merging
      merge-recursive: check for file level conflicts then get new name
      merge-recursive: when comparing files, don't include trees
      merge-recursive: apply necessary modifications for directory renames
      merge-recursive: avoid clobbering untracked files with directory renames
      merge-recursive: fix overwriting dirty files involved in renames
      merge-recursive: fix remaining directory rename + dirty overwrite cases
      directory rename detection: new testcases showcasing a pair of bugs
      merge-recursive: avoid spurious rename/rename conflict from dir renames
      merge-recursive: improve add_cacheinfo error handling
      merge-recursive: move more is_dirty handling to merge_content
      merge-recursive: avoid triggering add_cacheinfo error with dirty mod
      t6046: testcases checking whether updates can be skipped in a merge
      merge-recursive: fix was_tracked() to quit lying with some renamed paths
      merge-recursive: fix remainder of was_dirty() to use original index
      merge-recursive: make "Auto-merging" comment show for other merges
      merge-recursive: fix check for skipability of working tree updates
      merge-recursive: provide pair of `unpack_trees_{start,finish}()`

Eric Sunshine (5):
      t3200: verify "branch --list" sanity when rebasing from detached HEAD
      t2028: tighten grep expression to make "move worktree" test more robust
      git-worktree.txt: recommend 'git worktree remove' over manual deletion
      git-worktree.txt: unify command-line prompt in example blocks
      configure.ac: fix botched FREAD_READS_DIRECTORIES check

Erik E Brady (1):
      credential: ignore SIGPIPE when writing to credential helpers

Florian Gamböck (1):
      completion: load completion file for external subcommand

Harald Nordgren (1):
      ls-remote: create '--sort' option

Jameson Miller (3):
      fast-import: rename mem_pool type to mp_block
      fast-import: introduce mem_pool type
      mem-pool: move reusable parts of memory pool into its own file

Jeff King (54):
      diff-highlight: correct test graph diagram
      diff-highlight: use test_tick in graph test
      diff-highlight: prefer "echo" to "cat" in tests
      diff-highlight: test interleaved parallel lines of history
      diff-highlight: test graphs with --color
      diff-highlight: use flush() helper consistently
      diff-highlight: detect --graph by indent
      set_git_dir: die when setenv() fails
      add chdir-notify API
      set_work_tree: use chdir_notify
      refs: use chdir_notify to update cached relative paths
      config: move flockfile() closer to unlocked functions
      t5561: drop curl stderr redirects
      t5561: skip tests if curl is not available
      ref-filter: use "struct object_id" consistently
      ref-filter: make ref_array_item allocation more consistent
      ref-filter: factor ref_array pushing into its own function
      t7004: fix mistaken tag name
      gpg-interface: handle bool user.signingkey
      gpg-interface: modernize function declarations
      gpg-interface: use size_t for signature buffer size
      gpg-interface: fix const-correctness of "eol" pointer
      gpg-interface: extract gpg line matching helper
      gpg-interface: find the last gpg signature line
      apply: clarify "-p" documentation
      pager: set COLUMNS to term_columns()
      mark_tree_contents_uninteresting(): drop missing object check
      mark_parents_uninteresting(): drop missing object check
      mark_parents_uninteresting(): replace list with stack
      mark_parents_uninteresting(): avoid most allocation
      get_main_ref_store: BUG() when outside a repository
      config: die when --blob is used outside a repository
      http: use strbufs instead of fixed buffers
      log_write_email_headers: use strbufs
      shorten_unambiguous_ref: use xsnprintf
      fmt_with_err: add a comment that truncation is OK
      submodule-config: verify submodule names as paths
      is_ntfs_dotgit: use a size_t for traversing string
      is_hfs_dotgit: match other .git files
      skip_prefix: add case-insensitive variant
      verify_path: drop clever fallthrough
      verify_dotfile: mention case-insensitivity in comment
      update-index: stat updated files earlier
      verify_path: disallow symlinks in .gitmodules
      index-pack: make fsck error message more specific
      fsck: simplify ".git" check
      fsck: actually fsck blob data
      fsck: detect gitmodules files
      fsck: handle promisor objects in .gitmodules check
      fsck: check .gitmodules content
      fsck: call fsck_finish() after fscking objects
      unpack-objects: call fsck_finish() after fscking objects
      index-pack: check .gitmodules files with --strict
      fsck: complain when .gitmodules is a symlink

Johannes Schindelin (68):
      git_config_set: fix off-by-two
      t1300: rename it to reflect that `repo-config` was deprecated
      t1300: demonstrate that --replace-all can "invent" newlines
      config --replace-all: avoid extra line breaks
      t1300: avoid relying on a bug
      t1300: remove unreasonable expectation from TODO
      t5404: relax overzealous test
      t1300: add a few more hairy examples of sections becoming empty
      t1300: `--unset-all` can leave an empty section behind (bug)
      config: introduce an optional event stream while parsing
      config: avoid using the global variable `store`
      config_set_store: rename some fields for consistency
      git_config_set: do not use a state machine
      git_config_set: make use of the config parser's event stream
      git config --unset: remove empty sections (in the common case)
      git_config_set: reuse empty sections
      exec_cmd: provide a new-style RUNTIME_PREFIX helper for Windows
      mingw/msvc: use the new-style RUNTIME_PREFIX helper
      color: introduce support for colorizing stderr
      push: test to verify that push errors are colored
      config: document the settings to colorize push errors/hints
      gettext: avoid initialization if the locale dir is not present
      git_setup_gettext: plug memory leak
      sequencer: avoid using errno clobbered by rollback_lock_file()
      sequencer: make rearrange_squash() a bit more obvious
      sequencer: refactor how original todo list lines are accessed
      sequencer: offer helpful advice when a command was rescheduled
      sequencer: introduce new commands to reset the revision
      sequencer: introduce the `merge` command
      sequencer: fast-forward `merge` commands, if possible
      rebase-helper --make-script: introduce a flag to rebase merges
      rebase: introduce the --rebase-merges option
      sequencer: make refs generated by the `label` command worktree-local
      sequencer: handle post-rewrite for merge commands
      rebase --rebase-merges: avoid "empty merges"
      pull: accept --rebase=merges to recreate the branch topology
      rebase -i: introduce --rebase-merges=[no-]rebase-cousins
      rebase -i --rebase-merges: add a section to the man page
      argv_array: offer to split a string by whitespace
      commit: Let the callback of for_each_mergetag return on error
      replace: avoid using die() to indicate a bug
      tests: introduce test_unset_prereq, for debugging
      replace: "libify" create_graft() and callees
      replace: prepare create_graft() for converting graft files wholesale
      replace: introduce --convert-graft-file
      Add a test for `git replace --convert-graft-file`
      Deprecate support for .git/info/grafts
      filter-branch: stop suggesting to use grafts
      technical/shallow: stop referring to grafts
      technical/shallow: describe why shallow cannot use replace refs
      Remove obsolete script to convert grafts to replace refs
      rebase -i: demonstrate bugs with fixup!/squash! commit messages
      rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
      sequencer: always commit without editing when asked for
      rebase --skip: clean up commit message after a failed fixup/squash
      sequencer: extract helper to update active_cache_tree
      sequencer: learn about the special "fake root commit" handling
      rebase -i --root: let the sequencer handle even the initial part
      sequencer: allow introducing new root commits
      rebase --rebase-merges: a "merge" into a new root is a fast-forward
      rebase --rebase-merges: root commits can be cousins, too
      test-tool: help verifying BUG() code paths
      run-command: use BUG() to report bugs, not die()
      Replace all die("BUG: ...") calls by BUG() ones
      Convert remaining die*(BUG) messages
      config: a user-provided invalid section is not a BUG
      is_ntfs_dotgit: match other .git files
      is_{hfs,ntfs}_dotgitmodules: add tests

Johannes Sixt (2):
      sequencer: reset the committer date before commits
      git: add -P as a short option for --no-pager

Jonathan Nieder (5):
      sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
      sha1_file: allow sha1_loose_object_info to handle arbitrary repositories
      Makefile: remove unused @@PERLLIBDIR@@ substitution variable
      Makefile: quote $INSTLIBDIR when passing it to sed
      packfile: add repository argument to packed_object_info

Jonathan Tan (4):
      grep: remove "repo" arg from non-supporting funcs
      upload-pack: fix error message typo
      upload-pack: read config when serving protocol v2
      {fetch,upload}-pack: support filter in protocol v2

Junio C Hamano (18):
      stash: fix nonsense pipeline
      The first batch for 2.18 cycle
      The second batch for 2.18
      The third batch for 2.18
      Revert "Merge branch 'en/rename-directory-detection'"
      gc: do not upcase error message shown with die()
      parseopt: handle malformed --expire arguments more nicely
      The fourth batch for 2.18
      The fifth batch for 2.18
      argv-array: return the pushed string from argv_push*()
      Git 2.13.7
      Git 2.14.4
      Git 2.15.2
      Git 2.16.4
      Git 2.17.1
      The sixth batch for 2.18
      The seventh batch for 2.18
      Git 2.18-rc0

Kaartic Sivaraam (1):
      branch --list: print useful info whilst interactive rebasing a detached HEAD

Lars Schneider (10):
      strbuf: remove unnecessary NUL assignment in xstrdup_tolower()
      strbuf: add xstrdup_toupper()
      strbuf: add a case insensitive starts_with()
      utf8: teach same_encoding() alternative UTF encoding names
      utf8: add function to detect prohibited UTF-16/32 BOM
      utf8: add function to detect a missing UTF-16/32 BOM
      convert: add 'working-tree-encoding' attribute
      convert: check for detectable errors in UTF encodings
      convert: add tracing for 'working-tree-encoding' attribute
      convert: add round trip check based on 'core.checkRoundtripEncoding'

Leif Middelschulte (1):
      merge-recursive: give notice when submodule commit gets fast-forwarded

Loganaden Velvindron (1):
      http: allow use of TLS 1.3

Lucas Werkmeister (1):
      daemon.c: fix condition for redirecting stderr

Luis Marsano (2):
      git-credential-netrc: adapt to test framework for git
      git-credential-netrc: accept gpg option

Martin Ågren (24):
      git-shortlog.txt: reorder usages
      shortlog: add usage-string for stdin-reading
      shortlog: disallow left-over arguments outside repo
      doc: convert \--option to --option
      doc: convert [\--] to [--]
      git-[short]log.txt: unify quoted standalone --
      git-submodule.txt: quote usage in monospace, drop backslash
      fast-export: fix regression skipping some merge-commits
      http-fetch: make `-a` standard behaviour
      walker: drop fields of `struct walker` which are always 1
      t/helper/test-write-cache: clean up lock-handling
      refs.c: do not die if locking fails in `write_pseudoref()`
      refs.c: do not die if locking fails in `delete_pseudoref()`
      lock_file: make function-local locks non-static
      lock_file: move static locks into functions
      refs.c: refer to "object ID", not "sha1", in error messages
      t1400: add tests around adding/deleting pseudorefs
      refs: handle zero oid for pseudorefs
      merge: setup `opts` later in `checkout_fast_forward()`
      config: free resources of `struct config_store_data`
      config: let `config_store_data_clear()` handle `value_regex`
      config: let `config_store_data_clear()` handle `key`
      regex: do not call `regfree()` if compilation fails
      unpack_trees_options: free messages when done

Michal Nazarewicz (1):
      send-email: simplify Gmail example in the documentation

Michele Locati (1):
      filter-branch: return 2 when nothing to rewrite

Nguyễn Thái Ngọc Duy (88):
      repository: initialize the_repository in main()
      repository.c: move env-related setup code back to environment.c
      repository.c: delete dead functions
      sha1_file.c: move delayed getenv(altdb) back to setup_git_env()
      repository: delete ignore_env member
      gc.txt: more details about what gc does
      worktree: delete dead code
      worktree prune: improve prune logic when worktree is moved
      repository.h: add comment and clarify repo_set_gitdir
      git.c: move cmd_struct declaration up
      git.c: add hidden option --list-parseopt-builtins
      completion: mention the oldest version we need to support
      completion: factor out _git_xxx calling code
      completion: add --option completion for most builtin commands
      completion: delete option-only completion commands
      completion: use __gitcomp_builtin in _git_ls_tree
      completion: use __gitcomp_builtin in _git_cherry
      packfile: keep prepare_packed_git() private
      t/helper: add an empty test-tool program
      t/helper: merge test-chmtime into test-tool
      t/helper: merge test-sha1 into test-tool
      t/helper: merge test-lazy-init-name-hash into test-tool
      t/helper: merge test-config into test-tool
      t/helper: merge test-ctype into test-tool
      t/helper: merge test-date into test-tool
      t/helper: merge (unused) test-delta into test-tool
      t/helper: merge test-drop-caches into test-tool
      t/helper: merge test-dump-cache-tree into test-tool
      t/helper: merge test-dump-split-index into test-tool
      t/helper: merge test-example-decorate into test-tool
      t/helper: merge test-genrandom into test-tool
      t/helper: merge test-hashmap into test-tool
      t/helper: merge test-index-version into test-tool
      t/helper: merge (unused) test-match-trees into test-tool
      t/helper: merge (unused) test-mergesort into test-tool
      t/helper: merge test-mktemp into test-tool
      t/helper: merge test-online-cpus into test-tool
      t/helper: merge test-path-utils into test-tool
      t/helper: merge test-prio-queue into test-tool
      t/helper: merge test-read-cache into test-tool
      t/helper: merge test-ref-store into test-tool
      t/helper: merge test-regex into test-tool
      t/helper: merge test-revision-walking into test-tool
      t/helper: merge test-run-command into test-tool
      t/helper: merge test-scrap-cache-tree into test-tool
      t/helper: merge test-sha1-array into test-tool
      t/helper: merge test-sigchain into test-tool
      t/helper: merge test-strcmp-offset into test-tool
      t/helper: merge test-string-list into test-tool
      t/helper: merge test-submodule-config into test-tool
      t/helper: merge test-subprocess into test-tool
      t/helper: merge test-urlmatch-normalization into test-tool
      t/helper: merge test-wildmatch into test-tool
      t/helper: merge test-write-cache into test-tool
      trace.c: export trace_setup_key
      read-cache.c: make $GIT_TEST_SPLIT_INDEX boolean
      pack-objects: a bit of document about struct object_entry
      pack-objects: turn type and in_pack_type to bitfields
      pack-objects: use bitfield for object_entry::dfs_state
      pack-objects: use bitfield for object_entry::depth
      pack-objects: move in_pack_pos out of struct object_entry
      pack-objects: move in_pack out of struct object_entry
      pack-objects: refer to delta objects by index instead of pointer
      pack-objects: shrink z_delta_size field in struct object_entry
      pack-objects: don't check size when the object is bad
      pack-objects: clarify the use of object_entry::size
      pack-objects: shrink size field in struct object_entry
      pack-objects: shrink delta_size field in struct object_entry
      pack-objects: reorder members to shrink struct object_entry
      ci: exercise the whole test suite with uncommon code in pack-objects
      t7700: have closing quote of a test at the beginning of line
      repack: add --keep-pack option
      gc: add --keep-largest-pack option
      gc: add gc.bigPackThreshold config
      gc: handle a corner case in gc.bigPackThreshold
      gc --auto: exclude base pack if not enough mem to "repack -ad"
      pack-objects: show some progress when counting kept objects
      connect.c: mark die_initial_contact() NORETURN
      Makefile: detect compiler and enable more warnings in DEVELOPER=1
      submodule--helper: don't print null in 'submodule status'
      doc: keep first level section header in upper case
      pack-objects: validation and documentation about unreachable options
      completion: fix misspelled config key aliasesfiletype
      repository: fix free problem with repo_clear(the_repository)
      pack-format.txt: more details on pack file format
      column: fix off-by-one default width
      commit.h: rearrange 'index' to shrink struct commit
      travis-ci: run gcc-8 on linux-gcc jobs

Olga Telezhnaya (6):
      ref-filter: add shortcut to work with strbufs
      ref-filter: start adding strbufs with errors
      ref-filter: add return value && strbuf to handlers
      ref-filter: change parsing function error handling
      ref-filter: add return value to parsers
      ref-filter: libify get_ref_atom_value()

Paul-Sebastian Ungureanu (2):
      parse-options: do not show usage upon invalid option value
      t/helper: 'test-chmtime (--get|-g)' to print only the mtime

Philip Oakley (1):
      Avoid multiple PREFIX definitions

Phillip Wood (7):
      rebase --root: stop assuming squash_onto is unset
      rebase -i --keep-empty: don't prune empty commits
      rebase: respect --no-keep-empty
      rebase: extend --signoff support
      rebase -p: error out if --signoff is given
      rebase --keep-empty: always use interactive rebase
      rebase --rebase-merges: add test for --keep-empty

Pratik Karki (1):
      test: avoid pipes in git related commands for test

Ramsay Jones (1):
      BUG_exit_code: fix sparse "symbol not declared" warning

René Scharfe (8):
      sha1_name: use bsearch_pack() in unique_in_pack()
      bisect: use oid_to_hex() for converting object_id hashes to hex strings
      run-command: use strbuf_addstr() for adding a string to a strbuf
      submodule: check for NULL return of get_submodule_ref_store()
      replace_object: use oidmap
      fast-export: avoid NULL pointer arithmetic
      t5512: run git fetch inside test
      fsmonitor: use internal argv_array of struct child_process

Ryan Dammrose (1):
      push: colorize errors

SZEDER Gábor (22):
      test_must_be_empty: simplify file existence check
      t9902-completion: add tests demonstrating issues with quoted pathnames
      completion: move __git_complete_index_file() next to its helpers
      completion: simplify prefix path component handling during path completion
      completion: support completing non-ASCII pathnames
      completion: improve handling quoted paths on the command line
      completion: let 'ls-files' and 'diff-index' filter matching paths
      completion: use 'awk' to strip trailing path components
      t9902-completion: ignore COMPREPLY element order in some tests
      completion: remove repeated dirnames with 'awk' during path completion
      completion: improve handling quoted paths in 'git ls-files's output
      completion: fill COMPREPLY directly when completing paths
      completion: reduce overhead of clearing cached --options
      docs/git-gc: fix minor rendering issue
      coccinelle: avoid wrong transformation suggestions from commit.cocci
      t6050-replace: don't disable stdin for the whole test script
      t5310-pack-bitmaps: make JGit tests work with GIT_TEST_SPLIT_INDEX
      t5516-fetch-push: fix 'push with dry-run' test
      t5516-fetch-push: fix broken &&-chain
      t7005-editor: get rid of the SPACES_IN_FILENAMES prereq
      completion: don't return with error from __gitcomp_file_direct()
      t9902-completion: exercise __git_complete_index_file() directly

Sergey Organov (1):
      glossary: substitute "ancestor" for "direct ancestor" in 'push' description.

Stefan Agner (1):
      send-email: avoid duplicate In-Reply-To/References

Stefan Beller (81):
      repository: introduce raw object store field
      object-store: migrate alternates struct and functions from cache.h
      object-store: move alt_odb_list and alt_odb_tail to object store
      object-store: free alt_odb_list
      object-store: move packed_git and packed_git_mru to object store
      object-store: close all packs upon clearing the object store
      pack: move prepare_packed_git_run_once to object store
      pack: move approximate object count to object store
      sha1_file: add raw_object_store argument to alt_odb_usable
      sha1_file: add repository argument to link_alt_odb_entry
      sha1_file: add repository argument to read_info_alternates
      sha1_file: add repository argument to link_alt_odb_entries
      sha1_file: add repository argument to prepare_alt_odb
      sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
      sha1_file: allow prepare_alt_odb to handle arbitrary repositories
      sha1_file: add repository argument to sha1_file_name
      sha1_file: add repository argument to stat_sha1_file
      sha1_file: add repository argument to open_sha1_file
      sha1_file: add repository argument to map_sha1_file_1
      sha1_file: add repository argument to map_sha1_file
      sha1_file: add repository argument to sha1_loose_object_info
      sha1_file: allow sha1_file_name to handle arbitrary repositories
      sha1_file: allow stat_sha1_file to handle arbitrary repositories
      sha1_file: allow open_sha1_file to handle arbitrary repositories
      sha1_file: allow map_sha1_file to handle arbitrary repositories
      packfile: allow prepare_packed_git_mru to handle arbitrary repositories
      packfile: allow rearrange_packed_git to handle arbitrary repositories
      packfile: allow install_packed_git to handle arbitrary repositories
      packfile: add repository argument to prepare_packed_git_one
      packfile: add repository argument to prepare_packed_git
      packfile: add repository argument to reprepare_packed_git
      packfile: allow prepare_packed_git_one to handle arbitrary repositories
      packfile: allow prepare_packed_git to handle arbitrary repositories
      packfile: allow reprepare_packed_git to handle arbitrary repositories
      packfile: add repository argument to find_pack_entry
      packfile: allow find_pack_entry to handle arbitrary repositories
      submodule.h: drop declaration of connect_work_tree_and_git_dir
      submodule-config: allow submodule_free to handle arbitrary repositories
      submodule-config: add repository argument to submodule_from_{name, path}
      submodule-config: remove submodule_from_cache
      submodule: fixup nested submodules after moving the submodule
      write_or_die.c: rename to use dashes in file name
      unicode_width.h: rename to use dash in file name
      exec_cmd: rename to use dash in file name
      sha1_name.c: rename to use dash in file name
      sha1_file.c: rename to use dash in file name
      replace_object.c: rename to use dash in file name
      replace-object: move replace_map to object store
      object-store: move lookup_replace_object to replace-object.h
      replace-object: eliminate replace objects prepared flag
      replace-object: check_replace_refs is safe in multi repo environment
      refs: add repository argument to get_main_ref_store
      refs: add repository argument to for_each_replace_ref
      replace-object: add repository argument to prepare_replace_object
      replace-object: add repository argument to do_lookup_replace_object
      replace-object: add repository argument to lookup_replace_object
      refs: store the main ref store inside the repository struct
      refs: allow for_each_replace_ref to handle arbitrary repositories
      replace-object: allow prepare_replace_object to handle arbitrary repositories
      replace-object: allow do_lookup_replace_object to handle arbitrary repositories
      replace-object: allow lookup_replace_object to handle arbitrary repositories
      worktree: accept -f as short for --force for removal
      builtin/blame: dim uninteresting metadata lines
      builtin/blame: highlight recently changed lines
      builtin/blame: add new coloring scheme config
      cache.h: add repository argument to oid_object_info_extended
      cache.h: add repository argument to oid_object_info
      packfile: add repository argument to retry_bad_packed_offset
      packfile: add repository argument to packed_to_object_type
      packfile: add repository argument to read_object
      packfile: add repository argument to unpack_entry
      packfile: add repository argument to cache_or_unpack_entry
      cache.h: allow oid_object_info to handle arbitrary repositories
      git-rebase--interactive: clarify arguments
      object.c: free replace map in raw_object_store_clear
      replace-object.c: remove the_repository from prepare_replace_object
      grep: handle corrupt index files early
      git-submodule.sh: try harder to fetch a submodule
      submodule.c: move submodule merging to merge-recursive.c
      merge-recursive: i18n submodule merge output and respect verbosity
      object.c: clear replace map before freeing it

Takuto Ikuta (1):
      fetch-pack.c: use oidset to check existence of loose object

Tao Qingyun (1):
      t1510-repo-setup.sh: remove useless mkdir

Taylor Blau (5):
      builtin/config.c: treat type specifiers singularly
      builtin/config.c: support `--type=<type>` as preferred alias for `--<type>`
      builtin/config: introduce `--default`
      config.c: introduce 'git_config_color' to parse ANSI colors
      builtin/config: introduce `color` type specifier

Thomas Gummerer (10):
      stash push: avoid printing errors
      stash push -u: don't create empty stash
      stash: drop superfluos pathspec parameter
      SubmittingPatches: mention the git contacts command
      completion: stop showing 'save' for stash by default
      completion: make stash -p and alias for stash push -p
      worktree: remove extra members from struct add_opts
      worktree: improve message when creating a new worktree
      worktree: factor out dwim_branch function
      worktree: teach "add" to check out existing branches

Todd Zullinger (1):
      doc/clone: update caption for GIT URLS cross-reference

Torsten Bögershausen (1):
      test: correct detection of UTF8_NFD_TO_NFC for APFS

Wink Saville (8):
      rebase-interactive: simplify pick_on_preserving_merges
      rebase: update invocation of rebase dot-sourced scripts
      rebase: reindent function git_rebase__interactive
      rebase: extract functions out of git_rebase__interactive
      rebase: add and use git_rebase__interactive__preserve_merges
      rebase: remove unused code paths from git_rebase__interactive
      rebase: remove unused code paths from git_rebase__interactive__preserve_merges
      rebase: remove merges_option and a blank line

Yuki Kokubun (1):
      filter-branch: fix errors caused by refs that point at non-committish

brian m. carlson (121):
      bulk-checkin: convert index_bulk_checkin to struct object_id
      builtin/write-tree: convert to struct object_id
      cache-tree: convert write_*_as_tree to object_id
      cache-tree: convert remnants to struct object_id
      resolve-undo: convert struct resolve_undo_info to object_id
      tree: convert read_tree_recursive to struct object_id
      ref-filter: convert grab_objectname to struct object_id
      strbuf: convert strbuf_add_unique_abbrev to use struct object_id
      wt-status: convert struct wt_status_state to object_id
      Convert find_unique_abbrev* to struct object_id
      http-walker: convert struct object_request to use struct object_id
      send-pack: convert remaining functions to struct object_id
      replace_object: convert struct replace_object to object_id
      builtin/mktag: convert to struct object_id
      archive: convert write_archive_entry_fn_t to object_id
      archive: convert sha1_file_to_archive to struct object_id
      builtin/index-pack: convert struct ref_delta_entry to object_id
      sha1_file: convert read_loose_object to use struct object_id
      sha1_file: convert check_sha1_signature to struct object_id
      streaming: convert open_istream to use struct object_id
      builtin/mktree: convert to struct object_id
      sha1_file: convert assert_sha1_type to object_id
      sha1_file: convert retry_bad_packed_offset to struct object_id
      packfile: convert unpack_entry to struct object_id
      Convert remaining callers of sha1_object_info_extended to object_id
      sha1_file: convert sha1_object_info* to object_id
      builtin/fmt-merge-msg: convert remaining code to object_id
      builtin/notes: convert static functions to object_id
      tree-walk: convert get_tree_entry_follow_symlinks internals to object_id
      streaming: convert istream internals to struct object_id
      tree-walk: convert tree entry functions to object_id
      sha1_file: convert read_object_with_reference to object_id
      sha1_file: convert read_sha1_file to struct object_id
      Convert lookup_replace_object to struct object_id
      sha1_file: introduce a constant for max header length
      convert: convert to struct object_id
      sha1_name: convert struct min_abbrev_data to object_id
      t1011: abstract away SHA-1-specific constants
      t1304: abstract away SHA-1-specific constants
      t1300: abstract away SHA-1-specific constants
      t1405: sort reflog entries in a hash-independent way
      t1411: abstract away SHA-1-specific constants
      t1507: abstract away SHA-1-specific constants
      t2020: abstract away SHA-1 specific constants
      t2101: modernize test style
      t2101: abstract away SHA-1-specific constants
      t2107: abstract away SHA-1-specific constants
      format-patch: make cover letters always text/plain
      cache: add a function to read an object ID from a buffer
      server-info: remove unused members from struct pack_info
      Remove unused member in struct object_context
      packfile: remove unused member from struct pack_entry
      packfile: convert has_sha1_pack to object_id
      sha1-file: convert freshen functions to object_id
      packfile: convert find_pack_entry to object_id
      packfile: abstract away hash constant values
      pack-objects: abstract away hash algorithm
      pack-redundant: abstract away hash algorithm
      tree-walk: avoid hard-coded 20 constant
      tree-walk: convert get_tree_entry_follow_symlinks to object_id
      fsck: convert static functions to struct object_id
      submodule-config: convert structures to object_id
      split-index: convert struct split_index to object_id
      Update struct index_state to use struct object_id
      pack-redundant: convert linked lists to use struct object_id
      index-pack: abstract away hash function constant
      commit: convert uses of get_sha1_hex to get_oid_hex
      dir: convert struct untracked_cache_dir to object_id
      http: eliminate hard-coded constants
      revision: replace use of hard-coded constants
      upload-pack: replace use of several hard-coded constants
      diff: specify abbreviation size in terms of the_hash_algo
      builtin/receive-pack: avoid hard-coded constants for push certs
      sha1-file: add functions for hex empty tree and blob OIDs
      builtin/am: convert uses of EMPTY_TREE_SHA1_BIN to the_hash_algo
      builtin/merge: switch tree functions to use object_id
      merge: convert empty tree constant to the_hash_algo
      sequencer: convert one use of EMPTY_TREE_SHA1_HEX
      submodule: convert several uses of EMPTY_TREE_SHA1_HEX
      wt-status: convert two uses of EMPTY_TREE_SHA1_HEX
      builtin/receive-pack: convert one use of EMPTY_TREE_SHA1_HEX
      builtin/reset: convert use of EMPTY_TREE_SHA1_BIN
      sha1_file: convert cached object code to struct object_id
      cache-tree: use is_empty_tree_oid
      sequencer: use the_hash_algo for empty tree object ID
      dir: use the_hash_algo for empty blob object ID
      sha1_file: only expose empty object constants through git_hash_algo
      Update shell scripts to compute empty tree object ID
      add--interactive: compute the empty tree value
      merge-one-file: compute empty blob object ID
      Documentation: use 8-space tabs with Asciidoctor
      Documentation: render revisions correctly under Asciidoctor
      mailmap: update brian m. carlson's email address
      t/test-lib: add an SHA1 prerequisite
      t/test-lib: introduce ZERO_OID
      t: switch $_z40 to $ZERO_OID
      t/test-lib: introduce OID_REGEX
      t: switch $_x40 to $OID_REGEX
      t0000: annotate with SHA1 prerequisite
      t1007: annotate with SHA1 prerequisite
      t1512: skip test if not using SHA-1
      t4044: skip test if not using SHA-1
      t: skip pack tests if not using SHA-1
      t2203: abstract away SHA-1-specific constants
      t3103: abstract away SHA-1-specific constants
      t3702: abstract away SHA-1-specific constants
      t3905: abstract away SHA-1-specific constants
      t4007: abstract away SHA-1-specific constants
      t4008: abstract away SHA-1-specific constants
      t4014: abstract away SHA-1-specific constants
      t4020: abstract away SHA-1-specific constants
      t4022: abstract away SHA-1-specific constants
      t4029: fix test indentation
      t4029: abstract away SHA-1-specific constants
      t4030: abstract away SHA-1-specific constants
      t/lib-diff-alternative: abstract away SHA-1-specific constants
      t4205: sort log output in a hash-independent way
      t4042: abstract away SHA-1-specific constants
      t4045: abstract away SHA-1-specific constants
      t4208: abstract away SHA-1-specific constants
      t5300: abstract away SHA-1-specific constants

Ævar Arnfjörð Bjarmason (21):
      configure: fix a regression in PCRE v1 detection
      configure: detect redundant --with-libpcre & --with-libpcre1
      Makefile: make USE_LIBPCRE=YesPlease mean v2, not v1
      Makefile: fix broken bindir_relative variable
      Makefile: add a gitexecdir_relative variable
      Makefile: optionally symlink libexec/git-core binaries to bin/git
      Remove contrib/examples/*
      doc hash-function-transition: clarify how older gits die on NewHash
      doc hash-function-transition: clarify what SHAttered means
      git-svn: avoid warning on undef readline()
      Makefile: add a DEVOPTS to suppress -Werror under DEVELOPER
      Makefile: add a DEVOPTS to get all of -Wextra
      git{,-blame}.el: remove old bitrotting Emacs code
      .gitattributes: add *.pl extension for Perl
      .gitattributes: use the "perl" differ for Perl
      .gitattributes: add a diff driver for Python
      sha1-name.c: remove stray newline
      sha1-array.h: align function arguments
      git-p4: change "commitish" typo to "committish"
      sha1-name.c: move around the collect_ambiguous() function
      get_short_oid: sort ambiguous objects by type, then SHA-1


^ permalink raw reply	[relevance 3%]

* Re: [PATCH 3/3] submodule--helper: plug mem leak in print_default_remote
  2018-05-30 17:03 ` [PATCH 3/3] submodule--helper: plug mem leak in print_default_remote Stefan Beller
@ 2018-05-31 12:07   ` Johannes Schindelin
  2018-05-31 18:55     ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Johannes Schindelin @ 2018-05-31 12:07 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git

Hi Stefan,

On Wed, 30 May 2018, Stefan Beller wrote:

> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
>  builtin/submodule--helper.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> index 7c3cd9dbeba..96024fee1b1 100644
> --- a/builtin/submodule--helper.c
> +++ b/builtin/submodule--helper.c
> @@ -63,6 +63,7 @@ static int print_default_remote(int argc, const char **argv, const char *prefix)
>  	if (remote)
>  		printf("%s\n", remote);
>  
> +	free(remote);

Makes sense.

Out of curiosity (and because a cover letter is missing): how did you
stumble over these? Coverity?

Ciao,
Dscho

>  	return 0;
>  }
>  
> -- 
> 2.17.1.1185.g55be947832-goog
> 
> 

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 3/3] submodule--helper: plug mem leak in print_default_remote
  2018-05-31 12:07   ` Johannes Schindelin
@ 2018-05-31 18:55     ` Stefan Beller
  2018-06-01  8:46       ` Johannes Schindelin
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-31 18:55 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Thu, May 31, 2018 at 5:07 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Stefan,
>
> On Wed, 30 May 2018, Stefan Beller wrote:
>
>> Signed-off-by: Stefan Beller <sbeller@google.com>
>> ---
>>  builtin/submodule--helper.c | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
>> index 7c3cd9dbeba..96024fee1b1 100644
>> --- a/builtin/submodule--helper.c
>> +++ b/builtin/submodule--helper.c
>> @@ -63,6 +63,7 @@ static int print_default_remote(int argc, const char **argv, const char *prefix)
>>       if (remote)
>>               printf("%s\n", remote);
>>
>> +     free(remote);
>
> Makes sense.
>
> Out of curiosity (and because a cover letter is missing): how did you
> stumble over these? Coverity?

Yes I found them on coverity as I wanted to find out how bad their
false positives are these days. So I looked at the most recent findings.

I somehow imagined that we could redefine the _INIT macros which
usually lead to false positives (just alloc&UNLEAK memory instead of
pointing them all at the same memory for _INIT), but that experiment
did not work out.

Thanks,
Stefan

^ permalink raw reply	[relevance 5%]

* Re: [PATCH v4 9/9] checkout & worktree: introduce checkout.defaultRemote
      [irrelevant]   ` <20180531195252.29173-10-avarab@gmail.com>
@ 2018-05-31 21:49     ` Stefan Beller
  2018-06-01  9:47       ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-05-31 21:49 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, Jeff King, Johannes Schindelin,
	Nguyễn Thái Ngọc Duy, Thomas Gummerer,
	Eric Sunshine

Hi Ævar,

Sorry for chiming in late. I have a couple of thoughts:

>     (
>         cd /tmp &&
>         rm -rf tbdiff &&
>         git clone git@github.com:trast/tbdiff.git &&
>         cd tbdiff &&
>         git branch -m topic &&
>         git checkout master
>     )
>
> That will output:
>
>     Branch 'master' set up to track remote branch 'master' from 'origin'.
>     Switched to a new branch 'master'

I thought master is already there after the clone operation and
you'd merely switch back to the local branch that was created at
clone time?

    $ git clone git@github.com:trast/tbdiff.git && cd tbdiff
    $ git branch
    * master
    $ cat .git/config
...
[branch "master"]
    remote = origin
    merge = refs/heads/master

But the observation is right, we get that message. When do we
do the setup for the master branch specifically?

>
> But as soon as a new remote is added (e.g. just to inspect something
> from someone else) the DWIMery goes away:
>
>     (
>         cd /tmp &&
>         rm -rf tbdiff &&
>         git clone git@github.com:trast/tbdiff.git &&
>         cd tbdiff &&
>         git branch -m topic &&
>         git remote add avar git@github.com:avar/tbdiff.git &&
>         git fetch avar &&
>         git checkout master
>     )
>
> Will output (without the advice output added earlier in this series):
>
>     error: pathspec 'master' did not match any file(s) known to git.
>
> The new checkout.defaultRemote config allows me to say that whenever
> that ambiguity comes up I'd like to prefer "origin", and it'll still
> work as though the only remote I had was "origin".
>
> Also adjust the advice.checkoutAmbiguousRemoteBranchName message to
> mention this new config setting to the user, the full output on my
> git.git is now (the last paragraph is new):
>
>     $ ./git --exec-path=$PWD checkout master
>     error: pathspec 'master' did not match any file(s) known to git.
>     hint: The argument 'master' matched more than one remote tracking branch.
>     hint: We found 26 remotes with a reference that matched. So we fell back
>     hint: on trying to resolve the argument as a path, but failed there too!
>     hint:
>     hint: Perhaps you meant fully qualify the branch name? E.g. origin/<name>

s/meant fully/meant to fully/
s/? E.g./?\nFor example/

>     hint: instead of <name>?

In builtin/submodule--helper.c there is get_default_remote() which also
hardcodes "origin". I think that is a safe thing to do.

>     hint:
>     hint: If you'd like to always have checkouts of 'master' prefer one remote,
>     hint: e.g. the 'origin' remote, consider setting checkout.defaultRemote=origin
>     hint: in your config. See the 'git-config' manual page for details.

his new setting elevates one remote over all others, which may
be enough for most setups and not confusing, too.
Consider the following:

    git clone https://kernel.googlesource.com/pub/scm/git/git && cd git
    git remote add gitster https://github.com/gitster/git
    git remote add interesting-patches https://github.com/avar/git
    git remote add my-github https://github.com/stefanbeller/git

    git checkout master

This probably means I want to have origin/master (from kernel.org)

    git checkout ab/checkout-implicit-remote

This probably wants to have it from gitster/ (as it is not found on kernel.org);
I am not sure if it would want to look at interesting-patches/ that mirrors
github, but probably if it were not to be found at gitster.

So maybe we rather want a setup to give a defined priority for
the search order:

  git config dwim.remoteSearchOrder origin gitster avar

Stepping back a bit, there is already an order in the config file
for the remotes, and that order is used for example for 'fetch --all'.

I wonder if we want to take that order? (Or are the days of hand
editing the config over and this is too arcane? We would need a
config command to re order remotes). Then we could just have a
boolean switch to use the config order on ambiguity.
Although you might want to have a different order for fetching
and looking for the right checkout.

> I considered splitting this into checkout.defaultRemote and
> worktree.defaultRemote, but it's probably less confusing to break our
> own rules that anything shared between config should live in core.*
> than have two config settings, and I couldn't come up with a short
> name under core.* that made sense (core.defaultRemoteForCheckout?).

  core.dwimRemote ? It's a bit cryptic, though.

> See also 70c9ac2f19 ("DWIM "git checkout frotz" to "git checkout -b
> frotz origin/frotz"", 2009-10-18) which introduced this DWIM feature
> to begin with, and 4e85333197 ("worktree: make add <path> <branch>
> dwim", 2017-11-26) which added it to git-worktree.

Stefan

^ permalink raw reply	[relevance 3%]

* Re: [PATCH 3/3] submodule--helper: plug mem leak in print_default_remote
  2018-05-31 18:55     ` Stefan Beller
@ 2018-06-01  8:46       ` Johannes Schindelin
  0 siblings, 0 replies; 200+ results
From: Johannes Schindelin @ 2018-06-01  8:46 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git

Hi Stefan,

On Thu, 31 May 2018, Stefan Beller wrote:

> On Thu, May 31, 2018 at 5:07 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > Hi Stefan,
> >
> > On Wed, 30 May 2018, Stefan Beller wrote:
> >
> >> Signed-off-by: Stefan Beller <sbeller@google.com>
> >> ---
> >>  builtin/submodule--helper.c | 1 +
> >>  1 file changed, 1 insertion(+)
> >>
> >> diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
> >> index 7c3cd9dbeba..96024fee1b1 100644
> >> --- a/builtin/submodule--helper.c
> >> +++ b/builtin/submodule--helper.c
> >> @@ -63,6 +63,7 @@ static int print_default_remote(int argc, const char **argv, const char *prefix)
> >>       if (remote)
> >>               printf("%s\n", remote);
> >>
> >> +     free(remote);
> >
> > Makes sense.
> >
> > Out of curiosity (and because a cover letter is missing): how did you
> > stumble over these? Coverity?
> 
> Yes I found them on coverity as I wanted to find out how bad their
> false positives are these days. So I looked at the most recent findings.
> 
> I somehow imagined that we could redefine the _INIT macros which
> usually lead to false positives (just alloc&UNLEAK memory instead of
> pointing them all at the same memory for _INIT), but that experiment
> did not work out.

Yes, those many, many, *many* false positives really drown out the benefit
of Coverity for me. It takes all the fun out of looking for quick bug
fixes.

Ciao,
Dscho

^ permalink raw reply	[relevance 5%]

* Re: [PATCH v4 9/9] checkout & worktree: introduce checkout.defaultRemote
  2018-05-31 21:49     ` [PATCH v4 9/9] checkout & worktree: introduce checkout.defaultRemote Stefan Beller
@ 2018-06-01  9:47       ` Ævar Arnfjörð Bjarmason
  0 siblings, 0 replies; 200+ results
From: Ævar Arnfjörð Bjarmason @ 2018-06-01  9:47 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, Junio C Hamano, Jeff King, Johannes Schindelin,
	Nguyễn Thái Ngọc Duy, Thomas Gummerer,
	Eric Sunshine


On Thu, May 31 2018, Stefan Beller wrote:

> Hi Ævar,
>
> Sorry for chiming in late. I have a couple of thoughts:
>
>>     (
>>         cd /tmp &&
>>         rm -rf tbdiff &&
>>         git clone git@github.com:trast/tbdiff.git &&
>>         cd tbdiff &&
>>         git branch -m topic &&
>>         git checkout master
>>     )
>>
>> That will output:
>>
>>     Branch 'master' set up to track remote branch 'master' from 'origin'.
>>     Switched to a new branch 'master'
>
> I thought master is already there after the clone operation and
> you'd merely switch back to the local branch that was created at
> clone time?
>
>     $ git clone git@github.com:trast/tbdiff.git && cd tbdiff
>     $ git branch
>     * master
>     $ cat .git/config
> ...
> [branch "master"]
>     remote = origin
>     merge = refs/heads/master
>
> But the observation is right, we get that message. When do we
> do the setup for the master branch specifically?

What you're missing is this part:

    git branch -m topic

I.e. we clone the repo, and have a "master" branch, we then rename
"master" to "topic", now there's no local master branch. Then we
checkout master either with only one remote or two.

>>
>> But as soon as a new remote is added (e.g. just to inspect something
>> from someone else) the DWIMery goes away:
>>
>>     (
>>         cd /tmp &&
>>         rm -rf tbdiff &&
>>         git clone git@github.com:trast/tbdiff.git &&
>>         cd tbdiff &&
>>         git branch -m topic &&
>>         git remote add avar git@github.com:avar/tbdiff.git &&
>>         git fetch avar &&
>>         git checkout master
>>     )
>>
>> Will output (without the advice output added earlier in this series):
>>
>>     error: pathspec 'master' did not match any file(s) known to git.
>>
>> The new checkout.defaultRemote config allows me to say that whenever
>> that ambiguity comes up I'd like to prefer "origin", and it'll still
>> work as though the only remote I had was "origin".
>>
>> Also adjust the advice.checkoutAmbiguousRemoteBranchName message to
>> mention this new config setting to the user, the full output on my
>> git.git is now (the last paragraph is new):
>>
>>     $ ./git --exec-path=$PWD checkout master
>>     error: pathspec 'master' did not match any file(s) known to git.
>>     hint: The argument 'master' matched more than one remote tracking branch.
>>     hint: We found 26 remotes with a reference that matched. So we fell back
>>     hint: on trying to resolve the argument as a path, but failed there too!
>>     hint:
>>     hint: Perhaps you meant fully qualify the branch name? E.g. origin/<name>
>
> s/meant fully/meant to fully/
> s/? E.g./?\nFor example/

Thanks, will fix.

>>     hint: instead of <name>?
>
> In builtin/submodule--helper.c there is get_default_remote() which also
> hardcodes "origin". I think that is a safe thing to do.
>
>>     hint:
>>     hint: If you'd like to always have checkouts of 'master' prefer one remote,
>>     hint: e.g. the 'origin' remote, consider setting checkout.defaultRemote=origin
>>     hint: in your config. See the 'git-config' manual page for details.
>
> his new setting elevates one remote over all others, which may
> be enough for most setups and not confusing, too.
> Consider the following:
>
>     git clone https://kernel.googlesource.com/pub/scm/git/git && cd git
>     git remote add gitster https://github.com/gitster/git
>     git remote add interesting-patches https://github.com/avar/git
>     git remote add my-github https://github.com/stefanbeller/git
>
>     git checkout master
>
> This probably means I want to have origin/master (from kernel.org)
>
>     git checkout ab/checkout-implicit-remote
>
> This probably wants to have it from gitster/ (as it is not found on kernel.org);
> I am not sure if it would want to look at interesting-patches/ that mirrors
> github, but probably if it were not to be found at gitster.
>
> So maybe we rather want a setup to give a defined priority for
> the search order:
>
>   git config dwim.remoteSearchOrder origin gitster avar
>
> Stepping back a bit, there is already an order in the config file
> for the remotes, and that order is used for example for 'fetch --all'.
>
> I wonder if we want to take that order? (Or are the days of hand
> editing the config over and this is too arcane? We would need a
> config command to re order remotes). Then we could just have a
> boolean switch to use the config order on ambiguity.
> Although you might want to have a different order for fetching
> and looking for the right checkout.

I thought about this use-case, and if we want this in the future I think
the most straightforward way is not to invent some new search order
variable, but just make use of git config allowing multi-values, i.e.:

    [checkout]
        defaultRemote = origin
        defaultRemote = gitster

Although I'm not interested in implementing that now, and unlike just
having one special remote I don't think it's of interest to the vast
majority of git users.

>> I considered splitting this into checkout.defaultRemote and
>> worktree.defaultRemote, but it's probably less confusing to break our
>> own rules that anything shared between config should live in core.*
>> than have two config settings, and I couldn't come up with a short
>> name under core.* that made sense (core.defaultRemoteForCheckout?).
>
>   core.dwimRemote ? It's a bit cryptic, though.

Covered by Eric's reply in
<CAPig+cSk9Dt3ZLQRjWwpxqMyP3npu3KbEQxkNfjV5RxRtro82Q@mail.gmail.com>

>> See also 70c9ac2f19 ("DWIM "git checkout frotz" to "git checkout -b
>> frotz origin/frotz"", 2009-10-18) which introduced this DWIM feature
>> to begin with, and 4e85333197 ("worktree: make add <path> <branch>
>> dwim", 2017-11-26) which added it to git-worktree.

^ permalink raw reply	[relevance 2%]

* [ANNOUNCE] Git v2.18.0-rc1
@ 2018-06-04 13:53 Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-06-04 13:53 UTC (permalink / raw)
  To: git; +Cc: Linux Kernel, git-packagers

A release candidate Git v2.18.0-rc1 is now available for testing
at the usual places.  It is comprised of 842 non-merge commits
since v2.17.0, contributed by 65 people, 20 of which are new faces.

I plan to go offline for most of the remainder of the week, and then
tag (hopefully) the last -rc early next week, with any urgent
regression fix accumulated on the list during this week.  Happy bug
hunting ;-)

The tarballs are found at:

    https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the
'v2.18.0-rc1' tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.17.0 are as follows.
Welcome to the Git development community!

  Bill Ritcher, Birger Skogeng Pedersen, Casey Fitzpatrick,
  Dan Jacques, Drew DeVault, Eckhard S. Maaß, Erik E Brady,
  Florian Gamböck, Harald Nordgren, Leif Middelschulte,
  Loganaden Velvindron, Luis Marsano, Paul-Sebastian Ungureanu,
  Pedro Alvarez Piedehierro, Pratik Karki, Ryan Dammrose, Takuto
  Ikuta, Tao Qingyun, Wink Saville, and Yuki Kokubun.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Ævar Arnfjörð Bjarmason, Anders Kaseorg, Andreas Heiduk,
  Antonio Ospite, Beat Bolli, Ben Peart, Brandon Williams, brian
  m. carlson, Christian Couder, Christian Hesse, Clemens Buchacher,
  David Turner, Derrick Stolee, Elijah Newren, Eric Sunshine,
  Jameson Miller, Jeff King, Johannes Schindelin, Johannes Sixt,
  Jonathan Nieder, Jonathan Tan, Junio C Hamano, Kaartic Sivaraam,
  Lars Schneider, Lucas Werkmeister, Luke Diamand, Martin Ågren,
  Michal Nazarewicz, Michele Locati, Nguyễn Thái Ngọc Duy,
  Olga Telezhnaya, Orgad Shaneh, Philip Oakley, Phillip Wood,
  Ramsay Jones, René Scharfe, Robert P. J. Day, Sergey Organov,
  Stefan Agner, Stefan Beller, SZEDER Gábor, Taylor Blau, Thomas
  Gummerer, Todd Zullinger, and Torsten Bögershausen.

----------------------------------------------------------------

Git 2.18 Release Notes (draft)
==============================

Updates since v2.17
-------------------

UI, Workflows & Features

 * Rename detection logic in "diff" family that is used in "merge" has
   learned to guess when all of x/a, x/b and x/c have moved to z/a,
   z/b and z/c, it is likely that x/d added in the meantime would also
   want to move to z/d by taking the hint that the entire directory
   'x' moved to 'z'.  A bug causing dirty files involved in a rename
   to be overwritten during merge has also been fixed as part of this
   work.  Incidentally, this also avoids updating a file in the
   working tree after a (non-trivial) merge whose result matches what
   our side originally had.

 * "git filter-branch" learned to use a different exit code to allow
   the callers to tell the case where there was no new commits to
   rewrite from other error cases.

 * When built with more recent cURL, GIT_SSL_VERSION can now specify
   "tlsv1.3" as its value.

 * "git gui" learned that "~/.ssh/id_ecdsa.pub" and
   "~/.ssh/id_ed25519.pub" are also possible SSH key files.
   (merge 2e2f0288ef bb/git-gui-ssh-key-files later to maint).

 * "git gui" performs commit upon CTRL/CMD+ENTER but the
   CTRL/CMD+KP_ENTER (i.e. enter key on the numpad) did not have the
   same key binding.  It now does.
   (merge 28a1d94a06 bp/git-gui-bind-kp-enter later to maint).

 * "git gui" has been taught to work with old versions of tk (like
   8.5.7) that do not support "ttk::style theme use" as a way to query
   the current theme.
   (merge 4891961105 cb/git-gui-ttk-style later to maint).

 * "git rebase" has learned to honor "--signoff" option when using
   backends other than "am" (but not "--preserve-merges").

 * "git branch --list" during an interrupted "rebase -i" now lets
   users distinguish the case where a detached HEAD is being rebased
   and a normal branch is being rebased.

 * "git mergetools" learned talking to guiffy.

 * The scripts in contrib/emacs/ have outlived their usefulness and
   have been replaced with a stub that errors out and tells the user
   there are replacements.

 * The new "checkout-encoding" attribute can ask Git to convert the
   contents to the specified encoding when checking out to the working
   tree (and the other way around when checking in).

 * The "git config" command uses separate options e.g. "--int",
   "--bool", etc. to specify what type the caller wants the value to
   be interpreted as.  A new "--type=<typename>" option has been
   introduced, which would make it cleaner to define new types.

 * "git config --get" learned the "--default" option, to help the
   calling script.  Building on top of the above changes, the
   "git config" learns "--type=color" type.  Taken together, you can
   do things like "git config --get foo.color --default blue" and get
   the ANSI color sequence for the color given to foo.color variable,
   or "blue" if the variable does not exist.

 * "git ls-remote" learned an option to allow sorting its output based
   on the refnames being shown.

 * The command line completion (in contrib/) has been taught that "git
   stash save" has been deprecated ("git stash push" is the preferred
   spelling in the new world) and does not offer it as a possible
   completion candidate when "git stash push" can be.

 * "git gc --prune=nonsense" spent long time repacking and then
   silently failed when underlying "git prune --expire=nonsense"
   failed to parse its command line.  This has been corrected.

 * Error messages from "git push" can be painted for more visibility.

 * "git http-fetch" (deprecated) had an optional and experimental
   "feature" to fetch only commits and/or trees, which nobody used.
   This has been removed.

 * The functionality of "$GIT_DIR/info/grafts" has been superseded by
   the "refs/replace/" mechanism for some time now, but the internal
   code had support for it in many places, which has been cleaned up
   in order to drop support of the "grafts" mechanism.

 * "git worktree add" learned to check out an existing branch.

 * "git --no-pager cmd" did not have short-and-sweet single letter
   option. Now it does as "-P".
   (merge 7213c28818 js/no-pager-shorthand later to maint).

 * "git rebase" learned "--rebase-merges" to transplant the whole
   topology of commit graph elsewhere.

 * "git status" learned to pay attention to UI related diff
   configuration variables such as diff.renames.

 * The command line completion mechanism (in contrib/) learned to load
   custom completion file for "git $command" where $command is a
   custom "git-$command" that the end user has on the $PATH when using
   newer version of bash.

 * "git send-email" can sometimes offer confirmation dialog "Send this
   email?" with choices 'Yes', 'No', 'Quit', and 'All'.  A new action
   'Edit' has been added to this dialog's choice.

 * With merge.renames configuration set to false, the recursive merge
   strategy can be told not to spend cycles trying to find renamed
   paths and merge them accordingly.

 * "git status" learned to honor a new status.renames configuration to
   skip rename detection, which could be useful for those who want to
   do so without disabling the default rename detection done by the
   "git diff" command.

 * Command line completion (in contrib/) learned to complete pathnames
   for various commands better.

 * "git blame" learns to unhighlight uninteresting metadata from the
   originating commit on lines that are the same as the previous one,
   and also paint lines in different colors depending on the age of
   the commit.

 * Transfer protocol v2 learned to support the partial clone.

 * When a short hexadecimal string is used to name an object but there
   are multiple objects that share the string as the prefix of their
   names, the code lists these ambiguous candidates in a help message.
   These object names are now sorted according to their types for
   easier eyeballing.

 * "git fetch $there $refspec" that talks over protocol v2 can take
   advantage of server-side ref filtering; the code has been extended
   so that this mechanism triggers also when fetching with configured
   refspec.

 * Our HTTP client code used to advertise that we accept gzip encoding
   from the other side; instead, just let cURL library to advertise
   and negotiate the best one.

 * "git p4" learned to "unshelve" shelved commit from P4.
   (merge 123f631761 ld/p4-unshelve later to maint).


Performance, Internal Implementation, Development Support etc.

 * A "git fetch" from a repository with insane number of refs into a
   repository that is already up-to-date still wasted too many cycles
   making many lstat(2) calls to see if these objects at the tips
   exist as loose objects locally.  These lstat(2) calls are optimized
   away by enumerating all loose objects beforehand.
   It is unknown if the new strategy negatively affects existing use
   cases, fetching into a repository with many loose objects from a
   repository with small number of refs.

 * Git can be built to use either v1 or v2 of the PCRE library, and so
   far, the build-time configuration USE_LIBPCRE=YesPlease instructed
   the build procedure to use v1, but now it means v2.  USE_LIBPCRE1
   and USE_LIBPCRE2 can be used to explicitly choose which version to
   use, as before.

 * The build procedure learned to optionally use symbolic links
   (instead of hardlinks and copies) to install "git-foo" for built-in
   commands, whose binaries are all identical.

 * Conversion from uchar[20] to struct object_id continues.

 * The way "git worktree prune" worked internally has been simplified,
   by assuming how "git worktree move" moves an existing worktree to a
   different place.

 * Code clean-up for the "repository" abstraction.
   (merge 00a3da2a13 nd/remove-ignore-env-field later to maint).

 * Code to find the length to uniquely abbreviate object names based
   on packfile content, which is a relatively recent addtion, has been
   optimized to use the same fan-out table.

 * The mechanism to use parse-options API to automate the command line
   completion continues to get extended and polished.

 * Copies of old scripted Porcelain commands in contrib/examples/ have
   been removed.

 * Some tests that rely on the exact hardcoded values of object names
   have been updated in preparation for hash function migration.

 * Perf-test update.

 * Test helper update.

 * The effort continues to refactor the internal global data structure
   to make it possible to open multiple repositories, work with and
   then close them,

 * Small test-helper programs have been consolidated into a single
   binary.

 * API clean-up around ref-filter code.

 * Shell completion (in contrib) that gives list of paths have been
   optimized somewhat.

 * The index file is updated to record the fsmonitor section after a
   full scan was made, to avoid wasting the effort that has already
   spent.

 * Performance measuring framework in t/perf learned to help bisecting
   performance regressions.

 * Some multi-word source filenames are being renamed to separate
   words with dashes instead of underscores.

 * An reusable "memory pool" implementation has been extracted from
   fast-import.c, which in turn has become the first user of the
   mem-pool API.

 * A build-time option has been added to allow Git to be told to refer
   to its associated files relative to the main binary, in the same
   way that has been possible on Windows for quite some time, for
   Linux, BSDs and Darwin.

 * Precompute and store information necessary for ancestry traversal
   in a separate file to optimize graph walking.

 * The effort to pass the repository in-core structure throughout the
   API continues.  This round deals with the code that implements the
   refs/replace/ mechanism.

 * The build procedure "make DEVELOPER=YesPlease" learned to enable a
   bit more warning options depending on the compiler used to help
   developers more.  There also is "make DEVOPTS=tokens" knob
   available now, for those who want to help fixing warnings we
   usually ignore, for example.

 * A new version of the transport protocol is being worked on.

 * The code to interface to GPG has been restructured somewhat to make
   it cleaner to integrate with other types of signature systems later.

 * The code has been taught to use the duplicated information stored
   in the commit-graph file to learn the tree object name for a commit
   to avoid opening and parsing the commit object when it makes sense
   to do so.

 * "git gc" in a large repository takes a lot of time as it considers
   to repack all objects into one pack by default.  The command has
   been taught to pretend as if the largest existing packfile is
   marked with ".keep" so that it is left untouched while objects in
   other packs and loose ones are repacked.

 * The transport protocol v2 is getting updated further.

 * The codepath around object-info API has been taught to take the
   repository object (which in turn tells the API which object store
   the objects are to be located).

 * "git pack-objects" needs to allocate tons of "struct object_entry"
   while doing its work, and shrinking its size helps the performance
   quite a bit.

 * The implementation of "git rebase -i --root" has been updated to use
   the sequencer machinery more.

 * Developer support update, by using BUG() macro instead of die() to
   mark codepaths that should not happen more clearly.

 * Developer support.  Use newer GCC on one of the builds done at
   TravisCI.org to get more warnings and errors diagnosed.

 * Conversion from uchar[20] to struct object_id continues.

 * By code restructuring of submodule merge in merge-recursive,
   informational messages from the codepath are now given using the
   same mechanism as other output, and honor the merge.verbosity
   configuration.  The code also learned to give a few new messages
   when a submodule three-way merge resolves cleanly when one side
   records a descendant of the commit chosen by the other side.

 * Avoid unchecked snprintf() to make future code auditing easier.
   (merge ac4896f007 jk/snprintf-truncation later to maint).

 * Many tests hardcode the raw object names, which would change once
   we migrate away from SHA-1.  While some of them must test against
   exact object names, most of them do not have to use hardcoded
   constants in the test.  The latter kind of tests have been updated
   to test the moral equivalent of the original without hardcoding the
   actual object names.

 * The list of commands with their various attributes were spread
   across a few places in the build procedure, but it now is getting a
   bit more consolidated to allow more automation.

 * Quite a many tests assumed that newly created refs are made as
   loose refs using the files backend, which have been updated to use
   proper plumbing like rev-parse and update-ref, to avoid breakage
   once we start using different ref backends.


Also contains various documentation updates and code clean-ups.


Fixes since v2.17
-----------------

 * "git shortlog cruft" aborted with a BUG message when run outside a
   Git repository.  The command has been taught to complain about
   extra and unwanted arguments on its command line instead in such a
   case.
   (merge 4aa0161e83 ma/shortlog-revparse later to maint).

 * "git stash push -u -- <pathspec>" gave an unnecessary and confusing
   error message when there was no tracked files that match the
   <pathspec>, which has been fixed.
   (merge 353278687e tg/stash-untracked-with-pathspec-fix later to maint).

 * "git tag --contains no-such-commit" gave a full list of options
   after giving an error message.
   (merge 3bb0923f06 ps/contains-id-error-message later to maint).

 * "diff-highlight" filter (in contrib/) learned to undertand "git log
   --graph" output better.
   (merge 4551fbba14 jk/diff-highlight-graph-fix later to maint).

 * when refs that do not point at committish are given, "git
   filter-branch" gave a misleading error messages.  This has been
   corrected.
   (merge f78ab355e7 yk/filter-branch-non-committish-refs later to maint).

 * "git submodule status" misbehaved on a submodule that has been
   removed from the working tree.
   (merge 74b6bda32f rs/status-with-removed-submodule later to maint).

 * When credential helper exits very quickly without reading its
   input, it used to cause Git to die with SIGPIPE, which has been
   fixed.
   (merge a0d51e8d0e eb/cred-helper-ignore-sigpipe later to maint).

 * "git rebase --keep-empty" still removed an empty commit if the
   other side contained an empty commit (due to the "does an
   equivalent patch exist already?" check), which has been corrected.
   (merge 3d946165e1 pw/rebase-keep-empty-fixes later to maint).

 * Some codepaths, including the refs API, get and keep relative
   paths, that go out of sync when the process does chdir(2).  The
   chdir-notify API is introduced to let these codepaths adjust these
   cached paths to the new current directory.
   (merge fb9c2d2703 jk/relative-directory-fix later to maint).

 * "cd sub/dir && git commit ../path" ought to record the changes to
   the file "sub/path", but this regressed long time ago.
   (merge 86238e07ef bw/commit-partial-from-subdirectory-fix later to maint).

 * Recent introduction of "--log-destination" option to "git daemon"
   did not work well when the daemon was run under "--inetd" mode.
   (merge e67d906d73 lw/daemon-log-destination later to maint).

 * Small fix to the autoconf build procedure.
   (merge 249482daf0 es/fread-reads-dir-autoconf-fix later to maint).

 * Fix an unexploitable (because the oversized contents are not under
   attacker's control) buffer overflow.
   (merge d8579accfa bp/fsmonitor-bufsize-fix later to maint).

 * Recent simplification of build procedure forgot a bit of tweak to
   the build procedure of contrib/mw-to-git/
   (merge d8698987f3 ab/simplify-perl-makefile later to maint).

 * Moving a submodule that itself has submodule in it with "git mv"
   forgot to make necessary adjustment to the nested sub-submodules;
   now the codepath learned to recurse into the submodules.

 * "git config --unset a.b", when "a.b" is the last variable in an
   otherwise empty section "a", left an empty section "a" behind, and
   worse yet, a subsequent "git config a.c value" did not reuse that
   empty shell and instead created a new one.  These have been
   (partially) corrected.
   (merge c71d8bb38a js/empty-config-section-fix later to maint).

 * "git worktree remove" learned that "-f" is a shorthand for
   "--force" option, just like for "git worktree add".
   (merge d228eea514 sb/worktree-remove-opt-force later to maint).

 * The completion script (in contrib/) learned to clear cached list of
   command line options upon dot-sourcing it again in a more efficient
   way.
   (merge 94408dc71c sg/completion-clear-cached later to maint).

 * "git svn" had a minor thinko/typo which has been fixed.
   (merge 51db271587 ab/git-svn-get-record-typofix later to maint).

 * During a "rebase -i" session, the code could give older timestamp
   to commits created by later "pick" than an earlier "reword", which
   has been corrected.
   (merge 12f7babd6b js/ident-date-fix later to maint).

 * "git submodule status" did not check the symbolic revision name it
   computed for the submodule HEAD is not the NULL, and threw it at
   printf routines, which has been corrected.
   (merge 0b5e2ea7cf nd/submodule-status-fix later to maint).

 * When fed input that already has In-Reply-To: and/or References:
   headers and told to add the same information, "git send-email"
   added these headers separately, instead of appending to an existing
   one, which is a violation of the RFC.  This has been corrected.
   (merge 256be1d3f0 sa/send-email-dedup-some-headers later to maint).

 * "git fast-export" had a regression in v2.15.0 era where it skipped
   some merge commits in certain cases, which has been corrected.
   (merge be011bbe00 ma/fast-export-skip-merge-fix later to maint).

 * The code did not propagate the terminal width to subprocesses via
   COLUMNS environment variable, which it now does.  This caused
   trouble to "git column" helper subprocess when "git tag --column=row"
   tried to list the existing tags on a display with non-default width.
   (merge b5d5a567fb nd/term-columns later to maint).

 * We learned that our source files with ".pl" and ".py" extensions
   are Perl and Python files respectively and changes to them are
   better viewed as such with appropriate diff drivers.
   (merge 7818b619e2 ab/perl-python-attrs later to maint).

 * "git rebase -i" sometimes left intermediate "# This is a
   combination of N commits" message meant for the human consumption
   inside an editor in the final result in certain corner cases, which
   has been fixed.
   (merge 15ef69314d js/rebase-i-clean-msg-after-fixup-continue later to maint).

 * A test to see if the filesystem normalizes UTF-8 filename has been
   updated to check what we need to know in a more direct way, i.e. a
   path created in NFC form can be accessed with NFD form (or vice
   versa) to cope with APFS as well as HFS.
   (merge 742ae10e35 tb/test-apfs-utf8-normalization later to maint).

 * "git format-patch --cover --attach" created a broken MIME multipart
   message for the cover letter, which has been fixed by keeping the
   cover letter as plain text file.
   (merge 50cd54ef4e bc/format-patch-cover-no-attach later to maint).

 * The split-index feature had a long-standing and dormant bug in
   certain use of the in-core merge machinery, which has been fixed.
   (merge 7db118303a en/unpack-trees-split-index-fix later to maint).

 * Asciidoctor gives a reasonable imitation for AsciiDoc, but does not
   render illustration in a literal block correctly when indented with
   HT by default. The problem is fixed by forcing 8-space tabs.
   (merge 379805051d bc/asciidoctor-tab-width later to maint).

 * Code clean-up to adjust to a more recent lockfile API convention that
   allows lockfile instances kept on the stack.
   (merge 0fa5a2ed8d ma/lockfile-cleanup later to maint).

 * the_repository->index is not a allocated piece of memory but
   repo_clear() indiscriminately attempted to free(3) it, which has
   been corrected.
   (merge 74373b5f10 nd/repo-clear-keep-the-index later to maint).

 * Code clean-up to avoid non-standard-conformant pointer arithmetic.
   (merge c112084af9 rs/no-null-ptr-arith-in-fast-export later to maint).

 * Code clean-up to turn history traversal more robust in a
   semi-corrupt repository.
   (merge 8702b30fd7 jk/unavailable-can-be-missing later to maint).

 * "git update-ref A B" is supposed to ensure that ref A does not yet
   exist when B is a NULL OID, but this check was not done correctly
   for pseudo-refs outside refs/ hierarchy, e.g. MERGE_HEAD.

 * "git submodule update" and "git submodule add" supported the
   "--reference" option to borrow objects from a neighbouring local
   repository like "git clone" does, but lacked the more recent
   invention "--dissociate".  Also "git submodule add" has been taught
   to take the "--progress" option.
   (merge a0ef29341a cf/submodule-progress-dissociate later to maint).

 * Update credential-netrc helper (in contrib/) to allow customizing
   the GPG used to decrypt the encrypted .netrc file.
   (merge 786ef50a23 lm/credential-netrc later to maint).

 * "git submodule update" attempts two different kinds of "git fetch"
   against the upstream repository to grab a commit bound at the
   submodule's path, but it incorrectly gave up if the first kind
   (i.e. a normal fetch) failed, making the second "last resort" one
   (i.e. fetching an exact commit object by object name) ineffective.
   This has been corrected.
   (merge e30d833671 sb/submodule-update-try-harder later to maint).

 * Error behaviour of "git grep" when it cannot read the index was
   inconsistent with other commands that uses the index, which has
   been corrected to error out early.
   (merge b2aa84c789 sb/grep-die-on-unreadable-index later to maint).

 * We used to call regfree() after regcomp() failed in some codepaths,
   which have been corrected.
   (merge 17154b1576 ma/regex-no-regfree-after-comp-fail later to maint).

 * The import-tars script (in contrib/) has been taught to handle
   tarballs with overly long paths that use PAX extended headers.
   (merge 12ecea46e3 pa/import-tars-long-names later to maint).

 * "git rev-parse Y..." etc. misbehaved when given endpoints were
   not committishes.
   (merge 0ed556d38f en/rev-parse-invalid-range later to maint).

 * "git pull -recurse-submodules --rebase", when the submodule
   repository's history did not have anything common between ours and
   the upstream's, failed to execute.  We need to fetch from them to
   continue even in such a case.
   (merge 4d36f88be7 jt/submodule-pull-recurse-rebase later to maint).

 * "git remote update" can take both a single remote nickname and a
   nickname for remote groups, but only one of them was documented.
   (merge a97447a42a nd/remote-update-doc later to maint).

 * Other minor doc, test and build updates and code cleanups.
   (merge 248f66ed8e nd/trace-with-env later to maint).
   (merge 14ced5562c ys/bisect-object-id-missing-conversion-fix later to maint).
   (merge 5988eb631a ab/doc-hash-brokenness later to maint).
   (merge a4d4e32a70 pk/test-avoid-pipe-hiding-exit-status later to maint).
   (merge 05e293c1ac jk/flockfile-stdio later to maint).
   (merge e9184b0789 jk/t5561-missing-curl later to maint).
   (merge b1801b85a3 nd/worktree-move later to maint).
   (merge bbd374dd20 ak/bisect-doc-typofix later to maint).
   (merge 4855f06fb3 mn/send-email-credential-doc later to maint).
   (merge 8523b1e355 en/doc-typoes later to maint).
   (merge 43b44ccfe7 js/t5404-path-fix later to maint).
   (merge decf711fc1 ps/test-chmtime-get later to maint).
   (merge 22d11a6e8e es/worktree-docs later to maint).
   (merge 92a5dbbc22 tg/use-git-contacts later to maint).
   (merge adc887221f tq/t1510 later to maint).
   (merge bed21a8ad6 sg/doc-gc-quote-mismatch-fix later to maint).
   (merge 73364e4f10 tz/doc-git-urls-reference later to maint).
   (merge cd1e606bad bc/mailmap-self later to maint).
   (merge f7997e3682 ao/config-api-doc later to maint).
   (merge ee930754d8 jk/apply-p-doc later to maint).
   (merge 011b648646 nd/pack-format-doc later to maint).
   (merge 87a6bb701a sg/t5310-jgit-bitmap-test later to maint).
   (merge f6b82970aa sg/t5516-fixes later to maint).
   (merge 4362da078e sg/t7005-spaces-in-filenames-cleanup later to maint).
   (merge 7d0ee47c11 js/test-unset-prereq later to maint).
   (merge 5356a3c354 ah/misc-doc-updates later to maint).
   (merge 92c4a7a129 nd/completion-aliasfiletype-typofix later to maint).
   (merge 58bd77b66a nd/pack-unreachable-objects-doc later to maint).
   (merge 4ed79d5203 sg/t6500-no-redirect-of-stdin later to maint).
   (merge 17b8a2d6cd jk/config-blob-sans-repo later to maint).
   (merge 590551ca2c rd/tag-doc-lightweight later to maint).
   (merge 44f560fc16 rd/init-typo later to maint).
   (merge f156a0934a rd/p4-doc-markup-env later to maint).
   (merge 2a00502b14 tg/doc-sec-list later to maint).

----------------------------------------------------------------

Changes since v2.17.0 are as follows:

Anders Kaseorg (1):
      Documentation/git-bisect.txt: git bisect term → git bisect terms

Andreas Heiduk (9):
      git-svn: search --authors-prog in PATH too
      git-svn: allow empty email-address using authors-prog and authors-file
      doc: improve formatting in githooks.txt
      doc: align 'diff --no-index' in text and synopsis
      doc: clarify ignore rules for git ls-files
      doc: add '-d' and '-o' for 'git push'
      git-svn: remove ''--add-author-from' for 'commit-diff'
      doc: add note about shell quoting to revision.txt
      doc: normalize [--options] to [options] in git-diff

Antonio Ospite (1):
      doc: fix config API documentation about config_with_options

Beat Bolli (1):
      git-gui: search for all current SSH key types

Ben Peart (7):
      fsmonitor: fix incorrect buffer size when printing version number
      fsmonitor: force index write after full scan
      test-drop-caches: simplify delay loading of NtSetSystemInformation
      merge: update documentation for {merge,diff}.renameLimit
      merge: add merge.renames config setting
      merge: pass aggressive when rename detection is turned off
      add status config and command line options for rename detection

Bill Ritcher (1):
      mergetools: add support for guiffy

Birger Skogeng Pedersen (1):
      git-gui: bind CTRL/CMD+numpad ENTER to do_commit

Brandon Williams (79):
      pkt-line: introduce packet_read_with_status
      pkt-line: allow peeking a packet line without consuming it
      pkt-line: add delim packet support
      upload-pack: convert to a builtin
      upload-pack: factor out processing lines
      transport: use get_refs_via_connect to get refs
      connect: convert get_remote_heads to use struct packet_reader
      connect: discover protocol version outside of get_remote_heads
      transport: store protocol version
      protocol: introduce enum protocol_version value protocol_v2
      test-pkt-line: introduce a packet-line test helper
      serve: introduce git-serve
      ls-refs: introduce ls-refs server command
      connect: request remote refs using v2
      transport: convert get_refs_list to take a list of ref prefixes
      transport: convert transport_get_remote_refs to take a list of ref prefixes
      ls-remote: pass ref prefixes when requesting a remote's refs
      fetch: pass ref prefixes when fetching
      push: pass ref prefixes when pushing
      upload-pack: introduce fetch server command
      fetch-pack: perform a fetch using v2
      fetch-pack: support shallow requests
      connect: refactor git_connect to only get the protocol version once
      connect: don't request v2 when pushing
      transport-helper: remove name parameter
      transport-helper: refactor process_connect_service
      transport-helper: introduce stateless-connect
      pkt-line: add packet_buf_write_len function
      remote-curl: create copy of the service name
      remote-curl: store the protocol version the server responded with
      http: allow providing extra headers for http requests
      http: don't always add Git-Protocol header
      http: eliminate "# service" line when using protocol v2
      remote-curl: implement stateless-connect command
      remote-curl: don't request v2 when pushing
      commit: allow partial commits with relative paths
      serve: introduce the server-option capability
      ls-remote: send server options when using protocol v2
      fetch: send server options when using protocol v2
      refspec: move refspec parsing logic into its own file
      refspec: rename struct refspec to struct refspec_item
      refspec: factor out parsing a single refspec
      refspec: introduce struct refspec
      refspec: convert valid_fetch_refspec to use parse_refspec
      submodule--helper: convert push_check to use struct refspec
      pull: convert get_tracking_branch to use refspec_item_init
      transport: convert transport_push to use struct refspec
      remote: convert check_push_refs to use struct refspec
      remote: convert match_push_refs to use struct refspec
      clone: convert cmd_clone to use refspec_item_init
      fast-export: convert to use struct refspec
      remote: convert push refspecs to struct refspec
      remote: convert fetch refspecs to struct refspec
      remote: remove add_prune_tags_to_fetch_refspec
      transport-helper: convert to use struct refspec
      fetch: convert fetch_one to use struct refspec
      fetch: convert refmap to use struct refspec
      refspec: remove the deprecated functions
      fetch: convert do_fetch to take a struct refspec
      fetch: convert get_ref_map to take a struct refspec
      fetch: convert prune_refs to take a struct refspec
      remote: convert get_stale_heads to take a struct refspec
      remote: convert apply_refspecs to take a struct refspec
      remote: convert query_refspecs to take a struct refspec
      remote: convert get_ref_match to take a struct refspec
      remote: convert match_explicit_refs to take a struct refspec
      push: check for errors earlier
      push: convert to use struct refspec
      transport: convert transport_push to take a struct refspec
      send-pack: store refspecs in a struct refspec
      transport: remove transport_verify_remote_names
      http-push: store refspecs in a struct refspec
      remote: convert match_push_refs to take a struct refspec
      remote: convert check_push_refs to take a struct refspec
      submodule: convert push_unpushed_submodules to take a struct refspec
      refspec: consolidate ref-prefix generation logic
      fetch: generate ref-prefixes when using a configured refspec
      remote-curl: accept all encodings supported by curl
      remote-curl: accept compressed responses with protocol v2

Casey Fitzpatrick (3):
      submodule: clean up substitutions in script
      submodule: add --progress option to add command
      submodule: add --dissociate option to add/update commands

Christian Couder (7):
      perf/aggregate: add display_dir()
      perf/aggregate: add --sort-by=regression option
      perf/run: add --subsection option
      t/perf: add scripts to bisect performance regressions
      perf/aggregate: use Getopt::Long for option parsing
      perf/bisect_run_script: disable codespeed
      t990X: use '.git/objects' as 'deep inside .git' path

Christian Hesse (2):
      perl: fix installing modules from contrib
      Makefile: mark perllibdir as a .PHONY target

Clemens Buchacher (2):
      git-gui: workaround ttk:style theme use
      completion: improve ls-files filter performance

Dan Jacques (3):
      Makefile: generate Perl header from template file
      Makefile: add Perl runtime prefix support
      exec_cmd: RUNTIME_PREFIX on some POSIX systems

David Turner (1):
      t: make many tests depend less on the refs being files

Derrick Stolee (20):
      packfile: define and use bsearch_pack()
      sha1_name: use bsearch_pack() for abbreviations
      csum-file: rename hashclose() to finalize_hashfile()
      csum-file: refactor finalize_hashfile() method
      commit-graph: add format document
      graph: add commit graph design document
      commit-graph: create git-commit-graph builtin
      commit-graph: implement write_commit_graph()
      commit-graph: implement git-commit-graph write
      commit-graph: implement git commit-graph read
      commit-graph: add core.commitGraph setting
      commit-graph: close under reachability
      commit: integrate commit graph with commit parsing
      commit-graph: read only from specific pack-indexes
      commit-graph: build graph from starting commits
      commit-graph: implement "--append" option
      treewide: rename tree to maybe_tree
      commit: create get_commit_tree() method
      treewide: replace maybe_tree with accessor methods
      commit-graph: lazy-load trees for commits

Drew DeVault (1):
      git-send-email: allow re-editing of message

Eckhard S. Maaß (1):
      wt-status: use settings from git_diff_ui_config

Elijah Newren (72):
      directory rename detection: basic testcases
      directory rename detection: directory splitting testcases
      directory rename detection: testcases to avoid taking detection too far
      directory rename detection: partially renamed directory testcase/discussion
      directory rename detection: files/directories in the way of some renames
      directory rename detection: testcases checking which side did the rename
      directory rename detection: more involved edge/corner testcases
      directory rename detection: testcases exploring possibly suboptimal merges
      directory rename detection: miscellaneous testcases to complete coverage
      directory rename detection: tests for handling overwriting untracked files
      directory rename detection: tests for handling overwriting dirty files
      merge-recursive: move the get_renames() function
      merge-recursive: introduce new functions to handle rename logic
      merge-recursive: fix leaks of allocated renames and diff_filepairs
      merge-recursive: make !o->detect_rename codepath more obvious
      merge-recursive: split out code for determining diff_filepairs
      merge-recursive: make a helper function for cleanup for handle_renames
      merge-recursive: add get_directory_renames()
      merge-recursive: check for directory level conflicts
      merge-recursive: add computation of collisions due to dir rename & merging
      merge-recursive: check for file level conflicts then get new name
      merge-recursive: when comparing files, don't include trees
      merge-recursive: apply necessary modifications for directory renames
      merge-recursive: avoid clobbering untracked files with directory renames
      merge-recursive: fix overwriting dirty files involved in renames
      merge-recursive: fix remaining directory rename + dirty overwrite cases
      directory rename detection: new testcases showcasing a pair of bugs
      merge-recursive: avoid spurious rename/rename conflict from dir renames
      merge-recursive: ensure we write updates for directory-renamed file
      Documentation: fix several one-character-off spelling errors
      Documentation: normalize spelling of 'normalised'
      directory rename detection: basic testcases
      directory rename detection: directory splitting testcases
      directory rename detection: testcases to avoid taking detection too far
      directory rename detection: partially renamed directory testcase/discussion
      directory rename detection: files/directories in the way of some renames
      directory rename detection: testcases checking which side did the rename
      directory rename detection: more involved edge/corner testcases
      directory rename detection: testcases exploring possibly suboptimal merges
      directory rename detection: miscellaneous testcases to complete coverage
      directory rename detection: tests for handling overwriting untracked files
      directory rename detection: tests for handling overwriting dirty files
      merge-recursive: move the get_renames() function
      merge-recursive: introduce new functions to handle rename logic
      merge-recursive: fix leaks of allocated renames and diff_filepairs
      merge-recursive: make !o->detect_rename codepath more obvious
      merge-recursive: split out code for determining diff_filepairs
      merge-recursive: make a helper function for cleanup for handle_renames
      Make running git under other debugger-like programs easy
      unpack_trees: fix breakage when o->src_index != o->dst_index
      merge-recursive: add get_directory_renames()
      merge-recursive: check for directory level conflicts
      merge-recursive: add computation of collisions due to dir rename & merging
      merge-recursive: check for file level conflicts then get new name
      merge-recursive: when comparing files, don't include trees
      merge-recursive: apply necessary modifications for directory renames
      merge-recursive: avoid clobbering untracked files with directory renames
      merge-recursive: fix overwriting dirty files involved in renames
      merge-recursive: fix remaining directory rename + dirty overwrite cases
      directory rename detection: new testcases showcasing a pair of bugs
      merge-recursive: avoid spurious rename/rename conflict from dir renames
      merge-recursive: improve add_cacheinfo error handling
      merge-recursive: move more is_dirty handling to merge_content
      merge-recursive: avoid triggering add_cacheinfo error with dirty mod
      t6046: testcases checking whether updates can be skipped in a merge
      merge-recursive: fix was_tracked() to quit lying with some renamed paths
      merge-recursive: fix remainder of was_dirty() to use original index
      merge-recursive: make "Auto-merging" comment show for other merges
      merge-recursive: fix check for skipability of working tree updates
      merge-recursive: provide pair of `unpack_trees_{start,finish}()`
      rev-parse: check lookup'ed commit references for NULL
      RelNotes: remove duplicate release note

Eric Sunshine (5):
      t3200: verify "branch --list" sanity when rebasing from detached HEAD
      t2028: tighten grep expression to make "move worktree" test more robust
      git-worktree.txt: recommend 'git worktree remove' over manual deletion
      git-worktree.txt: unify command-line prompt in example blocks
      configure.ac: fix botched FREAD_READS_DIRECTORIES check

Erik E Brady (1):
      credential: ignore SIGPIPE when writing to credential helpers

Florian Gamböck (1):
      completion: load completion file for external subcommand

Harald Nordgren (1):
      ls-remote: create '--sort' option

Jameson Miller (3):
      fast-import: rename mem_pool type to mp_block
      fast-import: introduce mem_pool type
      mem-pool: move reusable parts of memory pool into its own file

Jeff King (54):
      diff-highlight: correct test graph diagram
      diff-highlight: use test_tick in graph test
      diff-highlight: prefer "echo" to "cat" in tests
      diff-highlight: test interleaved parallel lines of history
      diff-highlight: test graphs with --color
      diff-highlight: use flush() helper consistently
      diff-highlight: detect --graph by indent
      set_git_dir: die when setenv() fails
      add chdir-notify API
      set_work_tree: use chdir_notify
      refs: use chdir_notify to update cached relative paths
      config: move flockfile() closer to unlocked functions
      t5561: drop curl stderr redirects
      t5561: skip tests if curl is not available
      ref-filter: use "struct object_id" consistently
      ref-filter: make ref_array_item allocation more consistent
      ref-filter: factor ref_array pushing into its own function
      t7004: fix mistaken tag name
      gpg-interface: handle bool user.signingkey
      gpg-interface: modernize function declarations
      gpg-interface: use size_t for signature buffer size
      gpg-interface: fix const-correctness of "eol" pointer
      gpg-interface: extract gpg line matching helper
      gpg-interface: find the last gpg signature line
      apply: clarify "-p" documentation
      pager: set COLUMNS to term_columns()
      mark_tree_contents_uninteresting(): drop missing object check
      mark_parents_uninteresting(): drop missing object check
      mark_parents_uninteresting(): replace list with stack
      mark_parents_uninteresting(): avoid most allocation
      get_main_ref_store: BUG() when outside a repository
      config: die when --blob is used outside a repository
      http: use strbufs instead of fixed buffers
      log_write_email_headers: use strbufs
      shorten_unambiguous_ref: use xsnprintf
      fmt_with_err: add a comment that truncation is OK
      submodule-config: verify submodule names as paths
      is_ntfs_dotgit: use a size_t for traversing string
      is_hfs_dotgit: match other .git files
      skip_prefix: add case-insensitive variant
      verify_path: drop clever fallthrough
      verify_dotfile: mention case-insensitivity in comment
      update-index: stat updated files earlier
      verify_path: disallow symlinks in .gitmodules
      index-pack: make fsck error message more specific
      fsck: simplify ".git" check
      fsck: actually fsck blob data
      fsck: detect gitmodules files
      fsck: handle promisor objects in .gitmodules check
      fsck: check .gitmodules content
      fsck: call fsck_finish() after fscking objects
      unpack-objects: call fsck_finish() after fscking objects
      index-pack: check .gitmodules files with --strict
      fsck: complain when .gitmodules is a symlink

Johannes Schindelin (68):
      git_config_set: fix off-by-two
      t1300: rename it to reflect that `repo-config` was deprecated
      t1300: demonstrate that --replace-all can "invent" newlines
      config --replace-all: avoid extra line breaks
      t1300: avoid relying on a bug
      t1300: remove unreasonable expectation from TODO
      t5404: relax overzealous test
      t1300: add a few more hairy examples of sections becoming empty
      t1300: `--unset-all` can leave an empty section behind (bug)
      config: introduce an optional event stream while parsing
      config: avoid using the global variable `store`
      config_set_store: rename some fields for consistency
      git_config_set: do not use a state machine
      git_config_set: make use of the config parser's event stream
      git config --unset: remove empty sections (in the common case)
      git_config_set: reuse empty sections
      exec_cmd: provide a new-style RUNTIME_PREFIX helper for Windows
      mingw/msvc: use the new-style RUNTIME_PREFIX helper
      color: introduce support for colorizing stderr
      push: test to verify that push errors are colored
      config: document the settings to colorize push errors/hints
      gettext: avoid initialization if the locale dir is not present
      git_setup_gettext: plug memory leak
      sequencer: avoid using errno clobbered by rollback_lock_file()
      sequencer: make rearrange_squash() a bit more obvious
      sequencer: refactor how original todo list lines are accessed
      sequencer: offer helpful advice when a command was rescheduled
      sequencer: introduce new commands to reset the revision
      sequencer: introduce the `merge` command
      sequencer: fast-forward `merge` commands, if possible
      rebase-helper --make-script: introduce a flag to rebase merges
      rebase: introduce the --rebase-merges option
      sequencer: make refs generated by the `label` command worktree-local
      sequencer: handle post-rewrite for merge commands
      rebase --rebase-merges: avoid "empty merges"
      pull: accept --rebase=merges to recreate the branch topology
      rebase -i: introduce --rebase-merges=[no-]rebase-cousins
      rebase -i --rebase-merges: add a section to the man page
      argv_array: offer to split a string by whitespace
      commit: Let the callback of for_each_mergetag return on error
      replace: avoid using die() to indicate a bug
      tests: introduce test_unset_prereq, for debugging
      replace: "libify" create_graft() and callees
      replace: prepare create_graft() for converting graft files wholesale
      replace: introduce --convert-graft-file
      Add a test for `git replace --convert-graft-file`
      Deprecate support for .git/info/grafts
      filter-branch: stop suggesting to use grafts
      technical/shallow: stop referring to grafts
      technical/shallow: describe why shallow cannot use replace refs
      Remove obsolete script to convert grafts to replace refs
      rebase -i: demonstrate bugs with fixup!/squash! commit messages
      rebase -i: Handle "combination of <n> commits" with GETTEXT_POISON
      sequencer: always commit without editing when asked for
      rebase --skip: clean up commit message after a failed fixup/squash
      sequencer: extract helper to update active_cache_tree
      sequencer: learn about the special "fake root commit" handling
      rebase -i --root: let the sequencer handle even the initial part
      sequencer: allow introducing new root commits
      rebase --rebase-merges: a "merge" into a new root is a fast-forward
      rebase --rebase-merges: root commits can be cousins, too
      test-tool: help verifying BUG() code paths
      run-command: use BUG() to report bugs, not die()
      Replace all die("BUG: ...") calls by BUG() ones
      Convert remaining die*(BUG) messages
      config: a user-provided invalid section is not a BUG
      is_ntfs_dotgit: match other .git files
      is_{hfs,ntfs}_dotgitmodules: add tests

Johannes Sixt (2):
      sequencer: reset the committer date before commits
      git: add -P as a short option for --no-pager

Jonathan Nieder (6):
      sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
      sha1_file: allow sha1_loose_object_info to handle arbitrary repositories
      Makefile: remove unused @@PERLLIBDIR@@ substitution variable
      Makefile: quote $INSTLIBDIR when passing it to sed
      packfile: add repository argument to packed_object_info
      fetch: do not pass ref-prefixes for fetch by exact SHA1

Jonathan Tan (5):
      grep: remove "repo" arg from non-supporting funcs
      upload-pack: fix error message typo
      upload-pack: read config when serving protocol v2
      {fetch,upload}-pack: support filter in protocol v2
      submodule: do not pass null OID to setup_revisions

Junio C Hamano (21):
      stash: fix nonsense pipeline
      The first batch for 2.18 cycle
      The second batch for 2.18
      The third batch for 2.18
      Revert "Merge branch 'en/rename-directory-detection'"
      gc: do not upcase error message shown with die()
      parseopt: handle malformed --expire arguments more nicely
      The fourth batch for 2.18
      The fifth batch for 2.18
      argv-array: return the pushed string from argv_push*()
      Git 2.13.7
      Git 2.14.4
      Git 2.15.2
      Git 2.16.4
      Git 2.17.1
      The sixth batch for 2.18
      The seventh batch for 2.18
      Git 2.18-rc0
      refspec-api: avoid uninitialized field in refspec item
      A bit more topics before -rc1
      Git 2.18-rc1

Kaartic Sivaraam (1):
      branch --list: print useful info whilst interactive rebasing a detached HEAD

Lars Schneider (10):
      strbuf: remove unnecessary NUL assignment in xstrdup_tolower()
      strbuf: add xstrdup_toupper()
      strbuf: add a case insensitive starts_with()
      utf8: teach same_encoding() alternative UTF encoding names
      utf8: add function to detect prohibited UTF-16/32 BOM
      utf8: add function to detect a missing UTF-16/32 BOM
      convert: add 'working-tree-encoding' attribute
      convert: check for detectable errors in UTF encodings
      convert: add tracing for 'working-tree-encoding' attribute
      convert: add round trip check based on 'core.checkRoundtripEncoding'

Leif Middelschulte (1):
      merge-recursive: give notice when submodule commit gets fast-forwarded

Loganaden Velvindron (1):
      http: allow use of TLS 1.3

Lucas Werkmeister (1):
      daemon.c: fix condition for redirecting stderr

Luis Marsano (2):
      git-credential-netrc: adapt to test framework for git
      git-credential-netrc: accept gpg option

Luke Diamand (1):
      git-p4: add unshelve command

Martin Ågren (24):
      git-shortlog.txt: reorder usages
      shortlog: add usage-string for stdin-reading
      shortlog: disallow left-over arguments outside repo
      doc: convert \--option to --option
      doc: convert [\--] to [--]
      git-[short]log.txt: unify quoted standalone --
      git-submodule.txt: quote usage in monospace, drop backslash
      fast-export: fix regression skipping some merge-commits
      http-fetch: make `-a` standard behaviour
      walker: drop fields of `struct walker` which are always 1
      t/helper/test-write-cache: clean up lock-handling
      refs.c: do not die if locking fails in `write_pseudoref()`
      refs.c: do not die if locking fails in `delete_pseudoref()`
      lock_file: make function-local locks non-static
      lock_file: move static locks into functions
      refs.c: refer to "object ID", not "sha1", in error messages
      t1400: add tests around adding/deleting pseudorefs
      refs: handle zero oid for pseudorefs
      merge: setup `opts` later in `checkout_fast_forward()`
      config: free resources of `struct config_store_data`
      config: let `config_store_data_clear()` handle `value_regex`
      config: let `config_store_data_clear()` handle `key`
      regex: do not call `regfree()` if compilation fails
      unpack_trees_options: free messages when done

Michal Nazarewicz (1):
      send-email: simplify Gmail example in the documentation

Michele Locati (1):
      filter-branch: return 2 when nothing to rewrite

Nguyễn Thái Ngọc Duy (108):
      repository: initialize the_repository in main()
      repository.c: move env-related setup code back to environment.c
      repository.c: delete dead functions
      sha1_file.c: move delayed getenv(altdb) back to setup_git_env()
      repository: delete ignore_env member
      gc.txt: more details about what gc does
      worktree: delete dead code
      worktree prune: improve prune logic when worktree is moved
      repository.h: add comment and clarify repo_set_gitdir
      git.c: move cmd_struct declaration up
      git.c: add hidden option --list-parseopt-builtins
      completion: mention the oldest version we need to support
      completion: factor out _git_xxx calling code
      completion: add --option completion for most builtin commands
      completion: delete option-only completion commands
      completion: use __gitcomp_builtin in _git_ls_tree
      completion: use __gitcomp_builtin in _git_cherry
      packfile: keep prepare_packed_git() private
      t/helper: add an empty test-tool program
      t/helper: merge test-chmtime into test-tool
      t/helper: merge test-sha1 into test-tool
      t/helper: merge test-lazy-init-name-hash into test-tool
      t/helper: merge test-config into test-tool
      t/helper: merge test-ctype into test-tool
      t/helper: merge test-date into test-tool
      t/helper: merge (unused) test-delta into test-tool
      t/helper: merge test-drop-caches into test-tool
      t/helper: merge test-dump-cache-tree into test-tool
      t/helper: merge test-dump-split-index into test-tool
      t/helper: merge test-example-decorate into test-tool
      t/helper: merge test-genrandom into test-tool
      t/helper: merge test-hashmap into test-tool
      t/helper: merge test-index-version into test-tool
      t/helper: merge (unused) test-match-trees into test-tool
      t/helper: merge (unused) test-mergesort into test-tool
      t/helper: merge test-mktemp into test-tool
      t/helper: merge test-online-cpus into test-tool
      t/helper: merge test-path-utils into test-tool
      t/helper: merge test-prio-queue into test-tool
      t/helper: merge test-read-cache into test-tool
      t/helper: merge test-ref-store into test-tool
      t/helper: merge test-regex into test-tool
      t/helper: merge test-revision-walking into test-tool
      t/helper: merge test-run-command into test-tool
      t/helper: merge test-scrap-cache-tree into test-tool
      t/helper: merge test-sha1-array into test-tool
      t/helper: merge test-sigchain into test-tool
      t/helper: merge test-strcmp-offset into test-tool
      t/helper: merge test-string-list into test-tool
      t/helper: merge test-submodule-config into test-tool
      t/helper: merge test-subprocess into test-tool
      t/helper: merge test-urlmatch-normalization into test-tool
      t/helper: merge test-wildmatch into test-tool
      t/helper: merge test-write-cache into test-tool
      trace.c: export trace_setup_key
      read-cache.c: make $GIT_TEST_SPLIT_INDEX boolean
      pack-objects: a bit of document about struct object_entry
      pack-objects: turn type and in_pack_type to bitfields
      pack-objects: use bitfield for object_entry::dfs_state
      pack-objects: use bitfield for object_entry::depth
      pack-objects: move in_pack_pos out of struct object_entry
      pack-objects: move in_pack out of struct object_entry
      pack-objects: refer to delta objects by index instead of pointer
      pack-objects: shrink z_delta_size field in struct object_entry
      pack-objects: don't check size when the object is bad
      pack-objects: clarify the use of object_entry::size
      pack-objects: shrink size field in struct object_entry
      pack-objects: shrink delta_size field in struct object_entry
      pack-objects: reorder members to shrink struct object_entry
      ci: exercise the whole test suite with uncommon code in pack-objects
      t7700: have closing quote of a test at the beginning of line
      repack: add --keep-pack option
      gc: add --keep-largest-pack option
      gc: add gc.bigPackThreshold config
      gc: handle a corner case in gc.bigPackThreshold
      gc --auto: exclude base pack if not enough mem to "repack -ad"
      pack-objects: show some progress when counting kept objects
      connect.c: mark die_initial_contact() NORETURN
      Makefile: detect compiler and enable more warnings in DEVELOPER=1
      submodule--helper: don't print null in 'submodule status'
      doc: keep first level section header in upper case
      pack-objects: validation and documentation about unreachable options
      completion: fix misspelled config key aliasesfiletype
      repository: fix free problem with repo_clear(the_repository)
      generate-cmds.sh: factor out synopsis extract code
      generate-cmds.sh: export all commands to command-list.h
      help: use command-list.h for common command list
      Remove common-cmds.h
      pack-format.txt: more details on pack file format
      column: fix off-by-one default width
      commit.h: rearrange 'index' to shrink struct commit
      git.c: convert --list-* to --list-cmds=*
      git --list-cmds: collect command list in a string_list
      completion: implement and use --list-cmds=main,others
      git: support --list-cmds=list-<category>
      help: add "-a --verbose" to list all commands with synopsis
      help: use command-list.txt for the source of guides
      command-list.txt: documentation and guide line
      completion: let git provide the completable command list
      completion: reduce completable command list
      Move declaration for alias.c to alias.h
      completion: add and use --list-cmds=nohelpers
      completion: add and use --list-cmds=alias
      completion: allow to customize the completable command list
      travis-ci: run gcc-8 on linux-gcc jobs
      Use OPT_SET_INT_F() for cmdline option specification
      remote.txt: update documentation for 'update' command
      remote: doc typofix

Olga Telezhnaya (6):
      ref-filter: add shortcut to work with strbufs
      ref-filter: start adding strbufs with errors
      ref-filter: add return value && strbuf to handlers
      ref-filter: change parsing function error handling
      ref-filter: add return value to parsers
      ref-filter: libify get_ref_atom_value()

Orgad Shaneh (1):
      git-rebase--interactive: fix copy-paste mistake

Paul-Sebastian Ungureanu (2):
      parse-options: do not show usage upon invalid option value
      t/helper: 'test-chmtime (--get|-g)' to print only the mtime

Pedro Alvarez Piedehierro (1):
      import-tars: read overlong names from pax extended header

Philip Oakley (1):
      Avoid multiple PREFIX definitions

Phillip Wood (7):
      rebase --root: stop assuming squash_onto is unset
      rebase -i --keep-empty: don't prune empty commits
      rebase: respect --no-keep-empty
      rebase: extend --signoff support
      rebase -p: error out if --signoff is given
      rebase --keep-empty: always use interactive rebase
      rebase --rebase-merges: add test for --keep-empty

Pratik Karki (1):
      test: avoid pipes in git related commands for test

Ramsay Jones (1):
      BUG_exit_code: fix sparse "symbol not declared" warning

René Scharfe (8):
      sha1_name: use bsearch_pack() in unique_in_pack()
      bisect: use oid_to_hex() for converting object_id hashes to hex strings
      run-command: use strbuf_addstr() for adding a string to a strbuf
      submodule: check for NULL return of get_submodule_ref_store()
      replace_object: use oidmap
      fast-export: avoid NULL pointer arithmetic
      t5512: run git fetch inside test
      fsmonitor: use internal argv_array of struct child_process

Robert P. J. Day (4):
      Use proper syntax for replaceables in command docs
      tag: clarify in the doc that a tag can refer to a non-commit object
      init: fix grammar in "templates not found" msg
      p4.txt: Use backquotes for variable names

Ryan Dammrose (1):
      push: colorize errors

SZEDER Gábor (22):
      test_must_be_empty: simplify file existence check
      t9902-completion: add tests demonstrating issues with quoted pathnames
      completion: move __git_complete_index_file() next to its helpers
      completion: simplify prefix path component handling during path completion
      completion: support completing non-ASCII pathnames
      completion: improve handling quoted paths on the command line
      completion: let 'ls-files' and 'diff-index' filter matching paths
      completion: use 'awk' to strip trailing path components
      t9902-completion: ignore COMPREPLY element order in some tests
      completion: remove repeated dirnames with 'awk' during path completion
      completion: improve handling quoted paths in 'git ls-files's output
      completion: fill COMPREPLY directly when completing paths
      completion: reduce overhead of clearing cached --options
      docs/git-gc: fix minor rendering issue
      coccinelle: avoid wrong transformation suggestions from commit.cocci
      t6050-replace: don't disable stdin for the whole test script
      t5310-pack-bitmaps: make JGit tests work with GIT_TEST_SPLIT_INDEX
      t5516-fetch-push: fix 'push with dry-run' test
      t5516-fetch-push: fix broken &&-chain
      t7005-editor: get rid of the SPACES_IN_FILENAMES prereq
      completion: don't return with error from __gitcomp_file_direct()
      t9902-completion: exercise __git_complete_index_file() directly

Sergey Organov (1):
      glossary: substitute "ancestor" for "direct ancestor" in 'push' description.

Stefan Agner (1):
      send-email: avoid duplicate In-Reply-To/References

Stefan Beller (81):
      repository: introduce raw object store field
      object-store: migrate alternates struct and functions from cache.h
      object-store: move alt_odb_list and alt_odb_tail to object store
      object-store: free alt_odb_list
      object-store: move packed_git and packed_git_mru to object store
      object-store: close all packs upon clearing the object store
      pack: move prepare_packed_git_run_once to object store
      pack: move approximate object count to object store
      sha1_file: add raw_object_store argument to alt_odb_usable
      sha1_file: add repository argument to link_alt_odb_entry
      sha1_file: add repository argument to read_info_alternates
      sha1_file: add repository argument to link_alt_odb_entries
      sha1_file: add repository argument to prepare_alt_odb
      sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
      sha1_file: allow prepare_alt_odb to handle arbitrary repositories
      sha1_file: add repository argument to sha1_file_name
      sha1_file: add repository argument to stat_sha1_file
      sha1_file: add repository argument to open_sha1_file
      sha1_file: add repository argument to map_sha1_file_1
      sha1_file: add repository argument to map_sha1_file
      sha1_file: add repository argument to sha1_loose_object_info
      sha1_file: allow sha1_file_name to handle arbitrary repositories
      sha1_file: allow stat_sha1_file to handle arbitrary repositories
      sha1_file: allow open_sha1_file to handle arbitrary repositories
      sha1_file: allow map_sha1_file to handle arbitrary repositories
      packfile: allow prepare_packed_git_mru to handle arbitrary repositories
      packfile: allow rearrange_packed_git to handle arbitrary repositories
      packfile: allow install_packed_git to handle arbitrary repositories
      packfile: add repository argument to prepare_packed_git_one
      packfile: add repository argument to prepare_packed_git
      packfile: add repository argument to reprepare_packed_git
      packfile: allow prepare_packed_git_one to handle arbitrary repositories
      packfile: allow prepare_packed_git to handle arbitrary repositories
      packfile: allow reprepare_packed_git to handle arbitrary repositories
      packfile: add repository argument to find_pack_entry
      packfile: allow find_pack_entry to handle arbitrary repositories
      submodule.h: drop declaration of connect_work_tree_and_git_dir
      submodule-config: allow submodule_free to handle arbitrary repositories
      submodule-config: add repository argument to submodule_from_{name, path}
      submodule-config: remove submodule_from_cache
      submodule: fixup nested submodules after moving the submodule
      write_or_die.c: rename to use dashes in file name
      unicode_width.h: rename to use dash in file name
      exec_cmd: rename to use dash in file name
      sha1_name.c: rename to use dash in file name
      sha1_file.c: rename to use dash in file name
      replace_object.c: rename to use dash in file name
      replace-object: move replace_map to object store
      object-store: move lookup_replace_object to replace-object.h
      replace-object: eliminate replace objects prepared flag
      replace-object: check_replace_refs is safe in multi repo environment
      refs: add repository argument to get_main_ref_store
      refs: add repository argument to for_each_replace_ref
      replace-object: add repository argument to prepare_replace_object
      replace-object: add repository argument to do_lookup_replace_object
      replace-object: add repository argument to lookup_replace_object
      refs: store the main ref store inside the repository struct
      refs: allow for_each_replace_ref to handle arbitrary repositories
      replace-object: allow prepare_replace_object to handle arbitrary repositories
      replace-object: allow do_lookup_replace_object to handle arbitrary repositories
      replace-object: allow lookup_replace_object to handle arbitrary repositories
      worktree: accept -f as short for --force for removal
      builtin/blame: dim uninteresting metadata lines
      builtin/blame: highlight recently changed lines
      builtin/blame: add new coloring scheme config
      cache.h: add repository argument to oid_object_info_extended
      cache.h: add repository argument to oid_object_info
      packfile: add repository argument to retry_bad_packed_offset
      packfile: add repository argument to packed_to_object_type
      packfile: add repository argument to read_object
      packfile: add repository argument to unpack_entry
      packfile: add repository argument to cache_or_unpack_entry
      cache.h: allow oid_object_info to handle arbitrary repositories
      git-rebase--interactive: clarify arguments
      object.c: free replace map in raw_object_store_clear
      replace-object.c: remove the_repository from prepare_replace_object
      grep: handle corrupt index files early
      git-submodule.sh: try harder to fetch a submodule
      submodule.c: move submodule merging to merge-recursive.c
      merge-recursive: i18n submodule merge output and respect verbosity
      object.c: clear replace map before freeing it

Takuto Ikuta (1):
      fetch-pack.c: use oidset to check existence of loose object

Tao Qingyun (1):
      t1510-repo-setup.sh: remove useless mkdir

Taylor Blau (5):
      builtin/config.c: treat type specifiers singularly
      builtin/config.c: support `--type=<type>` as preferred alias for `--<type>`
      builtin/config: introduce `--default`
      config.c: introduce 'git_config_color' to parse ANSI colors
      builtin/config: introduce `color` type specifier

Thomas Gummerer (12):
      stash push: avoid printing errors
      stash push -u: don't create empty stash
      stash: drop superfluos pathspec parameter
      SubmittingPatches: mention the git contacts command
      completion: stop showing 'save' for stash by default
      completion: make stash -p and alias for stash push -p
      worktree: remove extra members from struct add_opts
      worktree: improve message when creating a new worktree
      worktree: factor out dwim_branch function
      worktree: teach "add" to check out existing branches
      SubmittingPatches: replace numbered attributes with names
      note git-security@googlegroups.com in more places

Todd Zullinger (1):
      doc/clone: update caption for GIT URLS cross-reference

Torsten Bögershausen (1):
      test: correct detection of UTF8_NFD_TO_NFC for APFS

Wink Saville (8):
      rebase-interactive: simplify pick_on_preserving_merges
      rebase: update invocation of rebase dot-sourced scripts
      rebase: reindent function git_rebase__interactive
      rebase: extract functions out of git_rebase__interactive
      rebase: add and use git_rebase__interactive__preserve_merges
      rebase: remove unused code paths from git_rebase__interactive
      rebase: remove unused code paths from git_rebase__interactive__preserve_merges
      rebase: remove merges_option and a blank line

Yuki Kokubun (1):
      filter-branch: fix errors caused by refs that point at non-committish

brian m. carlson (123):
      bulk-checkin: convert index_bulk_checkin to struct object_id
      builtin/write-tree: convert to struct object_id
      cache-tree: convert write_*_as_tree to object_id
      cache-tree: convert remnants to struct object_id
      resolve-undo: convert struct resolve_undo_info to object_id
      tree: convert read_tree_recursive to struct object_id
      ref-filter: convert grab_objectname to struct object_id
      strbuf: convert strbuf_add_unique_abbrev to use struct object_id
      wt-status: convert struct wt_status_state to object_id
      Convert find_unique_abbrev* to struct object_id
      http-walker: convert struct object_request to use struct object_id
      send-pack: convert remaining functions to struct object_id
      replace_object: convert struct replace_object to object_id
      builtin/mktag: convert to struct object_id
      archive: convert write_archive_entry_fn_t to object_id
      archive: convert sha1_file_to_archive to struct object_id
      builtin/index-pack: convert struct ref_delta_entry to object_id
      sha1_file: convert read_loose_object to use struct object_id
      sha1_file: convert check_sha1_signature to struct object_id
      streaming: convert open_istream to use struct object_id
      builtin/mktree: convert to struct object_id
      sha1_file: convert assert_sha1_type to object_id
      sha1_file: convert retry_bad_packed_offset to struct object_id
      packfile: convert unpack_entry to struct object_id
      Convert remaining callers of sha1_object_info_extended to object_id
      sha1_file: convert sha1_object_info* to object_id
      builtin/fmt-merge-msg: convert remaining code to object_id
      builtin/notes: convert static functions to object_id
      tree-walk: convert get_tree_entry_follow_symlinks internals to object_id
      streaming: convert istream internals to struct object_id
      tree-walk: convert tree entry functions to object_id
      sha1_file: convert read_object_with_reference to object_id
      sha1_file: convert read_sha1_file to struct object_id
      Convert lookup_replace_object to struct object_id
      sha1_file: introduce a constant for max header length
      convert: convert to struct object_id
      sha1_name: convert struct min_abbrev_data to object_id
      t1011: abstract away SHA-1-specific constants
      t1304: abstract away SHA-1-specific constants
      t1300: abstract away SHA-1-specific constants
      t1405: sort reflog entries in a hash-independent way
      t1411: abstract away SHA-1-specific constants
      t1507: abstract away SHA-1-specific constants
      t2020: abstract away SHA-1 specific constants
      t2101: modernize test style
      t2101: abstract away SHA-1-specific constants
      t2107: abstract away SHA-1-specific constants
      format-patch: make cover letters always text/plain
      cache: add a function to read an object ID from a buffer
      server-info: remove unused members from struct pack_info
      Remove unused member in struct object_context
      packfile: remove unused member from struct pack_entry
      packfile: convert has_sha1_pack to object_id
      sha1-file: convert freshen functions to object_id
      packfile: convert find_pack_entry to object_id
      packfile: abstract away hash constant values
      pack-objects: abstract away hash algorithm
      pack-redundant: abstract away hash algorithm
      tree-walk: avoid hard-coded 20 constant
      tree-walk: convert get_tree_entry_follow_symlinks to object_id
      fsck: convert static functions to struct object_id
      submodule-config: convert structures to object_id
      split-index: convert struct split_index to object_id
      Update struct index_state to use struct object_id
      pack-redundant: convert linked lists to use struct object_id
      index-pack: abstract away hash function constant
      commit: convert uses of get_sha1_hex to get_oid_hex
      dir: convert struct untracked_cache_dir to object_id
      http: eliminate hard-coded constants
      revision: replace use of hard-coded constants
      upload-pack: replace use of several hard-coded constants
      diff: specify abbreviation size in terms of the_hash_algo
      builtin/receive-pack: avoid hard-coded constants for push certs
      sha1-file: add functions for hex empty tree and blob OIDs
      builtin/am: convert uses of EMPTY_TREE_SHA1_BIN to the_hash_algo
      builtin/merge: switch tree functions to use object_id
      merge: convert empty tree constant to the_hash_algo
      sequencer: convert one use of EMPTY_TREE_SHA1_HEX
      submodule: convert several uses of EMPTY_TREE_SHA1_HEX
      wt-status: convert two uses of EMPTY_TREE_SHA1_HEX
      builtin/receive-pack: convert one use of EMPTY_TREE_SHA1_HEX
      builtin/reset: convert use of EMPTY_TREE_SHA1_BIN
      sha1_file: convert cached object code to struct object_id
      cache-tree: use is_empty_tree_oid
      sequencer: use the_hash_algo for empty tree object ID
      dir: use the_hash_algo for empty blob object ID
      sha1_file: only expose empty object constants through git_hash_algo
      Update shell scripts to compute empty tree object ID
      add--interactive: compute the empty tree value
      merge-one-file: compute empty blob object ID
      Documentation: use 8-space tabs with Asciidoctor
      Documentation: render revisions correctly under Asciidoctor
      mailmap: update brian m. carlson's email address
      t/test-lib: add an SHA1 prerequisite
      t/test-lib: introduce ZERO_OID
      t: switch $_z40 to $ZERO_OID
      t/test-lib: introduce OID_REGEX
      t: switch $_x40 to $OID_REGEX
      t0000: annotate with SHA1 prerequisite
      t1007: annotate with SHA1 prerequisite
      t1512: skip test if not using SHA-1
      t4044: skip test if not using SHA-1
      t: skip pack tests if not using SHA-1
      t2203: abstract away SHA-1-specific constants
      t3103: abstract away SHA-1-specific constants
      t3702: abstract away SHA-1-specific constants
      t3905: abstract away SHA-1-specific constants
      t4007: abstract away SHA-1-specific constants
      t4008: abstract away SHA-1-specific constants
      t4014: abstract away SHA-1-specific constants
      t4020: abstract away SHA-1-specific constants
      t4022: abstract away SHA-1-specific constants
      t4029: fix test indentation
      t4029: abstract away SHA-1-specific constants
      t4030: abstract away SHA-1-specific constants
      t/lib-diff-alternative: abstract away SHA-1-specific constants
      t4205: sort log output in a hash-independent way
      t4042: abstract away SHA-1-specific constants
      t4045: abstract away SHA-1-specific constants
      t4208: abstract away SHA-1-specific constants
      t5300: abstract away SHA-1-specific constants
      sequencer: ensure labels that are object IDs are rewritten
      t3430: test clean-up

Ævar Arnfjörð Bjarmason (21):
      configure: fix a regression in PCRE v1 detection
      configure: detect redundant --with-libpcre & --with-libpcre1
      Makefile: make USE_LIBPCRE=YesPlease mean v2, not v1
      Makefile: fix broken bindir_relative variable
      Makefile: add a gitexecdir_relative variable
      Makefile: optionally symlink libexec/git-core binaries to bin/git
      Remove contrib/examples/*
      doc hash-function-transition: clarify how older gits die on NewHash
      doc hash-function-transition: clarify what SHAttered means
      git-svn: avoid warning on undef readline()
      Makefile: add a DEVOPTS to suppress -Werror under DEVELOPER
      Makefile: add a DEVOPTS to get all of -Wextra
      git{,-blame}.el: remove old bitrotting Emacs code
      .gitattributes: add *.pl extension for Perl
      .gitattributes: use the "perl" differ for Perl
      .gitattributes: add a diff driver for Python
      sha1-name.c: remove stray newline
      sha1-array.h: align function arguments
      git-p4: change "commitish" typo to "committish"
      sha1-name.c: move around the collect_ambiguous() function
      get_short_oid: sort ambiguous objects by type, then SHA-1


^ permalink raw reply	[relevance 3%]

* Re: [PATCH 03/23] midx: add midx builtin
      [irrelevant] ` <20180607140338.32440-4-dstolee@microsoft.com>
@ 2018-06-11 21:02   ` Stefan Beller
  2018-06-18 19:40     ` Derrick Stolee
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-06-11 21:02 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: git, Derrick Stolee, Ævar Arnfjörð Bjarmason,
	Jonathan Nieder, Jonathan Tan, Martin Fick

Hi Derrick,
On Thu, Jun 7, 2018 at 7:03 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> This new 'git midx' builtin will be the plumbing access for writing,
> reading, and checking multi-pack-index (MIDX) files. The initial
> implementation is a no-op.

Let's talk about the name for a second:

.idx files are written by git-index-pack or as part of
git-pack-objects (which just calls write_idx_file as part
of finish_tmp_packfile), and the name actually suggests
it writes the index files. I have a hard time understanding
what the git-midx command does[1].

With both commit graph as well as multi index we introduce
a command that is centered around that concept (similar to
git-remote or git-config that are centered around a concept,
that is closely resembled by a file), but for indexes for packs
it was integrated differently into Git. So I am not sure if I want
to suggest to integrate it into the packfile commands as that
doesn't really fit. But maybe we can have a name that is human
readable instead of the file suffix? Maybe

  git multi-pack-index ?

I suppose that eventually this command is not really used by
users as it will be used by other porcelain commands in the
background or even as part of repack/gc so I am not worried
about a long name, but I'd be more worried about understandability.

[1] While these names are not perfect for the layman, it is okay?
  I am sure you are aware of https://git-man-page-generator.lokaltog.net/


> new file mode 100644
> index 0000000000..2bd886f1a2
> --- /dev/null
> +++ b/Documentation/git-midx.txt
> @@ -0,0 +1,29 @@
> +git-midx(1)
> +============
> +
> +NAME
> +----
> +git-midx - Write and verify multi-pack-indexes (MIDX files).

The reading is done as part of all other commands.

> +
> +
> +SYNOPSIS
> +--------
> +[verse]
> +'git midx' [--object-dir <dir>]
> +
> +DESCRIPTION
> +-----------
> +Write or verify a MIDX file.
> +
> +OPTIONS
> +-------
> +
> +--object-dir <dir>::
> +       Use given directory for the location of Git objects. We check
> +       <dir>/packs/multi-pack-index for the current MIDX file, and
> +       <dir>/packs for the pack-files to index.
> +
> +

Maybe we could have a SEE ALSO section that points at
the explanation of multi index files?
(c.f. man git-submodule that has a  SEE ALSO
gitsubmodules(7), gitmodules(5) explaining concepts(7)
and the file(5))

But as this is plumbing and users should not need to worry about it
this is optional, I would think.

^ permalink raw reply	[relevance 4%]

* Re: BUG: submodule code prints '(null)'
      [irrelevant] ` <20180609110414.GA5273@duynguyen.home>
@ 2018-06-11 22:56   ` Stefan Beller
  2018-06-14 15:15     ` Heiko Voigt
  2018-06-12 17:35   ` Stefan Beller
  1 sibling, 1 reply; 200+ results
From: Stefan Beller @ 2018-06-11 22:56 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: git, Heiko Voigt, Kaartic Sivaraam

On Sat, Jun 9, 2018 at 4:04 AM Duy Nguyen <pclouds@gmail.com> wrote:
>
> On Tue, Jun 05, 2018 at 05:31:41PM +0200, Duy Nguyen wrote:
> > I do not know how to reproduce this (and didn't bother to look deeply
> > into it after I found it was not a trivial fix) but one of my "git
> > fetch" showed
> >
> > warning: Submodule in commit be2db96a6c506464525f588da59cade0cedddb5e
> > at path: '(null)' collides with a submodule named the same. Skipping
> > it.
>
> The problem is default_name_or_path() can return NULL when a submodule
> is not populated. The fix could simply be printing path instead of
> name (because we are talking about path in the commit message), like
> below.
>
> But I don't really understand c68f837576 (implement fetching of moved
> submodules - 2017-10-16), the commit that made this change, and not
> sure if we should be reporting name here or path. Heiko?
>
> diff --git a/submodule.c b/submodule.c
> index 939d6870ec..61c2177755 100644
> --- a/submodule.c
> +++ b/submodule.c
> @@ -745,7 +745,7 @@ static void collect_changed_submodules_cb(struct diff_queue_struct *q,

[Not in the context of this patch, but in the code right before the
context starts:]

            name = default_name_or_path(p->two->path);
            /* make sure name does not collide with existing one */
            submodule = submodule_from_name(the_repository, commit_oid, name);
            if (submodule) {

Currently I see 4 callers of default_name_or_path and the other 3 except this
one have checks against NULL in place, which is good.
However I think we have to guard this even more, and have to check
for !name before we call submodule_from_name.

It is technically ok to call submodule_from_name with a NULL name,
but it is semantically broken, see the comment in config_from that
is called from submodule_from_name:

    /*
     * If any parameter except the cache is a NULL pointer just
     * return the first submodule. Can be used to check whether
     * there are any submodules parsed.
     */
    if (!treeish_name || !key) {
        ...


>                                 warning("Submodule in commit %s at path: "
>                                         "'%s' collides with a submodule named "
>                                         "the same. Skipping it.",
> -                                       oid_to_hex(commit_oid), name);
> +                                       oid_to_hex(commit_oid), p->two->path);

This is correct for the error message, both in terms of not crashing as well
as correctness, we really need to report a *path* here and no the name_or_path,
which  default_name_or_path gives.

^ permalink raw reply	[relevance 8%]

* Re: BUG: submodule code prints '(null)'
      [irrelevant] ` <20180609110414.GA5273@duynguyen.home>
  2018-06-11 22:56   ` BUG: submodule code prints '(null)' Stefan Beller
@ 2018-06-12 17:35   ` Stefan Beller
  1 sibling, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-12 17:35 UTC (permalink / raw)
  To: Duy Nguyen; +Cc: git, Heiko Voigt, Kaartic Sivaraam

On Sat, Jun 9, 2018 at 4:04 AM Duy Nguyen <pclouds@gmail.com> wrote:
>
> On Tue, Jun 05, 2018 at 05:31:41PM +0200, Duy Nguyen wrote:
> > I do not know how to reproduce this (and didn't bother to look deeply
> > into it after I found it was not a trivial fix) but one of my "git
> > fetch" showed
> >
> > warning: Submodule in commit be2db96a6c506464525f588da59cade0cedddb5e
> > at path: '(null)' collides with a submodule named the same. Skipping
> > it.
>
> The problem is default_name_or_path() can return NULL when a submodule
> is not populated. The fix could simply be printing path instead of
> name (because we are talking about path in the commit message), like
> below.
>
> But I don't really understand c68f837576 (implement fetching of moved
> submodules - 2017-10-16), the commit that made this change, and not
> sure if we should be reporting name here or path. Heiko?

That change is quite interesting as I did not understand it at first
sight as well.
See https://public-inbox.org/git/20171016135827.GC12756@book.hvoigt.net/
and the follow ups, specifically
https://public-inbox.org/git/20171019181109.27792-2-sbeller@google.com/
that tries to clean up the code, but was ultimately dropped.

^ permalink raw reply	[relevance 5%]

* Re: What's cooking in git.git (Jun 2018, #03; Tue, 12)
      [irrelevant] <xmqqbmcf90uf.fsf@gitster-ct.c.googlers.com>
@ 2018-06-12 21:13 ` Stefan Beller
  2018-06-13 15:56   ` Junio C Hamano
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-06-12 21:13 UTC (permalink / raw)
  To: Junio C Hamano, Duy Nguyen; +Cc: git

> A tl;dr summary for -rc2 that hopefully should happen in 24 hours:
> the following four topics
>
>  jk/submodule-fsck-loose-fixup
>  sb/submodule-merge-in-merge-recursive
>  jk/index-pack-maint
>  sg/completion-zsh-workaround
>
> are planned to be merged in -rc2.

It would be nice to have
https://public-inbox.org/git/20180609110414.GA5273@duynguyen.home/
included as well, but as it is not yet a proper patch, I can
understand why we err on the cautious side.

^ permalink raw reply	[relevance 2%]

* [PATCH] t7400: encapsulate setup code in test_expect_success
@ 2018-06-12 21:25 Stefan Beller
  2018-06-12 21:36 ` Eric Sunshine
  0 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-06-12 21:25 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

When running t7400 in a shell you observe more output than expected:

    ...
    ok 8 - setup - hide init subdirectory
    ok 9 - setup - repository to add submodules to
    ok 10 - submodule add
    [master (root-commit) d79ce16] one
     Author: A U Thor <author@example.com>
     1 file changed, 1 insertion(+)
     create mode 100644 one.t
    ok 11 - redirected submodule add does not show progress
    ok 12 - redirected submodule add --progress does show progress
    ok 13 - submodule add to .gitignored path fails
    ...

Fix the output by encapsulating the setup code in test_expect_success

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 t/t7400-submodule-basic.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 1a33040d94d..c6e1f749639 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -126,8 +126,10 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
-test_create_repo parent &&
-test_commit -C parent one
+test_expect_success 'setup parent and one repository for further tests' '
+	test_create_repo parent &&
+	test_commit -C parent one
+'
 
 test_expect_success 'redirected submodule add does not show progress' '
 	git -C addtest submodule add "file://$submodurl/parent" submod-redirected \
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 19%]

* Re: [PATCH] t7400: encapsulate setup code in test_expect_success
  2018-06-12 21:25 [PATCH] t7400: encapsulate setup code in test_expect_success Stefan Beller
@ 2018-06-12 21:36 ` Eric Sunshine
  2018-06-18 23:41   ` Stefan Beller
  0 siblings, 1 reply; 200+ results
From: Eric Sunshine @ 2018-06-12 21:36 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Git List

On Tue, Jun 12, 2018 at 5:25 PM, Stefan Beller <sbeller@google.com> wrote:
> When running t7400 in a shell you observe more output than expected:
>     ...
>     ok 10 - submodule add
>     [master (root-commit) d79ce16] one
>      Author: A U Thor <author@example.com>
>      1 file changed, 1 insertion(+)
>      create mode 100644 one.t
>     ok 11 - redirected submodule add does not show progress
>     ...
> Fix the output by encapsulating the setup code in test_expect_success
>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> @@ -126,8 +126,10 @@ test_expect_success 'submodule add' '
> -test_create_repo parent &&
> -test_commit -C parent one
> +test_expect_success 'setup parent and one repository for further tests' '

Nit: "for further tests" is implied for actions performed by a "setup"
function, so a bit redundant to say so.

> +       test_create_repo parent &&
> +       test_commit -C parent one
> +'

^ permalink raw reply	[relevance 2%]

* [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree
@ 2018-06-12 23:58 Stefan Beller
  2018-06-12 23:58 ` [PATCH 1/3] submodule: unset core.worktree if no working tree is present Stefan Beller
                   ` (4 more replies)
  0 siblings, 5 replies; 200+ results
From: Stefan Beller @ 2018-06-12 23:58 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

The first patch teaches checkout/reset (with --recurse-submodules) to unset
the core.worktree config when the new state of the superprojects working tree
doesn't contain the submodules working tree.

The last patch is teaching "git submodule deinit" to unset the core.worktree
setting as well. It turned out this one is tricky, as for that we also
have to set it in the counter part, such as "submodule update".

Thanks,
Stefan

Stefan Beller (3):
  submodule: unset core.worktree if no working tree is present
  submodule: ensure core.worktree is set after update
  submodule deinit: unset core.worktree

 builtin/submodule--helper.c | 26 ++++++++++++++++++++++++++
 git-submodule.sh            |  5 +++++
 submodule.c                 | 14 ++++++++++++++
 submodule.h                 |  2 ++
 t/lib-submodule-update.sh   |  5 +++--
 t/t7400-submodule-basic.sh  |  5 +++++
 6 files changed, 55 insertions(+), 2 deletions(-)

-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 11%]

* [PATCH 1/3] submodule: unset core.worktree if no working tree is present
  2018-06-12 23:58 [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree Stefan Beller
@ 2018-06-12 23:58 ` Stefan Beller
  2018-06-12 23:58 ` [PATCH 2/3] submodule: ensure core.worktree is set after update Stefan Beller
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-12 23:58 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

When a submodules work tree is removed, we should unset its core.worktree
setting as the worktree is no longer present. This is not just in line
with the conceptual view of submodules, but it fixes an inconvenience
for looking at submodules that are not checked out:

    git clone --recurse-submodules git://github.com/git/git && cd git &&
    git checkout --recurse-submodules v2.13.0
    git -C .git/modules/sha1collisiondetection log
    fatal: cannot chdir to '../../../sha1collisiondetection': \
        No such file or directory

With this patch applied, the final call to git log works instead of dying
in its setup, as the checkout will unset the core.worktree setting such
that following log will be run in a bare repository.

This patch covers all commands that are in the unpack machinery, i.e.
checkout, read-tree, reset. A follow up patch will address
"git submodule deinit", which will also make use of the new function
submodule_unset_core_worktree(), which is why we expose it in this patch.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c               | 14 ++++++++++++++
 submodule.h               |  2 ++
 t/lib-submodule-update.sh |  3 ++-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/submodule.c b/submodule.c
index 939d6870ecd..e127c074b04 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1532,6 +1532,18 @@ int bad_to_remove_submodule(const char *path, unsigned flags)
 	return ret;
 }
 
+void submodule_unset_core_worktree(const struct submodule *sub)
+{
+	char *config_path = xstrfmt("%s/modules/%s/config",
+				    get_git_common_dir(), sub->name);
+
+	if (git_config_set_in_file_gently(config_path, "core.worktree", NULL))
+		warning(_("Could not unset core.worktree setting in submodule '%s'"),
+			  sub->path);
+
+	free(config_path);
+}
+
 static const char *get_super_prefix_or_empty(void)
 {
 	const char *s = get_super_prefix();
@@ -1697,6 +1709,8 @@ int submodule_move_head(const char *path,
 
 			if (is_empty_dir(path))
 				rmdir_or_warn(path);
+
+			submodule_unset_core_worktree(sub);
 		}
 	}
 out:
diff --git a/submodule.h b/submodule.h
index 7856b8a0b3d..4644683e6cb 100644
--- a/submodule.h
+++ b/submodule.h
@@ -121,6 +121,8 @@ extern int submodule_move_head(const char *path,
 			       const char *new_head,
 			       unsigned flags);
 
+void submodule_unset_core_worktree(const struct submodule *sub);
+
 /*
  * Prepare the "env_array" parameter of a "struct child_process" for executing
  * a submodule by clearing any repo-specific environment variables, but
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 1f38a85371a..12cd4e9233e 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -709,7 +709,8 @@ test_submodule_recursing_with_args_common() {
 			git branch -t remove_sub1 origin/remove_sub1 &&
 			$command remove_sub1 &&
 			test_superproject_content origin/remove_sub1 &&
-			! test -e sub1
+			! test -e sub1 &&
+			test_must_fail git config -f .git/modules/sub1/config core.worktree
 		)
 	'
 	# ... absorbing a .git directory along the way.
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 20%]

* [PATCH 2/3] submodule: ensure core.worktree is set after update
  2018-06-12 23:58 [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree Stefan Beller
  2018-06-12 23:58 ` [PATCH 1/3] submodule: unset core.worktree if no working tree is present Stefan Beller
@ 2018-06-12 23:58 ` Stefan Beller
  2018-06-12 23:58 ` [PATCH 3/3] submodule deinit: unset core.worktree Stefan Beller
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-12 23:58 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 24 ++++++++++++++++++++++++
 git-submodule.sh            |  5 +++++
 2 files changed, 29 insertions(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index bd250ca2164..dffc55ed8ee 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1860,6 +1860,29 @@ static int check_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+static int connect_gitdir_workingtree(int argc, const char **argv, const char *prefix)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char *name, *path;
+	char *sm_gitdir;
+
+	if (argc != 3)
+		BUG("submodule--helper connect-gitdir-workingtree <name> <path>");
+
+	name = argv[1];
+	path = argv[2];
+
+	strbuf_addf(&sb, "%s/modules/%s", get_git_dir(), name);
+	sm_gitdir = absolute_pathdup(sb.buf);
+
+	connect_work_tree_and_git_dir(path, sm_gitdir, 0);
+
+	strbuf_release(&sb);
+	free(sm_gitdir);
+
+	return 0;
+}
+
 #define SUPPORT_SUPER_PREFIX (1<<0)
 
 struct cmd_struct {
@@ -1873,6 +1896,7 @@ static struct cmd_struct commands[] = {
 	{"name", module_name, 0},
 	{"clone", module_clone, 0},
 	{"update-clone", update_clone, 0},
+	{"connect-gitdir-workingtree", connect_gitdir_workingtree, 0},
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 78073cd87d1..6596a77c5ef 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -615,6 +615,11 @@ cmd_update()
 			die "$(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
 		fi
 
+		if ! $(git config -f "$(git rev-parse --git-common-dir)/modules/$name/config" core.worktree) 2>/dev/null
+		then
+			git submodule--helper connect-gitdir-workingtree $name $sm_path
+		fi
+
 		if test "$subsha1" != "$sha1" || test -n "$force"
 		then
 			subforce=$force
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 20%]

* [PATCH 3/3] submodule deinit: unset core.worktree
  2018-06-12 23:58 [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree Stefan Beller
  2018-06-12 23:58 ` [PATCH 1/3] submodule: unset core.worktree if no working tree is present Stefan Beller
  2018-06-12 23:58 ` [PATCH 2/3] submodule: ensure core.worktree is set after update Stefan Beller
@ 2018-06-12 23:58 ` Stefan Beller
  2018-06-13 18:00 ` [RFC PATCH 0/3] submodules with no working tree shall " Junio C Hamano
  2018-06-19  0:06 ` [PATCH " Stefan Beller
  4 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-12 23:58 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

When a submodule is deinit'd, the working tree is gone, so the setting of
core.worktree is bogus. Unset it.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 2 ++
 t/lib-submodule-update.sh   | 2 +-
 t/t7400-submodule-basic.sh  | 5 +++++
 3 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index dffc55ed8ee..19480902681 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -980,6 +980,8 @@ static void deinit_submodule(const char *path, const char *prefix,
 		if (!(flags & OPT_QUIET))
 			printf(format, displaypath);
 
+		submodule_unset_core_worktree(sub);
+
 		strbuf_release(&sb_rm);
 	}
 
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 12cd4e9233e..aa5ac03325a 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -235,7 +235,7 @@ reset_work_tree_to_interested () {
 	then
 		mkdir -p submodule_update/.git/modules/sub1/modules &&
 		cp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2
-		GIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree
+		# core.worktree is unset for sub2 as it is not checked out
 	fi &&
 	# indicate we are interested in the submodule:
 	git -C submodule_update config submodule.sub1.url "bogus" &&
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index be946fb1fd1..c6e1f749639 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -993,6 +993,11 @@ test_expect_success 'submodule deinit should remove the whole submodule section
 	rmdir init
 '
 
+test_expect_success 'submodule deinit should unset core.worktree' '
+	test_path_is_file .git/modules/example/config &&
+	test_must_fail git config -f .git/modules/example/config core.worktree
+'
+
 test_expect_success 'submodule deinit from subdirectory' '
 	git submodule update --init &&
 	git config submodule.example.foo bar &&
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 26%]

* Re: What's cooking in git.git (Jun 2018, #03; Tue, 12)
  2018-06-12 21:13 ` What's cooking in git.git (Jun 2018, #03; Tue, 12) Stefan Beller
@ 2018-06-13 15:56   ` Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-06-13 15:56 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Duy Nguyen, git

Stefan Beller <sbeller@google.com> writes:

>> A tl;dr summary for -rc2 that hopefully should happen in 24 hours:
>> the following four topics
>>
>>  jk/submodule-fsck-loose-fixup
>>  sb/submodule-merge-in-merge-recursive
>>  jk/index-pack-maint
>>  sg/completion-zsh-workaround
>>
>> are planned to be merged in -rc2.
>
> It would be nice to have
> https://public-inbox.org/git/20180609110414.GA5273@duynguyen.home/
> included as well, but as it is not yet a proper patch, I can
> understand why we err on the cautious side.

At this point, only clear fixes to regressions between v2.17.0..master
would be eligible for the final.  Is the one you are pointing at a
breakage during that timeframe?

Thanks.

^ permalink raw reply	[relevance 2%]

* Re: [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree
  2018-06-12 23:58 [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree Stefan Beller
                   ` (2 preceding siblings ...)
  2018-06-12 23:58 ` [PATCH 3/3] submodule deinit: unset core.worktree Stefan Beller
@ 2018-06-13 18:00 ` " Junio C Hamano
  2018-06-13 18:52   ` Stefan Beller
  2018-06-19  0:06 ` [PATCH " Stefan Beller
  4 siblings, 1 reply; 200+ results
From: Junio C Hamano @ 2018-06-13 18:00 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git

Stefan Beller <sbeller@google.com> writes:

> The first patch teaches checkout/reset (with --recurse-submodules) to unset
> the core.worktree config when the new state of the superprojects working tree
> doesn't contain the submodules working tree.

Are there two cases of "doesn't contain working tree of a submodule"?

The superproject's commit may not have a gitlink to a specific
submodule in its tree (i.e. there is no way to even "submodule init"
with such a commit in the superproject's history).  Or there may be
a gitlink but the user chose not to check it out in the working
tree.

Do they need to be treated differently, or can they be treated the
same way?

Also, is the "submodule" feature supposed to play well with multiple
worktree feature?  Let's imagine that you have two worktrees for a
single superproject, and the branches of the superproject these two
worktrees check out are different ones (which is the more sensible
set-up than checking out the same branch twice).  Further imagine
that the superproject started using a single submodule sometime in
the past and keeps using it throughout its life since then.

 1. if both of these two branches have the submodule, and two
    worktrees both are interested in having the submodule checked
    out via "submodule init/update", where does core.worktree point
    at?  Do we have two copies of the variable?

 2. what if one branch predates the use of the submodule in the
    superproject while the other branch is newer and uses the
    submodule?  Where does core.worktree point at?

Thanks.

> The last patch is teaching "git submodule deinit" to unset the core.worktree
> setting as well. It turned out this one is tricky, as for that we also
> have to set it in the counter part, such as "submodule update".
>
> Thanks,
> Stefan
>
> Stefan Beller (3):
>   submodule: unset core.worktree if no working tree is present
>   submodule: ensure core.worktree is set after update
>   submodule deinit: unset core.worktree
>
>  builtin/submodule--helper.c | 26 ++++++++++++++++++++++++++
>  git-submodule.sh            |  5 +++++
>  submodule.c                 | 14 ++++++++++++++
>  submodule.h                 |  2 ++
>  t/lib-submodule-update.sh   |  5 +++--
>  t/t7400-submodule-basic.sh  |  5 +++++
>  6 files changed, 55 insertions(+), 2 deletions(-)

^ permalink raw reply	[relevance 8%]

* Re: [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree
  2018-06-13 18:00 ` [RFC PATCH 0/3] submodules with no working tree shall " Junio C Hamano
@ 2018-06-13 18:52   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-13 18:52 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Wed, Jun 13, 2018 at 11:00 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Stefan Beller <sbeller@google.com> writes:
>
> > The first patch teaches checkout/reset (with --recurse-submodules) to unset
> > the core.worktree config when the new state of the superprojects working tree
> > doesn't contain the submodules working tree.
>
> Are there two cases of "doesn't contain working tree of a submodule"?
>
> The superproject's commit may not have a gitlink to a specific
> submodule in its tree (i.e. there is no way to even "submodule init"
> with such a commit in the superproject's history).

... for example git.git before 2.13 did not have a gitlink for its
sha1collision submodule ...

>   Or there may be
> a gitlink but the user chose not to check it out in the working
> tree.

This is when the submodule is not active or de-initialized or not initialized
to begin with. However there is an empty directory as a place holder.
As long as the empty place holder is there, we do not run into trouble.

> Do they need to be treated differently, or can they be treated the
> same way?

I would think they are the same, as both cases do not have a working tree
for the submodule.

> Also, is the "submodule" feature supposed to play well with multiple
> worktree feature?

It is a long term goal to get this working.

>  Let's imagine that you have two worktrees for a
> single superproject, and the branches of the superproject these two
> worktrees check out are different ones (which is the more sensible
> set-up than checking out the same branch twice).

(git-worktree even prevents you to checkout the same branch),
sounds all very sensible up to here.


> Further imagine
> that the superproject started using a single submodule sometime in
> the past and keeps using it throughout its life since then.
>
>  1. if both of these two branches have the submodule, and two
>     worktrees both are interested in having the submodule checked
>     out via "submodule init/update", where does core.worktree point
>     at?  Do we have two copies of the variable?

Currently in the real world (aka in origin/master), the submodules are
completely duplicated, and do not know about the worktree feature
themselves:

    $ cd .git && find . |grep config |grep sha1co
    ./modules/sha1collisiondetection/config
    ./worktrees/git/modules/sha1collisiondetection/config

(Yes I do have a worktree called "git".)

So for the sake of this patch series this should be okay, as there
is only ever one working tree for a submodule currently.

The current state of affairs is really bad as the submodule is
there twice with all objects and the rest.

>  2. what if one branch predates the use of the submodule in the
>     superproject while the other branch is newer and uses the
>     submodule?  Where does core.worktree point at?

As we have two copies of the repository, one of them points at the
correct place, and the other would be unset.

---

In an ideal world we would want the submodules to use the worktree
feature along-side the superproject, e.g. if the superproject has the
two worktrees as above, the submodule would also get these.

However I have not thought about this future too deeply, as there
are a couple bumps along the road:

Supposedly the submodules main (common) git dir would stay in the
common git dir of the superproject at /modules/<name>.
However where would we put the submodules non-main worktree
git dirs? There are two places, as seen from the superprojects:

  .git/modules/<submodule-name>/worktrees/<worktree-name>
  .git/worktrees/<worktree-name>/modules/<submodule-name>

As worktrees can be removed pretty easily ("git worktree prune"),
I would not put the main parts of a submodule into a worktree part
of the superproject (The user could be in a non-main worktree
when they first ask for the submodule -- we have to make sure
the main git dir goes into the superprojects main git dir under
modules/).

An advantage for
   .git/worktrees/<worktree-name>/modules/<submodule-name>
would be that the worktree of the submodule is coupled to the
worktree of the superproject, i.e. if the user wipes the superprojects
worktree, the submodules accompanying worktree is gone automatically.

An advantage for
  .git/modules/<submodule-name>/worktrees/<worktree-name>
is to have any submodule related things in one place in
  .git/modules/<submodule-name>

Cleanup of the worktrees would be a bit harder as we'd have to
remember to prune the worktrees for all submodules as well
when the superproject removes one of its worktrees.

It is relatively rare to delete the submodule git directory compared to
removing a worktree, such that
  .git/modules/<submodule-name>/worktrees/<worktree-name>
sounds a bit more promising to me.

However the worktree command needs to figure out the worktrees:
    /path/to/super/worktree/submodule $ git worktree list

This would look at its git dir, which is

 .git/modules/<submodule-name>/worktrees/<specific-worktree>

discover the common dir at

 .git/modules/<submodule-name>

and then proceed in listing worktrees by exploring worktrees/.
If we had put it in the other form of
   .git/worktrees/<worktree-name>/modules/<submodule-name>
we'd discover the common git dir at
  .git/modules/<submodule-name>
and list it from there.

However currently we use the existence of directories (both for
submodules as well as worktrees) to infer some knowledge about
the existence of submodules and worktrees, so we cannot just declare
one version or the other to be the future, but we'd have to add some
form of linking, I would think.

Stefan

^ permalink raw reply	[relevance 7%]

* [ANNOUNCE] Git v2.18.0-rc2
@ 2018-06-13 22:12 Junio C Hamano
  0 siblings, 0 replies; 200+ results
From: Junio C Hamano @ 2018-06-13 22:12 UTC (permalink / raw)
  To: git; +Cc: Linux Kernel, git-packagers

A release candidate Git v2.18.0-rc2 is now available for testing
at the usual places.  It is comprised of 852 non-merge commits
since v2.17.0, contributed by 65 people, 20 of which are new faces.

The tarballs are found at:

    https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the
'v2.18.0-rc2' tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.17.0 are as follows.
Welcome to the Git development community!

  Bill Ritcher, Birger Skogeng Pedersen, Casey Fitzpatrick,
  Dan Jacques, Drew DeVault, Eckhard S. Maaß, Erik E Brady,
  Florian Gamböck, Harald Nordgren, Leif Middelschulte,
  Loganaden Velvindron, Luis Marsano, Paul-Sebastian Ungureanu,
  Pedro Alvarez Piedehierro, Pratik Karki, Ryan Dammrose, Takuto
  Ikuta, Tao Qingyun, Wink Saville, and Yuki Kokubun.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Ævar Arnfjörð Bjarmason, Anders Kaseorg, Andreas Heiduk,
  Antonio Ospite, Beat Bolli, Ben Peart, Brandon Williams, brian
  m. carlson, Christian Couder, Christian Hesse, Clemens Buchacher,
  David Turner, Derrick Stolee, Elijah Newren, Eric Sunshine,
  Jameson Miller, Jeff King, Johannes Schindelin, Johannes Sixt,
  Jonathan Nieder, Jonathan Tan, Junio C Hamano, Kaartic Sivaraam,
  Lars Schneider, Lucas Werkmeister, Luke Diamand, Martin Ågren,
  Michal Nazarewicz, Michele Locati, Nguyễn Thái Ngọc Duy,
  Olga Telezhnaya, Orgad Shaneh, Philip Oakley, Phillip Wood,
  Ramsay Jones, René Scharfe, Robert P. J. Day, Sergey Organov,
  Stefan Agner, Stefan Beller, SZEDER Gábor, Taylor Blau, Thomas
  Gummerer, Todd Zullinger, and Torsten Bögershausen.

----------------------------------------------------------------

Git 2.18 Release Notes (draft)
==============================

Updates since v2.17
-------------------

UI, Workflows & Features

 * Rename detection logic that is used in "merge" and "cherry-pick" has
   learned to guess when all of x/a, x/b and x/c have moved to z/a,
   z/b and z/c, it is likely that x/d added in the meantime would also
   want to move to z/d by taking the hint that the entire directory
   'x' moved to 'z'.  A bug causing dirty files involved in a rename
   to be overwritten during merge has also been fixed as part of this
   work.  Incidentally, this also avoids updating a file in the
   working tree after a (non-trivial) merge whose result matches what
   our side originally had.

 * "git filter-branch" learned to use a different exit code to allow
   the callers to tell the case where there was no new commits to
   rewrite from other error cases.

 * When built with more recent cURL, GIT_SSL_VERSION can now specify
   "tlsv1.3" as its value.

 * "git gui" learned that "~/.ssh/id_ecdsa.pub" and
   "~/.ssh/id_ed25519.pub" are also possible SSH key files.
   (merge 2e2f0288ef bb/git-gui-ssh-key-files later to maint).

 * "git gui" performs commit upon CTRL/CMD+ENTER but the
   CTRL/CMD+KP_ENTER (i.e. enter key on the numpad) did not have the
   same key binding.  It now does.
   (merge 28a1d94a06 bp/git-gui-bind-kp-enter later to maint).

 * "git gui" has been taught to work with old versions of tk (like
   8.5.7) that do not support "ttk::style theme use" as a way to query
   the current theme.
   (merge 4891961105 cb/git-gui-ttk-style later to maint).

 * "git rebase" has learned to honor "--signoff" option when using
   backends other than "am" (but not "--preserve-merges").

 * "git branch --list" during an interrupted "rebase -i" now lets
   users distinguish the case where a detached HEAD is being rebased
   and a normal branch is being rebased.

 * "git mergetools" learned talking to guiffy.

 * The scripts in contrib/emacs/ have outlived their usefulness and
   have been replaced with a stub that errors out and tells the user
   there are replacements.

 * The new "working-tree-encoding" attribute can ask Git to convert the
   contents to the specified encoding when checking out to the working
   tree (and the other way around when checking in).

 * The "git config" command uses separate options e.g. "--int",
   "--bool", etc. to specify what type the caller wants the value to
   be interpreted as.  A new "--type=<typename>" option has been
   introduced, which would make it cleaner to define new types.

 * "git config --get" learned the "--default" option, to help the
   calling script.  Building on top of the above changes, the
   "git config" learns "--type=color" type.  Taken together, you can
   do things like "git config --get foo.color --default blue" and get
   the ANSI color sequence for the color given to foo.color variable,
   or "blue" if the variable does not exist.

 * "git ls-remote" learned an option to allow sorting its output based
   on the refnames being shown.

 * The command line completion (in contrib/) has been taught that "git
   stash save" has been deprecated ("git stash push" is the preferred
   spelling in the new world) and does not offer it as a possible
   completion candidate when "git stash push" can be.

 * "git gc --prune=nonsense" spent long time repacking and then
   silently failed when underlying "git prune --expire=nonsense"
   failed to parse its command line.  This has been corrected.

 * Error messages from "git push" can be painted for more visibility.

 * "git http-fetch" (deprecated) had an optional and experimental
   "feature" to fetch only commits and/or trees, which nobody used.
   This has been removed.

 * The functionality of "$GIT_DIR/info/grafts" has been superseded by
   the "refs/replace/" mechanism for some time now, but the internal
   code had support for it in many places, which has been cleaned up
   in order to drop support of the "grafts" mechanism.

 * "git worktree add" learned to check out an existing branch.

 * "git --no-pager cmd" did not have short-and-sweet single letter
   option. Now it does as "-P".
   (merge 7213c28818 js/no-pager-shorthand later to maint).

 * "git rebase" learned "--rebase-merges" to transplant the whole
   topology of commit graph elsewhere.

 * "git status" learned to pay attention to UI related diff
   configuration variables such as diff.renames.

 * The command line completion mechanism (in contrib/) learned to load
   custom completion file for "git $command" where $command is a
   custom "git-$command" that the end user has on the $PATH when using
   newer version of bash.

 * "git send-email" can sometimes offer confirmation dialog "Send this
   email?" with choices 'Yes', 'No', 'Quit', and 'All'.  A new action
   'Edit' has been added to this dialog's choice.

 * With merge.renames configuration set to false, the recursive merge
   strategy can be told not to spend cycles trying to find renamed
   paths and merge them accordingly.

 * "git status" learned to honor a new status.renames configuration to
   skip rename detection, which could be useful for those who want to
   do so without disabling the default rename detection done by the
   "git diff" command.

 * Command line completion (in contrib/) learned to complete pathnames
   for various commands better.

 * "git blame" learns to unhighlight uninteresting metadata from the
   originating commit on lines that are the same as the previous one,
   and also paint lines in different colors depending on the age of
   the commit.

 * Transfer protocol v2 learned to support the partial clone.

 * When a short hexadecimal string is used to name an object but there
   are multiple objects that share the string as the prefix of their
   names, the code lists these ambiguous candidates in a help message.
   These object names are now sorted according to their types for
   easier eyeballing.

 * "git fetch $there $refspec" that talks over protocol v2 can take
   advantage of server-side ref filtering; the code has been extended
   so that this mechanism triggers also when fetching with configured
   refspec.

 * Our HTTP client code used to advertise that we accept gzip encoding
   from the other side; instead, just let cURL library to advertise
   and negotiate the best one.

 * "git p4" learned to "unshelve" shelved commit from P4.
   (merge 123f631761 ld/p4-unshelve later to maint).


Performance, Internal Implementation, Development Support etc.

 * A "git fetch" from a repository with insane number of refs into a
   repository that is already up-to-date still wasted too many cycles
   making many lstat(2) calls to see if these objects at the tips
   exist as loose objects locally.  These lstat(2) calls are optimized
   away by enumerating all loose objects beforehand.
   It is unknown if the new strategy negatively affects existing use
   cases, fetching into a repository with many loose objects from a
   repository with small number of refs.

 * Git can be built to use either v1 or v2 of the PCRE library, and so
   far, the build-time configuration USE_LIBPCRE=YesPlease instructed
   the build procedure to use v1, but now it means v2.  USE_LIBPCRE1
   and USE_LIBPCRE2 can be used to explicitly choose which version to
   use, as before.

 * The build procedure learned to optionally use symbolic links
   (instead of hardlinks and copies) to install "git-foo" for built-in
   commands, whose binaries are all identical.

 * Conversion from uchar[20] to struct object_id continues.

 * The way "git worktree prune" worked internally has been simplified,
   by assuming how "git worktree move" moves an existing worktree to a
   different place.

 * Code clean-up for the "repository" abstraction.
   (merge 00a3da2a13 nd/remove-ignore-env-field later to maint).

 * Code to find the length to uniquely abbreviate object names based
   on packfile content, which is a relatively recent addtion, has been
   optimized to use the same fan-out table.

 * The mechanism to use parse-options API to automate the command line
   completion continues to get extended and polished.

 * Copies of old scripted Porcelain commands in contrib/examples/ have
   been removed.

 * Some tests that rely on the exact hardcoded values of object names
   have been updated in preparation for hash function migration.

 * Perf-test update.

 * Test helper update.

 * The effort continues to refactor the internal global data structure
   to make it possible to open multiple repositories, work with and
   then close them,

 * Small test-helper programs have been consolidated into a single
   binary.

 * API clean-up around ref-filter code.

 * Shell completion (in contrib) that gives list of paths have been
   optimized somewhat.

 * The index file is updated to record the fsmonitor section after a
   full scan was made, to avoid wasting the effort that has already
   spent.

 * Performance measuring framework in t/perf learned to help bisecting
   performance regressions.

 * Some multi-word source filenames are being renamed to separate
   words with dashes instead of underscores.

 * An reusable "memory pool" implementation has been extracted from
   fast-import.c, which in turn has become the first user of the
   mem-pool API.

 * A build-time option has been added to allow Git to be told to refer
   to its associated files relative to the main binary, in the same
   way that has been possible on Windows for quite some time, for
   Linux, BSDs and Darwin.

 * Precompute and store information necessary for ancestry traversal
   in a separate file to optimize graph walking.

 * The effort to pass the repository in-core structure throughout the
   API continues.  This round deals with the code that implements the
   refs/replace/ mechanism.

 * The build procedure "make DEVELOPER=YesPlease" learned to enable a
   bit more warning options depending on the compiler used to help
   developers more.  There also is "make DEVOPTS=tokens" knob
   available now, for those who want to help fixing warnings we
   usually ignore, for example.

 * A new version of the transport protocol is being worked on.

 * The code to interface to GPG has been restructured somewhat to make
   it cleaner to integrate with other types of signature systems later.

 * The code has been taught to use the duplicated information stored
   in the commit-graph file to learn the tree object name for a commit
   to avoid opening and parsing the commit object when it makes sense
   to do so.

 * "git gc" in a large repository takes a lot of time as it considers
   to repack all objects into one pack by default.  The command has
   been taught to pretend as if the largest existing packfile is
   marked with ".keep" so that it is left untouched while objects in
   other packs and loose ones are repacked.

 * The transport protocol v2 is getting updated further.

 * The codepath around object-info API has been taught to take the
   repository object (which in turn tells the API which object store
   the objects are to be located).

 * "git pack-objects" needs to allocate tons of "struct object_entry"
   while doing its work, and shrinking its size helps the performance
   quite a bit.

 * The implementation of "git rebase -i --root" has been updated to use
   the sequencer machinery more.

 * Developer support update, by using BUG() macro instead of die() to
   mark codepaths that should not happen more clearly.

 * Developer support.  Use newer GCC on one of the builds done at
   TravisCI.org to get more warnings and errors diagnosed.

 * Conversion from uchar[20] to struct object_id continues.

 * By code restructuring of submodule merge in merge-recursive,
   informational messages from the codepath are now given using the
   same mechanism as other output, and honor the merge.verbosity
   configuration.  The code also learned to give a few new messages
   when a submodule three-way merge resolves cleanly when one side
   records a descendant of the commit chosen by the other side.

 * Avoid unchecked snprintf() to make future code auditing easier.
   (merge ac4896f007 jk/snprintf-truncation later to maint).

 * Many tests hardcode the raw object names, which would change once
   we migrate away from SHA-1.  While some of them must test against
   exact object names, most of them do not have to use hardcoded
   constants in the test.  The latter kind of tests have been updated
   to test the moral equivalent of the original without hardcoding the
   actual object names.

 * The list of commands with their various attributes were spread
   across a few places in the build procedure, but it now is getting a
   bit more consolidated to allow more automation.

 * Quite a many tests assumed that newly created refs are made as
   loose refs using the files backend, which have been updated to use
   proper plumbing like rev-parse and update-ref, to avoid breakage
   once we start using different ref backends.


Also contains various documentation updates and code clean-ups.


Fixes since v2.17
-----------------

 * "git shortlog cruft" aborted with a BUG message when run outside a
   Git repository.  The command has been taught to complain about
   extra and unwanted arguments on its command line instead in such a
   case.
   (merge 4aa0161e83 ma/shortlog-revparse later to maint).

 * "git stash push -u -- <pathspec>" gave an unnecessary and confusing
   error message when there was no tracked files that match the
   <pathspec>, which has been fixed.
   (merge 353278687e tg/stash-untracked-with-pathspec-fix later to maint).

 * "git tag --contains no-such-commit" gave a full list of options
   after giving an error message.
   (merge 3bb0923f06 ps/contains-id-error-message later to maint).

 * "diff-highlight" filter (in contrib/) learned to undertand "git log
   --graph" output better.
   (merge 4551fbba14 jk/diff-highlight-graph-fix later to maint).

 * when refs that do not point at committish are given, "git
   filter-branch" gave a misleading error messages.  This has been
   corrected.
   (merge f78ab355e7 yk/filter-branch-non-committish-refs later to maint).

 * "git submodule status" misbehaved on a submodule that has been
   removed from the working tree.
   (merge 74b6bda32f rs/status-with-removed-submodule later to maint).

 * When credential helper exits very quickly without reading its
   input, it used to cause Git to die with SIGPIPE, which has been
   fixed.
   (merge a0d51e8d0e eb/cred-helper-ignore-sigpipe later to maint).

 * "git rebase --keep-empty" still removed an empty commit if the
   other side contained an empty commit (due to the "does an
   equivalent patch exist already?" check), which has been corrected.
   (merge 3d946165e1 pw/rebase-keep-empty-fixes later to maint).

 * Some codepaths, including the refs API, get and keep relative
   paths, that go out of sync when the process does chdir(2).  The
   chdir-notify API is introduced to let these codepaths adjust these
   cached paths to the new current directory.
   (merge fb9c2d2703 jk/relative-directory-fix later to maint).

 * "cd sub/dir && git commit ../path" ought to record the changes to
   the file "sub/path", but this regressed long time ago.
   (merge 86238e07ef bw/commit-partial-from-subdirectory-fix later to maint).

 * Recent introduction of "--log-destination" option to "git daemon"
   did not work well when the daemon was run under "--inetd" mode.
   (merge e67d906d73 lw/daemon-log-destination later to maint).

 * Small fix to the autoconf build procedure.
   (merge 249482daf0 es/fread-reads-dir-autoconf-fix later to maint).

 * Fix an unexploitable (because the oversized contents are not under
   attacker's control) buffer overflow.
   (merge d8579accfa bp/fsmonitor-bufsize-fix later to maint).

 * Recent simplification of build procedure forgot a bit of tweak to
   the build procedure of contrib/mw-to-git/
   (merge d8698987f3 ab/simplify-perl-makefile later to maint).

 * Moving a submodule that itself has submodule in it with "git mv"
   forgot to make necessary adjustment to the nested sub-submodules;
   now the codepath learned to recurse into the submodules.

 * "git config --unset a.b", when "a.b" is the last variable in an
   otherwise empty section "a", left an empty section "a" behind, and
   worse yet, a subsequent "git config a.c value" did not reuse that
   empty shell and instead created a new one.  These have been
   (partially) corrected.
   (merge c71d8bb38a js/empty-config-section-fix later to maint).

 * "git worktree remove" learned that "-f" is a shorthand for
   "--force" option, just like for "git worktree add".
   (merge d228eea514 sb/worktree-remove-opt-force later to maint).

 * The completion script (in contrib/) learned to clear cached list of
   command line options upon dot-sourcing it again in a more efficient
   way.
   (merge 94408dc71c sg/completion-clear-cached later to maint).

 * "git svn" had a minor thinko/typo which has been fixed.
   (merge 51db271587 ab/git-svn-get-record-typofix later to maint).

 * During a "rebase -i" session, the code could give older timestamp
   to commits created by later "pick" than an earlier "reword", which
   has been corrected.
   (merge 12f7babd6b js/ident-date-fix later to maint).

 * "git submodule status" did not check the symbolic revision name it
   computed for the submodule HEAD is not the NULL, and threw it at
   printf routines, which has been corrected.
   (merge 0b5e2ea7cf nd/submodule-status-fix later to maint).

 * When fed input that already has In-Reply-To: and/or References:
   headers and told to add the same information, "git send-email"
   added these headers separately, instead of appending to an existing
   one, which is a violation of the RFC.  This has been corrected.
   (merge 256be1d3f0 sa/send-email-dedup-some-headers later to maint).

 * "git fast-export" had a regression in v2.15.0 era where it skipped
   some merge commits in certain cases, which has been corrected.
   (merge be011bbe00 ma/fast-export-skip-merge-fix later to maint).

 * The code did not propagate the terminal width to subprocesses via
   COLUMNS environment variable, which it now does.  This caused
   trouble to "git column" helper subprocess when "git tag --column=row"
   tried to list the existing tags on a display with non-default width.
   (merge b5d5a567fb nd/term-columns later to maint).

 * We learned that our source files with ".pl" and ".py" extensions
   are Perl and Python files respectively and changes to them are
   better viewed as such with appropriate diff drivers.
   (merge 7818b619e2 ab/perl-python-attrs later to maint).

 * "git rebase -i" sometimes left intermediate "# This is a
   combination of N commits" message meant for the human consumption
   inside an editor in the final result in certain corner cases, which
   has been fixed.
   (merge 15ef69314d js/rebase-i-clean-msg-after-fixup-continue later to maint).

 * A test to see if the filesystem normalizes UTF-8 filename has been
   updated to check what we need to know in a more direct way, i.e. a
   path created in NFC form can be accessed with NFD form (or vice
   versa) to cope with APFS as well as HFS.
   (merge 742ae10e35 tb/test-apfs-utf8-normalization later to maint).

 * "git format-patch --cover --attach" created a broken MIME multipart
   message for the cover letter, which has been fixed by keeping the
   cover letter as plain text file.
   (merge 50cd54ef4e bc/format-patch-cover-no-attach later to maint).

 * The split-index feature had a long-standing and dormant bug in
   certain use of the in-core merge machinery, which has been fixed.
   (merge 7db118303a en/unpack-trees-split-index-fix later to maint).

 * Asciidoctor gives a reasonable imitation for AsciiDoc, but does not
   render illustration in a literal block correctly when indented with
   HT by default. The problem is fixed by forcing 8-space tabs.
   (merge 379805051d bc/asciidoctor-tab-width later to maint).

 * Code clean-up to adjust to a more recent lockfile API convention that
   allows lockfile instances kept on the stack.
   (merge 0fa5a2ed8d ma/lockfile-cleanup later to maint).

 * the_repository->index is not a allocated piece of memory but
   repo_clear() indiscriminately attempted to free(3) it, which has
   been corrected.
   (merge 74373b5f10 nd/repo-clear-keep-the-index later to maint).

 * Code clean-up to avoid non-standard-conformant pointer arithmetic.
   (merge c112084af9 rs/no-null-ptr-arith-in-fast-export later to maint).

 * Code clean-up to turn history traversal more robust in a
   semi-corrupt repository.
   (merge 8702b30fd7 jk/unavailable-can-be-missing later to maint).

 * "git update-ref A B" is supposed to ensure that ref A does not yet
   exist when B is a NULL OID, but this check was not done correctly
   for pseudo-refs outside refs/ hierarchy, e.g. MERGE_HEAD.

 * "git submodule update" and "git submodule add" supported the
   "--reference" option to borrow objects from a neighbouring local
   repository like "git clone" does, but lacked the more recent
   invention "--dissociate".  Also "git submodule add" has been taught
   to take the "--progress" option.
   (merge a0ef29341a cf/submodule-progress-dissociate later to maint).

 * Update credential-netrc helper (in contrib/) to allow customizing
   the GPG used to decrypt the encrypted .netrc file.
   (merge 786ef50a23 lm/credential-netrc later to maint).

 * "git submodule update" attempts two different kinds of "git fetch"
   against the upstream repository to grab a commit bound at the
   submodule's path, but it incorrectly gave up if the first kind
   (i.e. a normal fetch) failed, making the second "last resort" one
   (i.e. fetching an exact commit object by object name) ineffective.
   This has been corrected.
   (merge e30d833671 sb/submodule-update-try-harder later to maint).

 * Error behaviour of "git grep" when it cannot read the index was
   inconsistent with other commands that uses the index, which has
   been corrected to error out early.
   (merge b2aa84c789 sb/grep-die-on-unreadable-index later to maint).

 * We used to call regfree() after regcomp() failed in some codepaths,
   which have been corrected.
   (merge 17154b1576 ma/regex-no-regfree-after-comp-fail later to maint).

 * The import-tars script (in contrib/) has been taught to handle
   tarballs with overly long paths that use PAX extended headers.
   (merge 12ecea46e3 pa/import-tars-long-names later to maint).

 * "git rev-parse Y..." etc. misbehaved when given endpoints were
   not committishes.
   (merge 0ed556d38f en/rev-parse-invalid-range later to maint).

 * "git pull --recurse-submodules --rebase", when the submodule
   repository's history did not have anything common between ours and
   the upstream's, failed to execute.  We need to fetch from them to
   continue even in such a case.
   (merge 4d36f88be7 jt/submodule-pull-recurse-rebase later to maint).

 * "git remote update" can take both a single remote nickname and a
   nickname for remote groups, but only one of them was documented.
   (merge a97447a42a nd/remote-update-doc later to maint).

 * "index-pack --strict" has been taught to make sure that it runs the
   final object integrity checks after making the freshly indexed
   packfile available to itself.
   (merge 3737746120 jk/index-pack-maint later to maint).

 * Other minor doc, test and build updates and code cleanups.
   (merge 248f66ed8e nd/trace-with-env later to maint).
   (merge 14ced5562c ys/bisect-object-id-missing-conversion-fix later to maint).
   (merge 5988eb631a ab/doc-hash-brokenness later to maint).
   (merge a4d4e32a70 pk/test-avoid-pipe-hiding-exit-status later to maint).
   (merge 05e293c1ac jk/flockfile-stdio later to maint).
   (merge e9184b0789 jk/t5561-missing-curl later to maint).
   (merge b1801b85a3 nd/worktree-move later to maint).
   (merge bbd374dd20 ak/bisect-doc-typofix later to maint).
   (merge 4855f06fb3 mn/send-email-credential-doc later to maint).
   (merge 8523b1e355 en/doc-typoes later to maint).
   (merge 43b44ccfe7 js/t5404-path-fix later to maint).
   (merge decf711fc1 ps/test-chmtime-get later to maint).
   (merge 22d11a6e8e es/worktree-docs later to maint).
   (merge 92a5dbbc22 tg/use-git-contacts later to maint).
   (merge adc887221f tq/t1510 later to maint).
   (merge bed21a8ad6 sg/doc-gc-quote-mismatch-fix later to maint).
   (merge 73364e4f10 tz/doc-git-urls-reference later to maint).
   (merge cd1e606bad bc/mailmap-self later to maint).
   (merge f7997e3682 ao/config-api-doc later to maint).
   (merge ee930754d8 jk/apply-p-doc later to maint).
   (merge 011b648646 nd/pack-format-doc later to maint).
   (merge 87a6bb701a sg/t5310-jgit-bitmap-test later to maint).
   (merge f6b82970aa sg/t5516-fixes later to maint).
   (merge 4362da078e sg/t7005-spaces-in-filenames-cleanup later to maint).
   (merge 7d0ee47c11 js/test-unset-prereq later to maint).
   (merge 5356a3c354 ah/misc-doc-updates later to maint).
   (merge 92c4a7a129 nd/completion-aliasfiletype-typofix later to maint).
   (merge 58bd77b66a nd/pack-unreachable-objects-doc later to maint).
   (merge 4ed79d5203 sg/t6500-no-redirect-of-stdin later to maint).
   (merge 17b8a2d6cd jk/config-blob-sans-repo later to maint).
   (merge 590551ca2c rd/tag-doc-lightweight later to maint).
   (merge 44f560fc16 rd/init-typo later to maint).
   (merge f156a0934a rd/p4-doc-markup-env later to maint).
   (merge 2a00502b14 tg/doc-sec-list later to maint).
   (merge 47cc91310a jk/submodule-fsck-loose-fixup later to maint).

^ permalink raw reply	[relevance 3%]

* [PATCH v2 00/31] object-store: lookup_commit
  2018-05-30  0:47 [RFC PATCH 00/35] object-store: lookup_commit Stefan Beller
                   ` (3 preceding siblings ...)
  2018-05-30  0:47 ` [PATCH 18/35] commit: add repository argument to parse_commit Stefan Beller
@ 2018-06-13 23:04 ` Stefan Beller
  2018-06-13 23:04   ` [PATCH v2 07/31] commit: add repository argument to lookup_commit_reference Stefan Beller
                     ` (2 more replies)
  4 siblings, 3 replies; 200+ results
From: Stefan Beller @ 2018-06-13 23:04 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

* removed mentions of cooci patches
* added forward declaration of commit buffer slabs.
* dropped 3 patches that add the repository to lookup_unkonwn_object,
  parse_commit and parse_commit_gently, but were not converting those
  functions. We'll convert these in the next series, as this series is
  growing big already.
* This series can be found as branch 'object-store-lookup-commit' on github,
  it applies on top of nd/commit-util-to-slab merged with sb/object-store-grafts

v1, https://public-inbox.org/git/20180530004810.30076-1-sbeller@google.com/

This applies on the merge of nd/commit-util-to-slab and sb/object-store-grafts,
and is available at http://github.com/stefanbeller/ as branch object-store-lookup-commit
as the merge has some merge conflicts as well as syntactical conflicts (upload-pack.c
and fetch-pack.c introduce new calls of functions that would want to take a repository struct
in the object-store-grafts series)

As layed out in https://public-inbox.org/git/20180517225154.9200-1-sbeller@google.com/
this is getting close to finishing the set of object store series though the last
unfinished part of this RFC hints at new work on the plate:
* To give this series a nice polish, we'd want to convert parse_commit, too.
  But that requires the conversion of the new commit graph. Maybe we need
  to split this series into 2. 
* Once this is in good shape we can talk about converting parts of the revision
  walking code,
* which then can be used by the submodule code as the end goal for the
  object store series.

Thanks,
Stefan

Stefan Beller (31):
  object: add repository argument to lookup_object
  object: add repository argument to parse_object_buffer
  object: add repository argument to object_as_type
  blob: add repository argument to lookup_blob
  tree: add repository argument to lookup_tree
  commit: add repository argument to lookup_commit_reference_gently
  commit: add repository argument to lookup_commit_reference
  commit: add repository argument to lookup_commit
  commit: add repository argument to parse_commit_buffer
  commit: add repository argument to set_commit_buffer
  commit: add repository argument to get_cached_commit_buffer
  tag: add repository argument to lookup_tag
  tag: add repository argument to parse_tag_buffer
  tag: add repository argument to deref_tag
  object: allow object_as_type to handle arbitrary repositories
  object: allow lookup_object to handle arbitrary repositories
  blob: allow lookup_blob to handle arbitrary repositories
  tree: allow lookup_tree to handle arbitrary repositories
  commit: allow lookup_commit to handle arbitrary repositories
  tag: allow lookup_tag to handle arbitrary repositories
  tag: allow parse_tag_buffer to handle arbitrary repositories
  commit.c: allow parse_commit_buffer to handle arbitrary repositories
  commit-slabs: remove realloc counter outside of slab struct
  commit.c: migrate the commit buffer to the parsed object store
  commit.c: allow set_commit_buffer to handle arbitrary repositories
  commit.c: allow get_cached_commit_buffer to handle arbitrary
    repositories
  object.c: allow parse_object_buffer to handle arbitrary repositories
  object.c: allow parse_object to handle arbitrary repositories
  tag.c: allow deref_tag to handle arbitrary repositories
  commit.c: allow lookup_commit_reference_gently to handle arbitrary
    repositories
  commit.c: allow lookup_commit_reference to handle arbitrary
    repositories

 archive.c                |  2 +-
 bisect.c                 |  2 +-
 blame.c                  | 13 +++----
 blob.c                   | 10 +++---
 blob.h                   |  2 +-
 branch.c                 |  2 +-
 builtin/am.c             |  9 +++--
 builtin/branch.c         |  7 ++--
 builtin/checkout.c       |  6 ++--
 builtin/clone.c          |  3 +-
 builtin/commit-tree.c    |  4 ++-
 builtin/describe.c       | 13 +++----
 builtin/diff-tree.c      |  6 ++--
 builtin/diff.c           |  5 +--
 builtin/fast-export.c    | 12 ++++---
 builtin/fetch.c          |  9 +++--
 builtin/fmt-merge-msg.c  |  5 +--
 builtin/fsck.c           | 16 +++++----
 builtin/grep.c           |  3 +-
 builtin/index-pack.c     |  5 +--
 builtin/log.c            |  7 ++--
 builtin/merge-base.c     |  7 ++--
 builtin/merge-tree.c     |  3 +-
 builtin/name-rev.c       |  6 ++--
 builtin/notes.c          |  3 +-
 builtin/pack-objects.c   |  2 +-
 builtin/prune.c          |  2 +-
 builtin/pull.c           | 15 +++++---
 builtin/reflog.c         | 12 ++++---
 builtin/replace.c        |  4 +--
 builtin/reset.c          |  4 +--
 builtin/rev-parse.c      |  6 ++--
 builtin/show-branch.c    |  5 +--
 builtin/tag.c            |  2 +-
 builtin/unpack-objects.c |  7 ++--
 builtin/verify-commit.c  |  4 ++-
 bundle.c                 |  5 +--
 cache-tree.c             |  3 +-
 commit-graph.c           | 14 ++++----
 commit-slab-impl.h       |  3 --
 commit.c                 | 77 +++++++++++++++++++++++++---------------
 commit.h                 | 18 ++++++----
 fast-import.c            |  6 ++--
 fetch-pack.c             | 25 +++++++------
 fsck.c                   |  4 +--
 http-backend.c           |  2 +-
 http-push.c              | 10 +++---
 line-log.c               |  2 +-
 list-objects.c           |  4 +--
 log-tree.c               |  6 ++--
 merge-recursive.c        |  9 ++---
 notes-cache.c            |  3 +-
 notes-merge.c            |  5 +--
 notes-utils.c            |  4 ++-
 object.c                 | 59 ++++++++++++++++--------------
 object.h                 | 13 ++++---
 parse-options-cb.c       |  2 +-
 pretty.c                 |  2 +-
 reachable.c              |  8 ++---
 ref-filter.c             |  9 +++--
 refs.c                   |  2 +-
 remote.c                 | 19 +++++-----
 revision.c               | 12 +++----
 sequencer.c              | 14 ++++----
 server-info.c            |  2 +-
 sha1-file.c              |  4 +--
 sha1-name.c              | 23 ++++++------
 shallow.c                | 30 ++++++++++------
 submodule.c              | 10 +++---
 tag.c                    | 27 +++++++-------
 tag.h                    |  7 ++--
 tree.c                   | 14 ++++----
 tree.h                   |  2 +-
 upload-pack.c            |  4 +--
 walker.c                 |  9 +++--
 wt-status.c              |  2 +-
 76 files changed, 400 insertions(+), 297 deletions(-)

-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 5%]

* [PATCH v2 08/31] commit: add repository argument to lookup_commit
  2018-06-13 23:04 ` [PATCH v2 00/31] object-store: lookup_commit Stefan Beller
  2018-06-13 23:04   ` [PATCH v2 07/31] commit: add repository argument to lookup_commit_reference Stefan Beller
@ 2018-06-13 23:04   ` Stefan Beller
  2018-06-14 18:27   ` [PATCH v2 00/31] object-store: lookup_commit Brandon Williams
  2 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-13 23:04 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

Add a repository argument to allow callers of lookup_commit to be more
specific about which repository to handle. This is a small mechanical
change; it doesn't change the implementation to handle repositories
other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/am.c            |  3 ++-
 builtin/commit-tree.c   |  4 +++-
 builtin/diff-tree.c     |  2 +-
 builtin/fast-export.c   |  2 +-
 builtin/fmt-merge-msg.c |  2 +-
 builtin/merge-base.c    |  2 +-
 builtin/verify-commit.c |  4 +++-
 commit-graph.c          | 10 +++++-----
 commit.c                |  7 ++++---
 commit.h                |  3 ++-
 fetch-pack.c            |  5 +++--
 log-tree.c              |  2 +-
 notes-utils.c           |  4 +++-
 object.c                |  2 +-
 sequencer.c             |  4 ++--
 sha1-name.c             |  2 +-
 shallow.c               | 17 ++++++++++-------
 tag.c                   |  2 +-
 tree.c                  |  2 +-
 19 files changed, 46 insertions(+), 33 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index f4b510bcc5f..fb7c21f7103 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1633,7 +1633,8 @@ static void do_commit(const struct am_state *state)
 
 	if (!get_oid_commit("HEAD", &parent)) {
 		old_oid = &parent;
-		commit_list_insert(lookup_commit(&parent), &parents);
+		commit_list_insert(lookup_commit(the_repository, &parent),
+				   &parents);
 	} else {
 		old_oid = NULL;
 		say(state, stderr, _("applying to an empty history"));
diff --git a/builtin/commit-tree.c b/builtin/commit-tree.c
index 9fbd3529fb1..9ec36a82b63 100644
--- a/builtin/commit-tree.c
+++ b/builtin/commit-tree.c
@@ -6,6 +6,7 @@
 #include "cache.h"
 #include "config.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "tree.h"
 #include "builtin.h"
@@ -60,7 +61,8 @@ int cmd_commit_tree(int argc, const char **argv, const char *prefix)
 			if (get_oid_commit(argv[i], &oid))
 				die("Not a valid object name %s", argv[i]);
 			assert_oid_type(&oid, OBJ_COMMIT);
-			new_parent(lookup_commit(&oid), &parents);
+			new_parent(lookup_commit(the_repository, &oid),
+						 &parents);
 			continue;
 		}
 
diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index a5718d96ee2..91ba67070e2 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -25,7 +25,7 @@ static int stdin_diff_commit(struct commit *commit, const char *p)
 
 	/* Graft the fake parents locally to the commit */
 	while (isspace(*p++) && !parse_oid_hex(p, &oid, &p)) {
-		struct commit *parent = lookup_commit(&oid);
+		struct commit *parent = lookup_commit(the_repository, &oid);
 		if (!pptr) {
 			/* Free the real parent list */
 			free_commit_list(commit->parents);
diff --git a/builtin/fast-export.c b/builtin/fast-export.c
index 23ca46e6008..bc837d2593c 100644
--- a/builtin/fast-export.c
+++ b/builtin/fast-export.c
@@ -964,7 +964,7 @@ static void import_marks(char *input_file)
 			/* only commits */
 			continue;
 
-		commit = lookup_commit(&oid);
+		commit = lookup_commit(the_repository, &oid);
 		if (!commit)
 			die("not a commit? can't happen: %s", oid_to_hex(&oid));
 
diff --git a/builtin/fmt-merge-msg.c b/builtin/fmt-merge-msg.c
index 5e44589b545..36318ef46e7 100644
--- a/builtin/fmt-merge-msg.c
+++ b/builtin/fmt-merge-msg.c
@@ -572,7 +572,7 @@ static void find_merge_parents(struct merge_parents *result,
 		commit_list_insert(parent, &parents);
 		add_merge_parent(result, &obj->oid, &parent->object.oid);
 	}
-	head_commit = lookup_commit(head);
+	head_commit = lookup_commit(the_repository, head);
 	if (head_commit)
 		commit_list_insert(head_commit, &parents);
 	reduce_heads_replace(&parents);
diff --git a/builtin/merge-base.c b/builtin/merge-base.c
index bbead6f33e5..08d91b1f0c0 100644
--- a/builtin/merge-base.c
+++ b/builtin/merge-base.c
@@ -124,7 +124,7 @@ static void add_one_commit(struct object_id *oid, struct rev_collect *revs)
 	if (is_null_oid(oid))
 		return;
 
-	commit = lookup_commit(oid);
+	commit = lookup_commit(the_repository, oid);
 	if (!commit ||
 	    (commit->object.flags & TMP_MARK) ||
 	    parse_commit(commit))
diff --git a/builtin/verify-commit.c b/builtin/verify-commit.c
index f6922da16d6..7772c07ed7a 100644
--- a/builtin/verify-commit.c
+++ b/builtin/verify-commit.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "builtin.h"
 #include "object-store.h"
+#include "repository.h"
 #include "commit.h"
 #include "run-command.h"
 #include <signal.h>
@@ -27,7 +28,8 @@ static int run_gpg_verify(const struct object_id *oid, const char *buf, unsigned
 
 	memset(&signature_check, 0, sizeof(signature_check));
 
-	ret = check_commit_signature(lookup_commit(oid), &signature_check);
+	ret = check_commit_signature(lookup_commit(the_repository, oid),
+				     &signature_check);
 	print_signature_buffer(&signature_check, flags);
 
 	signature_check_clear(&signature_check);
diff --git a/commit-graph.c b/commit-graph.c
index 5345cc90ed1..6c08fc0a9ba 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -241,7 +241,7 @@ static struct commit_list **insert_parent_or_die(struct commit_graph *g,
 	struct commit *c;
 	struct object_id oid;
 	hashcpy(oid.hash, g->chunk_oid_lookup + g->hash_len * pos);
-	c = lookup_commit(&oid);
+	c = lookup_commit(the_repository, &oid);
 	if (!c)
 		die("could not find commit %s", oid_to_hex(&oid));
 	c->graph_pos = pos;
@@ -527,7 +527,7 @@ static void close_reachable(struct packed_oid_list *oids)
 	struct commit *commit;
 
 	for (i = 0; i < oids->nr; i++) {
-		commit = lookup_commit(&oids->list[i]);
+		commit = lookup_commit(the_repository, &oids->list[i]);
 		if (commit)
 			commit->object.flags |= UNINTERESTING;
 	}
@@ -538,14 +538,14 @@ static void close_reachable(struct packed_oid_list *oids)
 	 * closure.
 	 */
 	for (i = 0; i < oids->nr; i++) {
-		commit = lookup_commit(&oids->list[i]);
+		commit = lookup_commit(the_repository, &oids->list[i]);
 
 		if (commit && !parse_commit(commit))
 			add_missing_parents(oids, commit);
 	}
 
 	for (i = 0; i < oids->nr; i++) {
-		commit = lookup_commit(&oids->list[i]);
+		commit = lookup_commit(the_repository, &oids->list[i]);
 
 		if (commit)
 			commit->object.flags &= ~UNINTERESTING;
@@ -658,7 +658,7 @@ void write_commit_graph(const char *obj_dir,
 		if (i > 0 && !oidcmp(&oids.list[i-1], &oids.list[i]))
 			continue;
 
-		commits.list[commits.nr] = lookup_commit(&oids.list[i]);
+		commits.list[commits.nr] = lookup_commit(the_repository, &oids.list[i]);
 		parse_commit(commits.list[commits.nr]);
 
 		for (parent = commits.list[commits.nr]->parents;
diff --git a/commit.c b/commit.c
index e031a6c3175..5e3f18801a1 100644
--- a/commit.c
+++ b/commit.c
@@ -51,7 +51,7 @@ struct commit *lookup_commit_or_die(const struct object_id *oid, const char *ref
 	return c;
 }
 
-struct commit *lookup_commit(const struct object_id *oid)
+struct commit *lookup_commit_the_repository(const struct object_id *oid)
 {
 	struct object *obj = lookup_object(the_repository, oid->hash);
 	if (!obj)
@@ -373,7 +373,7 @@ int parse_commit_buffer(struct commit *item, const void *buffer, unsigned long s
 		 */
 		if (graft && (graft->nr_parent < 0 || grafts_replace_parents))
 			continue;
-		new_parent = lookup_commit(&parent);
+		new_parent = lookup_commit(the_repository, &parent);
 		if (new_parent)
 			pptr = &commit_list_insert(new_parent, pptr)->next;
 	}
@@ -381,7 +381,8 @@ int parse_commit_buffer(struct commit *item, const void *buffer, unsigned long s
 		int i;
 		struct commit *new_parent;
 		for (i = 0; i < graft->nr_parent; i++) {
-			new_parent = lookup_commit(&graft->parent[i]);
+			new_parent = lookup_commit(the_repository,
+						   &graft->parent[i]);
 			if (!new_parent)
 				continue;
 			pptr = &commit_list_insert(new_parent, pptr)->next;
diff --git a/commit.h b/commit.h
index a5f84466efd..431a7d97a24 100644
--- a/commit.h
+++ b/commit.h
@@ -53,7 +53,8 @@ enum decoration_type {
 void add_name_decoration(enum decoration_type type, const char *name, struct object *obj);
 const struct name_decoration *get_name_decoration(const struct object *obj);
 
-struct commit *lookup_commit(const struct object_id *oid);
+#define lookup_commit(r, o) lookup_commit_##r(o)
+struct commit *lookup_commit_the_repository(const struct object_id *oid);
 #define lookup_commit_reference(r, o) \
 		lookup_commit_reference_##r(o)
 struct commit *lookup_commit_reference_the_repository(const struct object_id *oid);
diff --git a/fetch-pack.c b/fetch-pack.c
index 4523e25ff4e..74ac2977e85 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -498,7 +498,8 @@ static int find_common(struct fetch_pack_args *args,
 				case ACK_ready:
 				case ACK_continue: {
 					struct commit *commit =
-						lookup_commit(result_oid);
+						lookup_commit(the_repository,
+							      result_oid);
 					if (!commit)
 						die(_("invalid commit %s"), oid_to_hex(result_oid));
 					if (args->stateless_rpc
@@ -1256,7 +1257,7 @@ static int process_acks(struct packet_reader *reader, struct oidset *common)
 			if (!get_oid_hex(arg, &oid)) {
 				struct commit *commit;
 				oidset_insert(common, &oid);
-				commit = lookup_commit(&oid);
+				commit = lookup_commit(the_repository, &oid);
 				mark_common(commit, 0, 1);
 			}
 			continue;
diff --git a/log-tree.c b/log-tree.c
index e8d2857bf06..a47283eca64 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -147,7 +147,7 @@ static int add_ref_decoration(const char *refname, const struct object_id *oid,
 
 static int add_graft_decoration(const struct commit_graft *graft, void *cb_data)
 {
-	struct commit *commit = lookup_commit(&graft->oid);
+	struct commit *commit = lookup_commit(the_repository, &graft->oid);
 	if (!commit)
 		return 0;
 	add_name_decoration(DECORATION_GRAFTED, "grafted", &commit->object);
diff --git a/notes-utils.c b/notes-utils.c
index 02407fe2a73..14ea03178e9 100644
--- a/notes-utils.c
+++ b/notes-utils.c
@@ -3,6 +3,7 @@
 #include "commit.h"
 #include "refs.h"
 #include "notes-utils.h"
+#include "repository.h"
 
 void create_notes_commit(struct notes_tree *t, struct commit_list *parents,
 			 const char *msg, size_t msg_len,
@@ -19,7 +20,8 @@ void create_notes_commit(struct notes_tree *t, struct commit_list *parents,
 		/* Deduce parent commit from t->ref */
 		struct object_id parent_oid;
 		if (!read_ref(t->ref, &parent_oid)) {
-			struct commit *parent = lookup_commit(&parent_oid);
+			struct commit *parent = lookup_commit(the_repository,
+							      &parent_oid);
 			if (parse_commit(parent))
 				die("Failed to find/parse commit %s", t->ref);
 			commit_list_insert(parent, &parents);
diff --git a/object.c b/object.c
index 1a67d6e341f..4eca16f4adb 100644
--- a/object.c
+++ b/object.c
@@ -212,7 +212,7 @@ struct object *parse_object_buffer_the_repository(const struct object_id *oid, e
 			}
 		}
 	} else if (type == OBJ_COMMIT) {
-		struct commit *commit = lookup_commit(oid);
+		struct commit *commit = lookup_commit(the_repository, oid);
 		if (commit) {
 			if (parse_commit_buffer(commit, buffer, size))
 				return NULL;
diff --git a/sequencer.c b/sequencer.c
index 8931c461f20..da6c9fc5511 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -543,7 +543,7 @@ static int is_index_unchanged(void)
 	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, &head_oid, NULL))
 		return error(_("could not resolve HEAD commit"));
 
-	head_commit = lookup_commit(&head_oid);
+	head_commit = lookup_commit(the_repository, &head_oid);
 
 	/*
 	 * If head_commit is NULL, check_commit, called from
@@ -972,7 +972,7 @@ void print_commit_summary(const char *prefix, const struct object_id *oid,
 	struct strbuf author_ident = STRBUF_INIT;
 	struct strbuf committer_ident = STRBUF_INIT;
 
-	commit = lookup_commit(oid);
+	commit = lookup_commit(the_repository, oid);
 	if (!commit)
 		die(_("couldn't look up newly created commit"));
 	if (parse_commit(commit))
diff --git a/sha1-name.c b/sha1-name.c
index d153d8c692b..5eef8ddd6d6 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -352,7 +352,7 @@ static int show_ambiguous_object(const struct object_id *oid, void *data)
 
 	type = oid_object_info(the_repository, oid, NULL);
 	if (type == OBJ_COMMIT) {
-		struct commit *commit = lookup_commit(oid);
+		struct commit *commit = lookup_commit(the_repository, oid);
 		if (commit) {
 			struct pretty_print_context pp = {0};
 			pp.date_mode.type = DATE_SHORT;
diff --git a/shallow.c b/shallow.c
index 9bb07a56dca..60fe1fe1e58 100644
--- a/shallow.c
+++ b/shallow.c
@@ -31,7 +31,7 @@ int register_shallow(struct repository *r, const struct object_id *oid)
 {
 	struct commit_graft *graft =
 		xmalloc(sizeof(struct commit_graft));
-	struct commit *commit = lookup_commit(oid);
+	struct commit *commit = lookup_commit(the_repository, oid);
 
 	oidcpy(&graft->oid, oid);
 	graft->nr_parent = -1;
@@ -256,7 +256,7 @@ static int write_one_shallow(const struct commit_graft *graft, void *cb_data)
 	if (graft->nr_parent != -1)
 		return 0;
 	if (data->flags & SEEN_ONLY) {
-		struct commit *c = lookup_commit(&graft->oid);
+		struct commit *c = lookup_commit(the_repository, &graft->oid);
 		if (!c || !(c->object.flags & SEEN)) {
 			if (data->flags & VERBOSE)
 				printf("Removing %s from .git/shallow\n",
@@ -621,7 +621,8 @@ void assign_shallow_commits_to_refs(struct shallow_info *info,
 
 	/* Mark potential bottoms so we won't go out of bound */
 	for (i = 0; i < nr_shallow; i++) {
-		struct commit *c = lookup_commit(&oid[shallow[i]]);
+		struct commit *c = lookup_commit(the_repository,
+						 &oid[shallow[i]]);
 		c->object.flags |= BOTTOM;
 	}
 
@@ -632,7 +633,8 @@ void assign_shallow_commits_to_refs(struct shallow_info *info,
 		int bitmap_size = DIV_ROUND_UP(pi.nr_bits, 32) * sizeof(uint32_t);
 		memset(used, 0, sizeof(*used) * info->shallow->nr);
 		for (i = 0; i < nr_shallow; i++) {
-			const struct commit *c = lookup_commit(&oid[shallow[i]]);
+			const struct commit *c = lookup_commit(the_repository,
+							       &oid[shallow[i]]);
 			uint32_t **map = ref_bitmap_at(&pi.ref_bitmap, c);
 			if (*map)
 				used[shallow[i]] = xmemdupz(*map, bitmap_size);
@@ -702,7 +704,7 @@ static void post_assign_shallow(struct shallow_info *info,
 	for (i = dst = 0; i < info->nr_theirs; i++) {
 		if (i != dst)
 			info->theirs[dst] = info->theirs[i];
-		c = lookup_commit(&oid[info->theirs[i]]);
+		c = lookup_commit(the_repository, &oid[info->theirs[i]]);
 		bitmap = ref_bitmap_at(ref_bitmap, c);
 		if (!*bitmap)
 			continue;
@@ -723,7 +725,7 @@ static void post_assign_shallow(struct shallow_info *info,
 	for (i = dst = 0; i < info->nr_ours; i++) {
 		if (i != dst)
 			info->ours[dst] = info->ours[i];
-		c = lookup_commit(&oid[info->ours[i]]);
+		c = lookup_commit(the_repository, &oid[info->ours[i]]);
 		bitmap = ref_bitmap_at(ref_bitmap, c);
 		if (!*bitmap)
 			continue;
@@ -745,7 +747,8 @@ static void post_assign_shallow(struct shallow_info *info,
 int delayed_reachability_test(struct shallow_info *si, int c)
 {
 	if (si->need_reachability_test[c]) {
-		struct commit *commit = lookup_commit(&si->shallow->oid[c]);
+		struct commit *commit = lookup_commit(the_repository,
+						      &si->shallow->oid[c]);
 
 		if (!si->commits) {
 			struct commit_array ca;
diff --git a/tag.c b/tag.c
index 310102150ab..5dcdf7bf6f9 100644
--- a/tag.c
+++ b/tag.c
@@ -158,7 +158,7 @@ int parse_tag_buffer(struct tag *item, const void *data, unsigned long size)
 	} else if (!strcmp(type, tree_type)) {
 		item->tagged = (struct object *)lookup_tree(the_repository, &oid);
 	} else if (!strcmp(type, commit_type)) {
-		item->tagged = (struct object *)lookup_commit(&oid);
+		item->tagged = (struct object *)lookup_commit(the_repository, &oid);
 	} else if (!strcmp(type, tag_type)) {
 		item->tagged = (struct object *)lookup_tag(&oid);
 	} else {
diff --git a/tree.c b/tree.c
index fbc8d4bc653..33063b8dde0 100644
--- a/tree.c
+++ b/tree.c
@@ -101,7 +101,7 @@ static int read_tree_1(struct tree *tree, struct strbuf *base,
 		else if (S_ISGITLINK(entry.mode)) {
 			struct commit *commit;
 
-			commit = lookup_commit(entry.oid);
+			commit = lookup_commit(the_repository, entry.oid);
 			if (!commit)
 				die("Commit %s in submodule path %s%s not found",
 				    oid_to_hex(entry.oid),
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 3%]

* [PATCH v2 07/31] commit: add repository argument to lookup_commit_reference
  2018-06-13 23:04 ` [PATCH v2 00/31] object-store: lookup_commit Stefan Beller
@ 2018-06-13 23:04   ` Stefan Beller
  2018-06-13 23:04   ` [PATCH v2 08/31] commit: add repository argument to lookup_commit Stefan Beller
  2018-06-14 18:27   ` [PATCH v2 00/31] object-store: lookup_commit Brandon Williams
  2 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-13 23:04 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller

Add a repository argument to allow callers of lookup_commit_reference
to be more specific about which repository to handle. This is a small
mechanical change; it doesn't change the implementation to handle
repositories other than the_repository yet.

As with the previous commits, use a macro to catch callers passing a
repository other than the_repository at compile time.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 bisect.c              |  2 +-
 blame.c               |  2 +-
 branch.c              |  2 +-
 builtin/branch.c      |  7 ++++---
 builtin/clone.c       |  3 ++-
 builtin/describe.c    |  2 +-
 builtin/diff-tree.c   |  2 +-
 builtin/log.c         |  7 ++++---
 builtin/merge-base.c  |  5 +++--
 builtin/notes.c       |  3 ++-
 builtin/pull.c        | 15 ++++++++++-----
 builtin/reset.c       |  4 ++--
 builtin/rev-parse.c   |  6 +++---
 builtin/show-branch.c |  2 +-
 builtin/tag.c         |  2 +-
 bundle.c              |  3 ++-
 commit.c              |  6 +++---
 commit.h              |  4 +++-
 notes-merge.c         |  5 +++--
 parse-options-cb.c    |  2 +-
 remote.c              |  4 ++--
 revision.c            |  4 ++--
 sequencer.c           |  6 +++---
 sha1-name.c           |  4 ++--
 submodule.c           | 10 +++++-----
 25 files changed, 63 insertions(+), 49 deletions(-)

diff --git a/bisect.c b/bisect.c
index 6de1abd407b..e1275ba79e8 100644
--- a/bisect.c
+++ b/bisect.c
@@ -724,7 +724,7 @@ static int bisect_checkout(const struct object_id *bisect_rev, int no_checkout)
 
 static struct commit *get_commit_reference(const struct object_id *oid)
 {
-	struct commit *r = lookup_commit_reference(oid);
+	struct commit *r = lookup_commit_reference(the_repository, oid);
 	if (!r)
 		die(_("Not a valid commit name %s"), oid_to_hex(oid));
 	return r;
diff --git a/blame.c b/blame.c
index 726a7a76f20..86d0dd73338 100644
--- a/blame.c
+++ b/blame.c
@@ -119,7 +119,7 @@ static struct commit_list **append_parent(struct commit_list **tail, const struc
 {
 	struct commit *parent;
 
-	parent = lookup_commit_reference(oid);
+	parent = lookup_commit_reference(the_repository, oid);
 	if (!parent)
 		die("no such commit %s", oid_to_hex(oid));
 	return &commit_list_insert(parent, tail)->next;
diff --git a/branch.c b/branch.c
index 9b2742de32a..08d4efc1be6 100644
--- a/branch.c
+++ b/branch.c
@@ -301,7 +301,7 @@ void create_branch(const char *name, const char *start_name,
 		break;
 	}
 
-	if ((commit = lookup_commit_reference(&oid)) == NULL)
+	if ((commit = lookup_commit_reference(the_repository, &oid)) == NULL)
 		die(_("Not a valid branch point: '%s'."), start_name);
 	oidcpy(&oid, &commit->object.oid);
 
diff --git a/builtin/branch.c b/builtin/branch.c
index efc9ac1922c..fbbe311e270 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -126,7 +126,8 @@ static int branch_merged(int kind, const char *name,
 		    (reference_name = reference_name_to_free =
 		     resolve_refdup(upstream, RESOLVE_REF_READING,
 				    &oid, NULL)) != NULL)
-			reference_rev = lookup_commit_reference(&oid);
+			reference_rev = lookup_commit_reference(the_repository,
+								&oid);
 	}
 	if (!reference_rev)
 		reference_rev = head_rev;
@@ -159,7 +160,7 @@ static int check_branch_commit(const char *branchname, const char *refname,
 			       const struct object_id *oid, struct commit *head_rev,
 			       int kinds, int force)
 {
-	struct commit *rev = lookup_commit_reference(oid);
+	struct commit *rev = lookup_commit_reference(the_repository, oid);
 	if (!rev) {
 		error(_("Couldn't look up commit object for '%s'"), refname);
 		return -1;
@@ -213,7 +214,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 	}
 
 	if (!force) {
-		head_rev = lookup_commit_reference(&head_oid);
+		head_rev = lookup_commit_reference(the_repository, &head_oid);
 		if (!head_rev)
 			die(_("Couldn't look up commit object for HEAD"));
 	}
diff --git a/builtin/clone.c b/builtin/clone.c
index 2f10c468300..5b10ddfc832 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -695,7 +695,8 @@ static void update_head(const struct ref *our, const struct ref *remote,
 			install_branch_config(0, head, option_origin, our->name);
 		}
 	} else if (our) {
-		struct commit *c = lookup_commit_reference(&our->old_oid);
+		struct commit *c = lookup_commit_reference(the_repository,
+							   &our->old_oid);
 		/* --branch specifies a non-branch (i.e. tags), detach HEAD */
 		update_ref(msg, "HEAD", &c->object.oid, NULL, REF_NO_DEREF,
 			   UPDATE_REFS_DIE_ON_ERR);
diff --git a/builtin/describe.c b/builtin/describe.c
index f3dfd0228be..080b5ce082c 100644
--- a/builtin/describe.c
+++ b/builtin/describe.c
@@ -303,7 +303,7 @@ static void describe_commit(struct object_id *oid, struct strbuf *dst)
 	unsigned long seen_commits = 0;
 	unsigned int unannotated_cnt = 0;
 
-	cmit = lookup_commit_reference(oid);
+	cmit = lookup_commit_reference(the_repository, oid);
 
 	n = find_commit_name(&cmit->object.oid);
 	if (n && (tags || all || n->prio == 2)) {
diff --git a/builtin/diff-tree.c b/builtin/diff-tree.c
index 29901515a13..a5718d96ee2 100644
--- a/builtin/diff-tree.c
+++ b/builtin/diff-tree.c
@@ -11,7 +11,7 @@ static struct rev_info log_tree_opt;
 
 static int diff_tree_commit_oid(const struct object_id *oid)
 {
-	struct commit *commit = lookup_commit_reference(oid);
+	struct commit *commit = lookup_commit_reference(the_repository, oid);
 	if (!commit)
 		return -1;
 	return log_tree_commit(&log_tree_opt, commit);
diff --git a/builtin/log.c b/builtin/log.c
index 063057e45de..c962d22707b 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -907,8 +907,8 @@ static void get_patch_ids(struct rev_info *rev, struct patch_ids *ids)
 	o2 = rev->pending.objects[1].item;
 	flags1 = o1->flags;
 	flags2 = o2->flags;
-	c1 = lookup_commit_reference(&o1->oid);
-	c2 = lookup_commit_reference(&o2->oid);
+	c1 = lookup_commit_reference(the_repository, &o1->oid);
+	c2 = lookup_commit_reference(the_repository, &o2->oid);
 
 	if ((flags1 & UNINTERESTING) == (flags2 & UNINTERESTING))
 		die(_("Not a range."));
@@ -1864,7 +1864,8 @@ static int add_pending_commit(const char *arg, struct rev_info *revs, int flags)
 {
 	struct object_id oid;
 	if (get_oid(arg, &oid) == 0) {
-		struct commit *commit = lookup_commit_reference(&oid);
+		struct commit *commit = lookup_commit_reference(the_repository,
+								&oid);
 		if (commit) {
 			commit->object.flags |= flags;
 			add_pending_object(revs, &commit->object, arg);
diff --git a/builtin/merge-base.c b/builtin/merge-base.c
index 3b7600150b6..bbead6f33e5 100644
--- a/builtin/merge-base.c
+++ b/builtin/merge-base.c
@@ -6,6 +6,7 @@
 #include "diff.h"
 #include "revision.h"
 #include "parse-options.h"
+#include "repository.h"
 
 static int show_merge_base(struct commit **rev, int rev_nr, int show_all)
 {
@@ -42,7 +43,7 @@ static struct commit *get_commit_reference(const char *arg)
 
 	if (get_oid(arg, &revkey))
 		die("Not a valid object name %s", arg);
-	r = lookup_commit_reference(&revkey);
+	r = lookup_commit_reference(the_repository, &revkey);
 	if (!r)
 		die("Not a valid commit name %s", arg);
 
@@ -171,7 +172,7 @@ static int handle_fork_point(int argc, const char **argv)
 	if (get_oid(commitname, &oid))
 		die("Not a valid object name: '%s'", commitname);
 
-	derived = lookup_commit_reference(&oid);
+	derived = lookup_commit_reference(the_repository, &oid);
 	memset(&revs, 0, sizeof(revs));
 	revs.initial = 1;
 	for_each_reflog_ent(refname, collect_one_reflog_ent, &revs);
diff --git a/builtin/notes.c b/builtin/notes.c
index 6011cd2a074..50963642075 100644
--- a/builtin/notes.c
+++ b/builtin/notes.c
@@ -12,6 +12,7 @@
 #include "builtin.h"
 #include "notes.h"
 #include "object-store.h"
+#include "repository.h"
 #include "blob.h"
 #include "pretty.h"
 #include "refs.h"
@@ -711,7 +712,7 @@ static int merge_commit(struct notes_merge_options *o)
 
 	if (get_oid("NOTES_MERGE_PARTIAL", &oid))
 		die(_("failed to read ref NOTES_MERGE_PARTIAL"));
-	else if (!(partial = lookup_commit_reference(&oid)))
+	else if (!(partial = lookup_commit_reference(the_repository, &oid)))
 		die(_("could not find commit from NOTES_MERGE_PARTIAL."));
 	else if (parse_commit(partial))
 		die(_("could not parse commit from NOTES_MERGE_PARTIAL."));
diff --git a/builtin/pull.c b/builtin/pull.c
index 324c4ffa1a7..ec3f90545dd 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -760,10 +760,13 @@ static int get_octopus_merge_base(struct object_id *merge_base,
 {
 	struct commit_list *revs = NULL, *result;
 
-	commit_list_insert(lookup_commit_reference(curr_head), &revs);
-	commit_list_insert(lookup_commit_reference(merge_head), &revs);
+	commit_list_insert(lookup_commit_reference(the_repository, curr_head),
+			   &revs);
+	commit_list_insert(lookup_commit_reference(the_repository, merge_head),
+			   &revs);
 	if (!is_null_oid(fork_point))
-		commit_list_insert(lookup_commit_reference(fork_point), &revs);
+		commit_list_insert(lookup_commit_reference(the_repository, fork_point),
+				   &revs);
 
 	result = get_octopus_merge_bases(revs);
 	free_commit_list(revs);
@@ -937,9 +940,11 @@ int cmd_pull(int argc, const char **argv, const char *prefix)
 			struct commit_list *list = NULL;
 			struct commit *merge_head, *head;
 
-			head = lookup_commit_reference(&orig_head);
+			head = lookup_commit_reference(the_repository,
+						       &orig_head);
 			commit_list_insert(head, &list);
-			merge_head = lookup_commit_reference(&merge_heads.oid[0]);
+			merge_head = lookup_commit_reference(the_repository,
+							     &merge_heads.oid[0]);
 			if (is_descendant_of(merge_head, list)) {
 				/* we can fast-forward this without invoking rebase */
 				opt_ff = "--ff-only";
diff --git a/builtin/reset.c b/builtin/reset.c
index a9136a163c0..aeab0c7aea5 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -319,7 +319,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 		struct commit *commit;
 		if (get_oid_committish(rev, &oid))
 			die(_("Failed to resolve '%s' as a valid revision."), rev);
-		commit = lookup_commit_reference(&oid);
+		commit = lookup_commit_reference(the_repository, &oid);
 		if (!commit)
 			die(_("Could not parse object '%s'."), rev);
 		oidcpy(&oid, &commit->object.oid);
@@ -396,7 +396,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 		update_ref_status = reset_refs(rev, &oid);
 
 		if (reset_type == HARD && !update_ref_status && !quiet)
-			print_new_head_line(lookup_commit_reference(&oid));
+			print_new_head_line(lookup_commit_reference(the_repository, &oid));
 	}
 	if (!pathspec.nr)
 		remove_branch_state();
diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c
index a8a9b506ff6..cc6a6c9a019 100644
--- a/builtin/rev-parse.c
+++ b/builtin/rev-parse.c
@@ -280,8 +280,8 @@ static int try_difference(const char *arg)
 		if (symmetric) {
 			struct commit_list *exclude;
 			struct commit *a, *b;
-			a = lookup_commit_reference(&start_oid);
-			b = lookup_commit_reference(&end_oid);
+			a = lookup_commit_reference(the_repository, &start_oid);
+			b = lookup_commit_reference(the_repository, &end_oid);
 			exclude = get_merge_bases(a, b);
 			while (exclude) {
 				struct commit *commit = pop_commit(&exclude);
@@ -333,7 +333,7 @@ static int try_parent_shorthands(const char *arg)
 		return 0;
 	}
 
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (exclude_parent &&
 	    exclude_parent > commit_list_count(commit->parents)) {
 		*dotdot = '^';
diff --git a/builtin/show-branch.c b/builtin/show-branch.c
index 2456b47d344..4b9d3c0059b 100644
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@ -831,7 +831,7 @@ int cmd_show_branch(int ac, const char **av, const char *prefix)
 			       MAX_REVS), MAX_REVS);
 		if (get_oid(ref_name[num_rev], &revkey))
 			die(_("'%s' is not a valid ref."), ref_name[num_rev]);
-		commit = lookup_commit_reference(&revkey);
+		commit = lookup_commit_reference(the_repository, &revkey);
 		if (!commit)
 			die(_("cannot find commit %s (%s)"),
 			    ref_name[num_rev], oid_to_hex(&revkey));
diff --git a/builtin/tag.c b/builtin/tag.c
index 9919b03b2d6..9a19ffb49f6 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -313,7 +313,7 @@ static void create_reflog_msg(const struct object_id *oid, struct strbuf *sb)
 		}
 		free(buf);
 
-		if ((c = lookup_commit_reference(oid)) != NULL)
+		if ((c = lookup_commit_reference(the_repository, oid)) != NULL)
 			strbuf_addf(sb, ", %s", show_date(c->date, 0, DATE_MODE(SHORT)));
 		break;
 	case OBJ_TREE:
diff --git a/bundle.c b/bundle.c
index 14a209ff149..904b59e72a5 100644
--- a/bundle.c
+++ b/bundle.c
@@ -375,7 +375,8 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
 			 * in terms of a tag (e.g. v2.0 from the range
 			 * "v1.0..v2.0")?
 			 */
-			struct commit *one = lookup_commit_reference(&oid);
+			struct commit *one = lookup_commit_reference(the_repository,
+								     &oid);
 			struct object *obj;
 
 			if (e->item == &(one->object)) {
diff --git a/commit.c b/commit.c
index b039444aa62..e031a6c3175 100644
--- a/commit.c
+++ b/commit.c
@@ -34,14 +34,14 @@ struct commit *lookup_commit_reference_gently_the_repository(
 	return object_as_type(the_repository, obj, OBJ_COMMIT, quiet);
 }
 
-struct commit *lookup_commit_reference(const struct object_id *oid)
+struct commit *lookup_commit_reference_the_repository(const struct object_id *oid)
 {
 	return lookup_commit_reference_gently(the_repository, oid, 0);
 }
 
 struct commit *lookup_commit_or_die(const struct object_id *oid, const char *ref_name)
 {
-	struct commit *c = lookup_commit_reference(oid);
+	struct commit *c = lookup_commit_reference(the_repository, oid);
 	if (!c)
 		die(_("could not parse %s"), ref_name);
 	if (oidcmp(oid, &c->object.oid)) {
@@ -67,7 +67,7 @@ struct commit *lookup_commit_reference_by_name(const char *name)
 
 	if (get_oid_committish(name, &oid))
 		return NULL;
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (parse_commit(commit))
 		return NULL;
 	return commit;
diff --git a/commit.h b/commit.h
index 73ccdd8005b..a5f84466efd 100644
--- a/commit.h
+++ b/commit.h
@@ -54,7 +54,9 @@ void add_name_decoration(enum decoration_type type, const char *name, struct obj
 const struct name_decoration *get_name_decoration(const struct object *obj);
 
 struct commit *lookup_commit(const struct object_id *oid);
-struct commit *lookup_commit_reference(const struct object_id *oid);
+#define lookup_commit_reference(r, o) \
+		lookup_commit_reference_##r(o)
+struct commit *lookup_commit_reference_the_repository(const struct object_id *oid);
 #define lookup_commit_reference_gently(r, o, q) \
 		lookup_commit_reference_gently_##r(o, q)
 struct commit *lookup_commit_reference_gently_the_repository(
diff --git a/notes-merge.c b/notes-merge.c
index fd96c5ab3d6..2572e435432 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -2,6 +2,7 @@
 #include "commit.h"
 #include "refs.h"
 #include "object-store.h"
+#include "repository.h"
 #include "diff.h"
 #include "diffcore.h"
 #include "xdiff-interface.h"
@@ -553,7 +554,7 @@ int notes_merge(struct notes_merge_options *o,
 	else if (!check_refname_format(o->local_ref, 0) &&
 		is_null_oid(&local_oid))
 		local = NULL; /* local_oid == null_oid indicates unborn ref */
-	else if (!(local = lookup_commit_reference(&local_oid)))
+	else if (!(local = lookup_commit_reference(the_repository, &local_oid)))
 		die("Could not parse local commit %s (%s)",
 		    oid_to_hex(&local_oid), o->local_ref);
 	trace_printf("\tlocal commit: %.7s\n", oid_to_hex(&local_oid));
@@ -571,7 +572,7 @@ int notes_merge(struct notes_merge_options *o,
 			die("Failed to resolve remote notes ref '%s'",
 			    o->remote_ref);
 		}
-	} else if (!(remote = lookup_commit_reference(&remote_oid))) {
+	} else if (!(remote = lookup_commit_reference(the_repository, &remote_oid))) {
 		die("Could not parse remote commit %s (%s)",
 		    oid_to_hex(&remote_oid), o->remote_ref);
 	}
diff --git a/parse-options-cb.c b/parse-options-cb.c
index 0f9f311a7a9..e8236534ac8 100644
--- a/parse-options-cb.c
+++ b/parse-options-cb.c
@@ -91,7 +91,7 @@ int parse_opt_commits(const struct option *opt, const char *arg, int unset)
 		return -1;
 	if (get_oid(arg, &oid))
 		return error("malformed object name %s", arg);
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (!commit)
 		return error("no such commit %s", arg);
 	commit_list_insert(commit, opt->value);
diff --git a/remote.c b/remote.c
index 956fd0fd923..2a6ff7d60c1 100644
--- a/remote.c
+++ b/remote.c
@@ -2058,13 +2058,13 @@ int stat_tracking_info(struct branch *branch, int *num_ours, int *num_theirs,
 	/* Cannot stat if what we used to build on no longer exists */
 	if (read_ref(base, &oid))
 		return -1;
-	theirs = lookup_commit_reference(&oid);
+	theirs = lookup_commit_reference(the_repository, &oid);
 	if (!theirs)
 		return -1;
 
 	if (read_ref(branch->refname, &oid))
 		return -1;
-	ours = lookup_commit_reference(&oid);
+	ours = lookup_commit_reference(the_repository, &oid);
 	if (!ours)
 		return -1;
 
diff --git a/revision.c b/revision.c
index 79f228b1586..25df00cc16c 100644
--- a/revision.c
+++ b/revision.c
@@ -1581,8 +1581,8 @@ static int handle_dotdot_1(const char *arg, char *dotdot,
 		struct commit *a, *b;
 		struct commit_list *exclude;
 
-		a = lookup_commit_reference(&a_obj->oid);
-		b = lookup_commit_reference(&b_obj->oid);
+		a = lookup_commit_reference(the_repository, &a_obj->oid);
+		b = lookup_commit_reference(the_repository, &b_obj->oid);
 		if (!a || !b)
 			return dotdot_missing(arg, dotdot, revs, symmetric);
 
diff --git a/sequencer.c b/sequencer.c
index 4e89e2b1123..8931c461f20 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -1047,7 +1047,7 @@ static int parse_head(struct commit **head)
 	if (get_oid("HEAD", &oid)) {
 		current_head = NULL;
 	} else {
-		current_head = lookup_commit_reference(&oid);
+		current_head = lookup_commit_reference(the_repository, &oid);
 		if (!current_head)
 			return error(_("could not parse HEAD"));
 		if (oidcmp(&oid, &current_head->object.oid)) {
@@ -1370,7 +1370,7 @@ static int update_squash_messages(enum todo_command command,
 
 		if (get_oid("HEAD", &head))
 			return error(_("need a HEAD to fixup"));
-		if (!(head_commit = lookup_commit_reference(&head)))
+		if (!(head_commit = lookup_commit_reference(the_repository, &head)))
 			return error(_("could not read HEAD"));
 		if (!(head_message = get_commit_buffer(head_commit, NULL)))
 			return error(_("could not read HEAD's commit message"));
@@ -1824,7 +1824,7 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	if (status < 0)
 		return -1;
 
-	item->commit = lookup_commit_reference(&commit_oid);
+	item->commit = lookup_commit_reference(the_repository, &commit_oid);
 	return !item->commit;
 }
 
diff --git a/sha1-name.c b/sha1-name.c
index 412a5257d6f..d153d8c692b 100644
--- a/sha1-name.c
+++ b/sha1-name.c
@@ -810,7 +810,7 @@ static int get_parent(const char *name, int len,
 
 	if (ret)
 		return ret;
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (parse_commit(commit))
 		return -1;
 	if (!idx) {
@@ -838,7 +838,7 @@ static int get_nth_ancestor(const char *name, int len,
 	ret = get_oid_1(name, len, &oid, GET_OID_COMMITTISH);
 	if (ret)
 		return ret;
-	commit = lookup_commit_reference(&oid);
+	commit = lookup_commit_reference(the_repository, &oid);
 	if (!commit)
 		return -1;
 
diff --git a/submodule.c b/submodule.c
index 8fd8e5d178d..e45150a3f51 100644
--- a/submodule.c
+++ b/submodule.c
@@ -516,8 +516,8 @@ static void show_submodule_header(struct diff_options *o, const char *path,
 	 * Attempt to lookup the commit references, and determine if this is
 	 * a fast forward or fast backwards update.
 	 */
-	*left = lookup_commit_reference(one);
-	*right = lookup_commit_reference(two);
+	*left = lookup_commit_reference(the_repository, one);
+	*right = lookup_commit_reference(the_repository, two);
 
 	/*
 	 * Warn about missing commits in the submodule project, but only if
@@ -1800,9 +1800,9 @@ int merge_submodule(struct object_id *result, const char *path,
 		return 0;
 	}
 
-	if (!(commit_base = lookup_commit_reference(base)) ||
-	    !(commit_a = lookup_commit_reference(a)) ||
-	    !(commit_b = lookup_commit_reference(b))) {
+	if (!(commit_base = lookup_commit_reference(the_repository, base)) ||
+	    !(commit_a = lookup_commit_reference(the_repository, a)) ||
+	    !(commit_b = lookup_commit_reference(the_repository, b))) {
 		MERGE_WARNING(path, "commits not present");
 		return 0;
 	}
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 4%]

* Re: BUG: submodule code prints '(null)'
  2018-06-11 22:56   ` BUG: submodule code prints '(null)' Stefan Beller
@ 2018-06-14 15:15     ` Heiko Voigt
  0 siblings, 0 replies; 200+ results
From: Heiko Voigt @ 2018-06-14 15:15 UTC (permalink / raw)
  To: Stefan Beller; +Cc: Duy Nguyen, git, Kaartic Sivaraam

On Mon, Jun 11, 2018 at 03:56:16PM -0700, Stefan Beller wrote:
> On Sat, Jun 9, 2018 at 4:04 AM Duy Nguyen <pclouds@gmail.com> wrote:
> >
> > On Tue, Jun 05, 2018 at 05:31:41PM +0200, Duy Nguyen wrote:
> > > I do not know how to reproduce this (and didn't bother to look deeply
> > > into it after I found it was not a trivial fix) but one of my "git
> > > fetch" showed
> > >
> > > warning: Submodule in commit be2db96a6c506464525f588da59cade0cedddb5e
> > > at path: '(null)' collides with a submodule named the same. Skipping
> > > it.
> >
> > The problem is default_name_or_path() can return NULL when a submodule
> > is not populated. The fix could simply be printing path instead of
> > name (because we are talking about path in the commit message), like
> > below.
> >
> > But I don't really understand c68f837576 (implement fetching of moved
> > submodules - 2017-10-16), the commit that made this change, and not
> > sure if we should be reporting name here or path. Heiko?
> >
> > diff --git a/submodule.c b/submodule.c
> > index 939d6870ec..61c2177755 100644
> > --- a/submodule.c
> > +++ b/submodule.c
> > @@ -745,7 +745,7 @@ static void collect_changed_submodules_cb(struct diff_queue_struct *q,
> 
> [Not in the context of this patch, but in the code right before the
> context starts:]
> 
>             name = default_name_or_path(p->two->path);
>             /* make sure name does not collide with existing one */
>             submodule = submodule_from_name(the_repository, commit_oid, name);
>             if (submodule) {
> 
> Currently I see 4 callers of default_name_or_path and the other 3 except this
> one have checks against NULL in place, which is good.
> However I think we have to guard this even more, and have to check
> for !name before we call submodule_from_name.
> 
> It is technically ok to call submodule_from_name with a NULL name,
> but it is semantically broken, see the comment in config_from that
> is called from submodule_from_name:
> 
>     /*
>      * If any parameter except the cache is a NULL pointer just
>      * return the first submodule. Can be used to check whether
>      * there are any submodules parsed.
>      */
>     if (!treeish_name || !key) {
>         ...
> 
> 
> >                                 warning("Submodule in commit %s at path: "
> >                                         "'%s' collides with a submodule named "
> >                                         "the same. Skipping it.",
> > -                                       oid_to_hex(commit_oid), name);
> > +                                       oid_to_hex(commit_oid), p->two->path);
> 
> This is correct for the error message, both in terms of not crashing as well
> as correctness, we really need to report a *path* here and no the name_or_path,
> which  default_name_or_path gives.

Sorry for the late reply. I agree with Stefan, this change is correct,
since we are talking about the path in the warning message anyway. It
seems to me that this resulted from the name being the same as the path
in this location here.

I think we should report both here, if we can, the path and the name.

We are skipping the submodule if we can not get a name later:

                if (!name)     
                        continue;

I also agree, that it does not make sense to call submodule_from_name
with a NULL name. I think we should simply skip calling it in case the
name is NULL and then let the code later handle it.

E.g.: 

...
                        /* make sure name does not collide with existing one */
		        if (name)	
                            submodule = submodule_from_name(the_repository, commit_oid, name);
                        if (submodule) {
                                warning("Submodule in commit %s at path: "
...

Would you want to update your patch? Or should I put one on top?

Cheers Heiko

^ permalink raw reply	[relevance 8%]

* [PATCH] submodule: fix NULL correctness in renamed broken submodules
      [irrelevant] <CACsJy8Ab3HoVWSWOtCBRYcsnnHnpO-2oEfV60f=H15RuzwpWwQ@mail.gmail.com>
@ 2018-06-14 17:31 ` Stefan Beller
  2018-06-14 17:37   ` [PATCH] t5526: test recursive submodules when fetching moved submodules Stefan Beller
                     ` (2 more replies)
  0 siblings, 3 replies; 200+ results
From: Stefan Beller @ 2018-06-14 17:31 UTC (permalink / raw)
  To: pclouds; +Cc: git, hvoigt, kaartic.sivaraam, sbeller

When fetching with recursing into submodules, the fetch logic inspects
the superproject which submodules actually need to be fetched. This is
tricky for submodules that were renamed in the fetched range of commits.
This was implemented in c68f8375760 (implement fetching of moved
submodules, 2017-10-16), and this patch fixes a mistake in the logic
there.

When the warning is printed, the `name` might be NULL as
default_name_or_path can return NULL, so fix the warning to use the path
as obtained from the diff machinery, as that is not NULL.

While at it, make sure we only attempt to load the submodule if a git
directory of the submodule is found as default_name_or_path will return
NULL in case the git directory cannot be found. Note that passing NULL
to submodule_from_name is just a semantic error, as submodule_from_name
accepts NULL as a value, but then the return value is not the submodule
that was asked for, but some arbitrary other submodule. (Cf. 'config_from'
in submodule-config.c: "If any parameter except the cache is a NULL
pointer just return the first submodule. Can be used to check whether
there are any submodules parsed.")

Reported-by: Duy Nguyen <pclouds@gmail.com>
Helped-by: Duy Nguyen <pclouds@gmail.com>
Helped-by: Heiko Voigt <hvoigt@hvoigt.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/submodule.c b/submodule.c
index 939d6870ecd..0998ea23458 100644
--- a/submodule.c
+++ b/submodule.c
@@ -740,12 +740,14 @@ static void collect_changed_submodules_cb(struct diff_queue_struct *q,
 		else {
 			name = default_name_or_path(p->two->path);
 			/* make sure name does not collide with existing one */
-			submodule = submodule_from_name(the_repository, commit_oid, name);
+			if (name)
+				submodule = submodule_from_name(the_repository,
+								commit_oid, name);
 			if (submodule) {
 				warning("Submodule in commit %s at path: "
 					"'%s' collides with a submodule named "
 					"the same. Skipping it.",
-					oid_to_hex(commit_oid), name);
+					oid_to_hex(commit_oid), p->two->path);
 				name = NULL;
 			}
 		}
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 23%]

* [PATCH] t5526: test recursive submodules when fetching moved submodules
  2018-06-14 17:31 ` [PATCH] submodule: fix NULL correctness in renamed broken submodules Stefan Beller
@ 2018-06-14 17:37   ` Stefan Beller
  2018-06-14 20:23     ` Heiko Voigt
  2018-06-14 20:24   ` [PATCH] submodule: fix NULL correctness in renamed broken submodules Heiko Voigt
  2018-06-17 14:02   ` Kaartic Sivaraam
  2 siblings, 1 reply; 200+ results
From: Stefan Beller @ 2018-06-14 17:37 UTC (permalink / raw)
  To: sbeller; +Cc: git, hvoigt, kaartic.sivaraam, pclouds

The topic merged in 0c7ecb7c311 (Merge branch 'sb/submodule-move-nested',
2018-05-08) provided support for moving nested submodules.

Remove the NEEDSWORK comment and implement the nested submodules test as
the comment hinted at.

Signed-off-by: Stefan Beller <sbeller@google.com>
---

I found this when digging around for the previous patch.

Thanks,
Stefan

 t/t5526-fetch-submodules.sh | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 9cc4b569c05..359e03ff836 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -574,11 +574,7 @@ test_expect_success "fetch new commits when submodule got renamed" '
 	git clone . downstream_rename &&
 	(
 		cd downstream_rename &&
-		git submodule update --init &&
-# NEEDSWORK: we omitted --recursive for the submodule update here since
-# that does not work. See test 7001 for mv "moving nested submodules"
-# for details. Once that is fixed we should add the --recursive option
-# here.
+		git submodule update --init --recursive &&
 		git checkout -b rename &&
 		git mv submodule submodule_renamed &&
 		(
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 29%]

* Re: [PATCH v2 00/31] object-store: lookup_commit
  2018-06-13 23:04 ` [PATCH v2 00/31] object-store: lookup_commit Stefan Beller
  2018-06-13 23:04   ` [PATCH v2 07/31] commit: add repository argument to lookup_commit_reference Stefan Beller
  2018-06-13 23:04   ` [PATCH v2 08/31] commit: add repository argument to lookup_commit Stefan Beller
@ 2018-06-14 18:27   ` Brandon Williams
  2 siblings, 0 replies; 200+ results
From: Brandon Williams @ 2018-06-14 18:27 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git

On 06/13, Stefan Beller wrote:
> * removed mentions of cooci patches
> * added forward declaration of commit buffer slabs.
> * dropped 3 patches that add the repository to lookup_unkonwn_object,
>   parse_commit and parse_commit_gently, but were not converting those
>   functions. We'll convert these in the next series, as this series is
>   growing big already.
> * This series can be found as branch 'object-store-lookup-commit' on github,
>   it applies on top of nd/commit-util-to-slab merged with sb/object-store-grafts
> 
> v1, https://public-inbox.org/git/20180530004810.30076-1-sbeller@google.com/
> 
> This applies on the merge of nd/commit-util-to-slab and sb/object-store-grafts,
> and is available at http://github.com/stefanbeller/ as branch object-store-lookup-commit
> as the merge has some merge conflicts as well as syntactical conflicts (upload-pack.c
> and fetch-pack.c introduce new calls of functions that would want to take a repository struct
> in the object-store-grafts series)
> 
> As layed out in https://public-inbox.org/git/20180517225154.9200-1-sbeller@google.com/
> this is getting close to finishing the set of object store series though the last
> unfinished part of this RFC hints at new work on the plate:
> * To give this series a nice polish, we'd want to convert parse_commit, too.
>   But that requires the conversion of the new commit graph. Maybe we need
>   to split this series into 2. 
> * Once this is in good shape we can talk about converting parts of the revision
>   walking code,
> * which then can be used by the submodule code as the end goal for the
>   object store series.

I've taken a look at the series and it looks good.  I'm glad we're
getting closer to this set of series being completed.  Thanks for
pushing this through :)

-- 
Brandon Williams

^ permalink raw reply	[relevance 2%]

* Re: [PATCH] submodule: fix NULL correctness in renamed broken submodules
  2018-06-14 17:31 ` [PATCH] submodule: fix NULL correctness in renamed broken submodules Stefan Beller
  2018-06-14 17:37   ` [PATCH] t5526: test recursive submodules when fetching moved submodules Stefan Beller
@ 2018-06-14 20:24   ` Heiko Voigt
  2018-06-17 14:02   ` Kaartic Sivaraam
  2 siblings, 0 replies; 200+ results
From: Heiko Voigt @ 2018-06-14 20:24 UTC (permalink / raw)
  To: Stefan Beller; +Cc: pclouds, git, kaartic.sivaraam

Hi,

On Thu, Jun 14, 2018 at 10:31:07AM -0700, Stefan Beller wrote:
> When fetching with recursing into submodules, the fetch logic inspects
> the superproject which submodules actually need to be fetched. This is
> tricky for submodules that were renamed in the fetched range of commits.
> This was implemented in c68f8375760 (implement fetching of moved
> submodules, 2017-10-16), and this patch fixes a mistake in the logic
> there.
> 
> When the warning is printed, the `name` might be NULL as
> default_name_or_path can return NULL, so fix the warning to use the path
> as obtained from the diff machinery, as that is not NULL.
> 
> While at it, make sure we only attempt to load the submodule if a git
> directory of the submodule is found as default_name_or_path will return
> NULL in case the git directory cannot be found. Note that passing NULL
> to submodule_from_name is just a semantic error, as submodule_from_name
> accepts NULL as a value, but then the return value is not the submodule
> that was asked for, but some arbitrary other submodule. (Cf. 'config_from'
> in submodule-config.c: "If any parameter except the cache is a NULL
> pointer just return the first submodule. Can be used to check whether
> there are any submodules parsed.")
> 
> Reported-by: Duy Nguyen <pclouds@gmail.com>
> Helped-by: Duy Nguyen <pclouds@gmail.com>
> Helped-by: Heiko Voigt <hvoigt@hvoigt.net>
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---

Looks good to me.

Cheers Heiko

^ permalink raw reply	[relevance 5%]

* Re: [PATCH] t5526: test recursive submodules when fetching moved submodules
  2018-06-14 17:37   ` [PATCH] t5526: test recursive submodules when fetching moved submodules Stefan Beller
@ 2018-06-14 20:23     ` Heiko Voigt
  0 siblings, 0 replies; 200+ results
From: Heiko Voigt @ 2018-06-14 20:23 UTC (permalink / raw)
  To: Stefan Beller; +Cc: git, kaartic.sivaraam, pclouds

On Thu, Jun 14, 2018 at 10:37:30AM -0700, Stefan Beller wrote:
> The topic merged in 0c7ecb7c311 (Merge branch 'sb/submodule-move-nested',
> 2018-05-08) provided support for moving nested submodules.
> 
> Remove the NEEDSWORK comment and implement the nested submodules test as
> the comment hinted at.
> 
> Signed-off-by: Stefan Beller <sbeller@google.com>
> ---

Looks good to me.

Cheers Heiko

^ permalink raw reply	[relevance 5%]

* Re: fatal: could not reset submodule index
      [irrelevant] <DB6PR0101MB2344D682511891E4E9528598D97D0@DB6PR0101MB2344.eurprd01.prod.exchangelabs.com>
@ 2018-06-15 19:47 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-15 19:47 UTC (permalink / raw)
  To: awe; +Cc: git

> # Trying again with submodule.recurse
> git config --global submodule.recurse true
> git clone main clone4 cd clone4
> git checkout with-submodule
> # Submodule is still not automatically updated (sub folder is empty)
> # It seems like submodule.recurse does not affect git clone

That is true; submodule.recurse is not affecting git clone.
This was a design decision once it was introduced, as the git clone
might be too large. Maybe we need to revisit that decision and
just clone the submodules if submodule.recurse is set.

The problem that this bug report highlights, is the following:
Each branch has its own .gitmodules file and they can be
all different, however at clone time we only clone submodules
from the currently active branch.

So you could get your test case to pass with
    git clone --branch with-submodule main clone5

^ permalink raw reply	[relevance 8%]

* Re: [PATCH] submodule: fix NULL correctness in renamed broken submodules
  2018-06-14 17:31 ` [PATCH] submodule: fix NULL correctness in renamed broken submodules Stefan Beller
  2018-06-14 17:37   ` [PATCH] t5526: test recursive submodules when fetching moved submodules Stefan Beller
  2018-06-14 20:24   ` [PATCH] submodule: fix NULL correctness in renamed broken submodules Heiko Voigt
@ 2018-06-17 14:02   ` Kaartic Sivaraam
  2 siblings, 0 replies; 200+ results
From: Kaartic Sivaraam @ 2018-06-17 14:02 UTC (permalink / raw)
  To: Stefan Beller, pclouds; +Cc: git, hvoigt

[-- Attachment #1.1: Type: text/plain, Size: 2207 bytes --]

On Thursday 14 June 2018 11:01 PM, Stefan Beller wrote:
> While at it, make sure we only attempt to load the submodule if a git
> directory of the submodule is found as default_name_or_path will return
> NULL in case the git directory cannot be found.

I found this a little hard to read. Maybe it could be sentence could be
shrinked a little. Possibly,

    While at it, make sure we only attempt to load the submodule if
    a git directory of the submodule is found.

I guess the other part of the sentence doesn't make much sense in the
log message. Maybe it could be an in-code comment. Speaking of in-code
comment, the following would also fit there, wouldn't it?

> Note that passing NULL
> to submodule_from_name is just a semantic error, as submodule_from_name
> accepts NULL as a value, but then the return value is not the submodule
> that was asked for, but some arbitrary other submodule. (Cf. 'config_from'
> in submodule-config.c: "If any parameter except the cache is a NULL
> pointer just return the first submodule. Can be used to check whether
> there are any submodules parsed.")
> 


-- 
Sivaraam

QUOTE:

“The three principal virtues of a programmer are Laziness, Impatience,
and Hubris.”

	- Camel book

Sivaraam?

You possibly might have noticed that my signature recently changed from
'Kaartic' to 'Sivaraam' both of which are parts of my name. I find the
new signature to be better for several reasons one of which is that the
former signature has a lot of ambiguities in the place I live as it is a
common name (NOTE: it's not a common spelling, just a common name). So,
I switched signatures before it's too late.

That said, I won't mind you calling me 'Kaartic' if you like it [of
course ;-)]. You can always call me using either of the names.


KIND NOTE TO THE NATIVE ENGLISH SPEAKER:

As I'm not a native English speaker myself, there might be mistaeks in
my usage of English. I apologise for any mistakes that I make.

It would be "helpful" if you take the time to point out the mistakes.

It would be "super helpful" if you could provide suggestions about how
to correct those mistakes.

Thanks in advance!


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[relevance 7%]

* Re: need for `git submodule update` over `git pull --recurse-submodules`?
      [irrelevant] <CAH-HCWW2xUqO9HNfzaduXbn1PnwWinVJcTh9AG3Ubmq0LtAQLg@mail.gmail.com>
@ 2018-06-18 17:44 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-18 17:44 UTC (permalink / raw)
  To: samjnaa; +Cc: git

On Sun, Jun 17, 2018 at 8:41 PM Shriramana Sharma <samjnaa@gmail.com> wrote:

> Do I need to execute any `git submodule` commands separately even if I
> do `git pull --recurse-submodules`?

Ideally you don't need "git submodule" commands any more, the rest of git
is slowly converging to have builtin submodule functionality.

> All I want is to stay in sync with
> the cloned repo.

That should just work with "pull --recurse"

> But if so what is the use of `git submodule update --recursive` or
> `git submodule update --remote --rebase` or such, which is somewhat
> confusing to me I'm sorry to say.

The git-submodule command was the first command implemented that
dealt with submodules. In the beginning there was no "git pull --recurse"
but the only way was to run "git submodule update" to change
the state of submodules. Now there are better ways to do that, such as
the recursive pull.

Hope that helps,
Stefan

^ permalink raw reply	[relevance 9%]

* Re: [PATCH] t7406-submodule-update: fix broken &&-chains
      [irrelevant] <20180616203319.9558-1-szeder.dev@gmail.com>
@ 2018-06-18 17:48 ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-18 17:48 UTC (permalink / raw)
  To: SZEDER Gábor; +Cc: Junio C Hamano, git

On Sat, Jun 16, 2018 at 1:33 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>
> Three tests in 't7406-submodule-update' contain broken &&-chains, but
> since they are all in subshells, chain-lint couldn't notice them.
>
> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>

This looks good to me;

Thanks for spotting and fixing it,
Stefan

^ permalink raw reply	[relevance 5%]

* Re: [PATCH 03/23] midx: add midx builtin
  2018-06-11 21:02   ` [PATCH 03/23] midx: add midx builtin Stefan Beller
@ 2018-06-18 19:40     ` Derrick Stolee
  0 siblings, 0 replies; 200+ results
From: Derrick Stolee @ 2018-06-18 19:40 UTC (permalink / raw)
  To: Stefan Beller
  Cc: git, Derrick Stolee, Ævar Arnfjörð Bjarmason,
	Jonathan Nieder, Jonathan Tan, Martin Fick

On 6/11/2018 5:02 PM, Stefan Beller wrote:
> Hi Derrick,
> On Thu, Jun 7, 2018 at 7:03 AM Derrick Stolee <stolee@gmail.com> wrote:
>> This new 'git midx' builtin will be the plumbing access for writing,
>> reading, and checking multi-pack-index (MIDX) files. The initial
>> implementation is a no-op.
> Let's talk about the name for a second:
>
> .idx files are written by git-index-pack or as part of
> git-pack-objects (which just calls write_idx_file as part
> of finish_tmp_packfile), and the name actually suggests
> it writes the index files. I have a hard time understanding
> what the git-midx command does[1].
>
> With both commit graph as well as multi index we introduce
> a command that is centered around that concept (similar to
> git-remote or git-config that are centered around a concept,
> that is closely resembled by a file), but for indexes for packs
> it was integrated differently into Git. So I am not sure if I want
> to suggest to integrate it into the packfile commands as that
> doesn't really fit. But maybe we can have a name that is human
> readable instead of the file suffix? Maybe
>
>    git multi-pack-index ?
>
> I suppose that eventually this command is not really used by
> users as it will be used by other porcelain commands in the
> background or even as part of repack/gc so I am not worried
> about a long name, but I'd be more worried about understandability.

I'll use "git multi-pack-index" in v2. I'll keep "midx.c" in the root, 
though, if that is OK.

> [1] While these names are not perfect for the layman, it is okay?
>    I am sure you are aware of https://git-man-page-generator.lokaltog.net/

I was not, and enjoyed that quite a bit.

Thanks,
-Stolee

>
>
>> new file mode 100644
>> index 0000000000..2bd886f1a2
>> --- /dev/null
>> +++ b/Documentation/git-midx.txt
>> @@ -0,0 +1,29 @@
>> +git-midx(1)
>> +============
>> +
>> +NAME
>> +----
>> +git-midx - Write and verify multi-pack-indexes (MIDX files).
> The reading is done as part of all other commands.

I like to think the 'read' verb is a subset of "verify" because we are 
checking for information about the MIDX, and mostly for tests or debugging.

>
>> +
>> +
>> +SYNOPSIS
>> +--------
>> +[verse]
>> +'git midx' [--object-dir <dir>]
>> +
>> +DESCRIPTION
>> +-----------
>> +Write or verify a MIDX file.
>> +
>> +OPTIONS
>> +-------
>> +
>> +--object-dir <dir>::
>> +       Use given directory for the location of Git objects. We check
>> +       <dir>/packs/multi-pack-index for the current MIDX file, and
>> +       <dir>/packs for the pack-files to index.
>> +
>> +
> Maybe we could have a SEE ALSO section that points at
> the explanation of multi index files?
> (c.f. man git-submodule that has a  SEE ALSO
> gitsubmodules(7), gitmodules(5) explaining concepts(7)
> and the file(5))
>
> But as this is plumbing and users should not need to worry about it
> this is optional, I would think.

The design document is also in 'Documentation/technical' instead of just 
'Documentation/'. Do we have a pattern of linking to the technical 
documents?

Thanks,
-Stolee

^ permalink raw reply	[relevance 2%]

* [PATCH] t7400: encapsulate setup code in test_expect_success
  2018-06-12 21:36 ` Eric Sunshine
@ 2018-06-18 23:41   ` Stefan Beller
  0 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-18 23:41 UTC (permalink / raw)
  To: sunshine; +Cc: git, sbeller

When running t7400 in a shell you observe more output than expected:

    ...
    ok 8 - setup - hide init subdirectory
    ok 9 - setup - repository to add submodules to
    ok 10 - submodule add
    [master (root-commit) d79ce16] one
     Author: A U Thor <author@example.com>
     1 file changed, 1 insertion(+)
     create mode 100644 one.t
    ok 11 - redirected submodule add does not show progress
    ok 12 - redirected submodule add --progress does show progress
    ok 13 - submodule add to .gitignored path fails
    ...

Fix the output by encapsulating the setup code in test_expect_success

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 t/t7400-submodule-basic.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 2f532529b82..812db137b8d 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -126,8 +126,10 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
-test_create_repo parent &&
-test_commit -C parent one
+test_expect_success 'setup parent and one repository' '
+	test_create_repo parent &&
+	test_commit -C parent one
+'
 
 test_expect_success 'redirected submodule add does not show progress' '
 	git -C addtest submodule add "file://$submodurl/parent" submod-redirected \
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 19%]

* [PATCH 0/3] submodules with no working tree shall unset core.worktree
  2018-06-12 23:58 [RFC PATCH 0/3] submodules with no working tree shall unset core.worktree Stefan Beller
                   ` (3 preceding siblings ...)
  2018-06-13 18:00 ` [RFC PATCH 0/3] submodules with no working tree shall " Junio C Hamano
@ 2018-06-19  0:06 ` " Stefan Beller
  2018-06-19  0:06   ` [PATCH 1/3] submodule: unset core.worktree if no working tree is present Stefan Beller
                     ` (2 more replies)
  4 siblings, 3 replies; 200+ results
From: Stefan Beller @ 2018-06-19  0:06 UTC (permalink / raw)
  To: sbeller; +Cc: git

v2:
* fixed quoting issues for white space'd names/paths

v1:
The first patch teaches checkout/reset (with --recurse-submodules) to unset
the core.worktree config when the new state of the superprojects working tree
doesn't contain the submodules working tree.

The last patch is teaching "git submodule deinit" to unset the core.worktree
setting as well. It turned out this one is tricky, as for that we also
have to set it in the counter part, such as "submodule update".

Thanks,
Stefan

Stefan Beller (3):
  submodule: unset core.worktree if no working tree is present
  submodule: ensure core.worktree is set after update
  submodule deinit: unset core.worktree

 builtin/submodule--helper.c | 26 ++++++++++++++++++++++++++
 git-submodule.sh            |  5 +++++
 submodule.c                 | 14 ++++++++++++++
 submodule.h                 |  2 ++
 t/lib-submodule-update.sh   |  5 +++--
 t/t7400-submodule-basic.sh  |  5 +++++
 6 files changed, 55 insertions(+), 2 deletions(-)

-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 11%]

* [PATCH 1/3] submodule: unset core.worktree if no working tree is present
  2018-06-19  0:06 ` [PATCH " Stefan Beller
@ 2018-06-19  0:06   ` Stefan Beller
  2018-06-19  0:06   ` [PATCH 2/3] submodule: ensure core.worktree is set after update Stefan Beller
  2018-06-19  0:06   ` [PATCH 3/3] submodule deinit: unset core.worktree Stefan Beller
  2 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-19  0:06 UTC (permalink / raw)
  To: sbeller; +Cc: git

When a submodules work tree is removed, we should unset its core.worktree
setting as the worktree is no longer present. This is not just in line
with the conceptual view of submodules, but it fixes an inconvenience
for looking at submodules that are not checked out:

    git clone --recurse-submodules git://github.com/git/git && cd git &&
    git checkout --recurse-submodules v2.13.0
    git -C .git/modules/sha1collisiondetection log
    fatal: cannot chdir to '../../../sha1collisiondetection': \
        No such file or directory

With this patch applied, the final call to git log works instead of dying
in its setup, as the checkout will unset the core.worktree setting such
that following log will be run in a bare repository.

This patch covers all commands that are in the unpack machinery, i.e.
checkout, read-tree, reset. A follow up patch will address
"git submodule deinit", which will also make use of the new function
submodule_unset_core_worktree(), which is why we expose it in this patch.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 submodule.c               | 14 ++++++++++++++
 submodule.h               |  2 ++
 t/lib-submodule-update.sh |  3 ++-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/submodule.c b/submodule.c
index 939d6870ecd..e127c074b04 100644
--- a/submodule.c
+++ b/submodule.c
@@ -1532,6 +1532,18 @@ int bad_to_remove_submodule(const char *path, unsigned flags)
 	return ret;
 }
 
+void submodule_unset_core_worktree(const struct submodule *sub)
+{
+	char *config_path = xstrfmt("%s/modules/%s/config",
+				    get_git_common_dir(), sub->name);
+
+	if (git_config_set_in_file_gently(config_path, "core.worktree", NULL))
+		warning(_("Could not unset core.worktree setting in submodule '%s'"),
+			  sub->path);
+
+	free(config_path);
+}
+
 static const char *get_super_prefix_or_empty(void)
 {
 	const char *s = get_super_prefix();
@@ -1697,6 +1709,8 @@ int submodule_move_head(const char *path,
 
 			if (is_empty_dir(path))
 				rmdir_or_warn(path);
+
+			submodule_unset_core_worktree(sub);
 		}
 	}
 out:
diff --git a/submodule.h b/submodule.h
index 7856b8a0b3d..4644683e6cb 100644
--- a/submodule.h
+++ b/submodule.h
@@ -121,6 +121,8 @@ extern int submodule_move_head(const char *path,
 			       const char *new_head,
 			       unsigned flags);
 
+void submodule_unset_core_worktree(const struct submodule *sub);
+
 /*
  * Prepare the "env_array" parameter of a "struct child_process" for executing
  * a submodule by clearing any repo-specific environment variables, but
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 1f38a85371a..12cd4e9233e 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -709,7 +709,8 @@ test_submodule_recursing_with_args_common() {
 			git branch -t remove_sub1 origin/remove_sub1 &&
 			$command remove_sub1 &&
 			test_superproject_content origin/remove_sub1 &&
-			! test -e sub1
+			! test -e sub1 &&
+			test_must_fail git config -f .git/modules/sub1/config core.worktree
 		)
 	'
 	# ... absorbing a .git directory along the way.
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 20%]

* [PATCH 2/3] submodule: ensure core.worktree is set after update
  2018-06-19  0:06 ` [PATCH " Stefan Beller
  2018-06-19  0:06   ` [PATCH 1/3] submodule: unset core.worktree if no working tree is present Stefan Beller
@ 2018-06-19  0:06   ` Stefan Beller
  2018-06-19  0:06   ` [PATCH 3/3] submodule deinit: unset core.worktree Stefan Beller
  2 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-19  0:06 UTC (permalink / raw)
  To: sbeller; +Cc: git

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 24 ++++++++++++++++++++++++
 git-submodule.sh            |  5 +++++
 2 files changed, 29 insertions(+)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index bd250ca2164..dffc55ed8ee 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -1860,6 +1860,29 @@ static int check_name(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
+static int connect_gitdir_workingtree(int argc, const char **argv, const char *prefix)
+{
+	struct strbuf sb = STRBUF_INIT;
+	const char *name, *path;
+	char *sm_gitdir;
+
+	if (argc != 3)
+		BUG("submodule--helper connect-gitdir-workingtree <name> <path>");
+
+	name = argv[1];
+	path = argv[2];
+
+	strbuf_addf(&sb, "%s/modules/%s", get_git_dir(), name);
+	sm_gitdir = absolute_pathdup(sb.buf);
+
+	connect_work_tree_and_git_dir(path, sm_gitdir, 0);
+
+	strbuf_release(&sb);
+	free(sm_gitdir);
+
+	return 0;
+}
+
 #define SUPPORT_SUPER_PREFIX (1<<0)
 
 struct cmd_struct {
@@ -1873,6 +1896,7 @@ static struct cmd_struct commands[] = {
 	{"name", module_name, 0},
 	{"clone", module_clone, 0},
 	{"update-clone", update_clone, 0},
+	{"connect-gitdir-workingtree", connect_gitdir_workingtree, 0},
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url", resolve_relative_url, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
diff --git a/git-submodule.sh b/git-submodule.sh
index 78073cd87d1..6bd0db02b33 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -615,6 +615,11 @@ cmd_update()
 			die "$(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
 		fi
 
+		if ! $(git config -f "$(git rev-parse --git-common-dir)/modules/$name/config" core.worktree) 2>/dev/null
+		then
+			git submodule--helper connect-gitdir-workingtree "$name" "$sm_path"
+		fi
+
 		if test "$subsha1" != "$sha1" || test -n "$force"
 		then
 			subforce=$force
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 20%]

* [PATCH 3/3] submodule deinit: unset core.worktree
  2018-06-19  0:06 ` [PATCH " Stefan Beller
  2018-06-19  0:06   ` [PATCH 1/3] submodule: unset core.worktree if no working tree is present Stefan Beller
  2018-06-19  0:06   ` [PATCH 2/3] submodule: ensure core.worktree is set after update Stefan Beller
@ 2018-06-19  0:06   ` Stefan Beller
  2 siblings, 0 replies; 200+ results
From: Stefan Beller @ 2018-06-19  0:06 UTC (permalink / raw)
  To: sbeller; +Cc: git

When a submodule is deinit'd, the working tree is gone, so the setting of
core.worktree is bogus. Unset it.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 builtin/submodule--helper.c | 2 ++
 t/lib-submodule-update.sh   | 2 +-
 t/t7400-submodule-basic.sh  | 5 +++++
 3 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c
index dffc55ed8ee..19480902681 100644
--- a/builtin/submodule--helper.c
+++ b/builtin/submodule--helper.c
@@ -980,6 +980,8 @@ static void deinit_submodule(const char *path, const char *prefix,
 		if (!(flags & OPT_QUIET))
 			printf(format, displaypath);
 
+		submodule_unset_core_worktree(sub);
+
 		strbuf_release(&sb_rm);
 	}
 
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 12cd4e9233e..aa5ac03325a 100755
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -235,7 +235,7 @@ reset_work_tree_to_interested () {
 	then
 		mkdir -p submodule_update/.git/modules/sub1/modules &&
 		cp -r submodule_update_repo/.git/modules/sub1/modules/sub2 submodule_update/.git/modules/sub1/modules/sub2
-		GIT_WORK_TREE=. git -C submodule_update/.git/modules/sub1/modules/sub2 config --unset core.worktree
+		# core.worktree is unset for sub2 as it is not checked out
 	fi &&
 	# indicate we are interested in the submodule:
 	git -C submodule_update config submodule.sub1.url "bogus" &&
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 812db137b8d..48fd14fae6e 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -993,6 +993,11 @@ test_expect_success 'submodule deinit should remove the whole submodule section
 	rmdir init
 '
 
+test_expect_success 'submodule deinit should unset core.worktree' '
+	test_path_is_file .git/modules/example/config &&
+	test_must_fail git config -f .git/modules/example/config core.worktree
+'
+
 test_expect_success 'submodule deinit from subdirectory' '
 	git submodule update --init &&
 	git config submodule.example.foo bar &&
-- 
2.18.0.rc1.244.gcf134e6275-goog


^ permalink raw reply	[relevance 26%]

* Re: Branch switching with submodules where the submodule replaces a folder aborts unexpectedly
@ 2018-06-19 16:03 Sam Kuper
  0 siblings, 0 replies; 200+ results
From: Sam Kuper @ 2018-06-19 16:03 UTC (permalink / raw)
  To: git; +Cc: Stefan Beller, Thomas Braun

On 12 Oct 2017 at 11:48 Thomas Braun wrote:
> On 9 Oct 2017 at 23:59, Stefan Beller wrote:
>> On 9 Oct 2017 at 14:29, Thomas Braun wrote:
>>> I'm currently in the progress of pulling some subprojects in a git repository of mine into their
>>> own repositories and adding these subprojects back as submodules.
>>>
>>> While doing this I enountered a potential bug as checkout complains on branch switching that a
>>> file already exists. [...]
>>>
>>> `error: The following untracked working tree files would be overwritten by checkout:`

I am currently attempting the same thing, and experiencing the same
bug, using Git 2.17.1.


>> (And I presume you know about --recurse-submodules as a flag for git-checkout)

The behaviour seems to be the same regardless of whether I use the
--recurse-submodules flag with git-checkout.


>> This is consistent with our tests, unfortunately. [...]
>>
>>> If I'm misusing git here I'm glad for any advice.
>>
>> You are not.
>
> Glad to know that.

I was also glad to see this reassurance :)

It might have been nice if the reassurance came at an earlier stage,
however: at the CLI, rather than only after searching the mailing
list. A user who does not think to do the latter might well labour
under the misapprehension that they have done something wrong, rather
than encountered a bug. Perhaps, if the bug is not going to be fixed
terribly soon, a sentence or two could be added to the error message
explaining the situation and advising the user of a workaround?

Speaking of which, what is a good workaround in this case?

`git checkout --force <branch>`?


>> Apart from this bug report, would you think that such filtering of
>> trees into submodules (and back again) might be an interesting feature
>> of Git or are these cases rare and special?
>
> For me not particularly. In my case it is a one time thing going from an embedded project folder to a submodule.

The option to convert an existing directory into a submodule is
something that I think developers like to have available. It seems
intuitive: "Oh, I see now that what this directory holds is
effectively a separate project. Let me check out a new branch, and
replace the directory with a submodule on that branch. Assuming it
goes well, then I will afterwards merge this new branch into master."

Regardless, the bug has clearly been giving people headaches for
several years (forgive me if you were already aware of these data
points):

- https://ryansechrest.com/2014/03/git-error-switching-branch-replacing-directory-submodule/

- http://blog.dcycle.com/blog/105/gitsubmodulizing-and-gitflow/

- https://stackoverflow.com/q/9299063

- https://stackoverflow.com/a/48402543

- https://stackoverflow.com/q/24091246

- https://stackoverflow.com/q/29372450

- https://github.com/supercollider/supercollider/issues/2001

- https://github.com/supercollider/supercollider/issues/2221

Thank you to Thomas for reporting this issue to the mailing list, and
to Stefan for the helpful reply. Thanks as always to the Git
maintainers, and good luck with fixing this bug :)

^ permalink raw reply	[relevance 5%]

* The state of the object store series
@ 2018-06-19 22:07 Stefan Beller
      [irrelevant] ` <20180619223712.127990-1-jonathantanmy@google.com>
  2018-06-22  0:59 ` Derrick Stolee
  0 siblings, 2 replies; 200+ results
From: Stefan Beller @ 2018-06-19 22:07 UTC (