[PATCH 0/1] rebase: understand -C again, refactor

[PATCH 0/1] rebase: understand -C again, refactor

[Johannes Schindelin via GitGitGadget]

Phillip Wood reported a problem where the built-in rebase did not understand
options like -C1, i.e. it did not expect the option argument.

While investigating how to address this best, I stumbled upon 
OPT_PASSTHRU_ARGV (which I was so far happily unaware of).

Instead of just fixing the -C<n> bug, I decided to simply convert all of the
options intended for git am (or, eventually, for git apply). This happens to
fix that bug, and does so much more: it simplifies the entire logic (and
removes more lines than it adds).

Johannes Schindelin (1):
  rebase: really just passthru the `git am` options

 builtin/rebase.c | 98 +++++++++++++++++-------------------------------
 1 file changed, 35 insertions(+), 63 deletions(-)


base-commit: 8858448bb49332d353febc078ce4a3abcc962efe
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-76%2Fdscho%2Frebase-Cn-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-76/dscho/rebase-Cn-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/76
-- 
gitgitgadget

[PATCH 1/1] rebase: really just passthru the `git am` options

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Currently, we parse the options intended for `git am` as if we wanted to
handle them in `git rebase`, and then reconstruct them painstakingly to
define the `git_am_opt` variable.

However, there is a much better way (that I was unaware of, at the time
when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV.
It is intended for exactly this use case, where command-line options
want to be parsed into a separate `argv_array`.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip
Wood, where the built-in rebase failed to understand that the `-C`
option takes an optional argument.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase.c | 98 +++++++++++++++++-------------------------------
 1 file changed, 35 insertions(+), 63 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 0ee06aa363..96ffa80b71 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -87,7 +87,7 @@ struct rebase_options {
 		REBASE_FORCE = 1<<3,
 		REBASE_INTERACTIVE_EXPLICIT = 1<<4,
 	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
 	const char *action;
 	int signoff;
 	int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as resolved with\n"
 static int run_specific_rebase(struct rebase_options *opts)
 {
 	const char *argv[] = { NULL, NULL };
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
 	int status;
 	const char *backend, *backend_func;
 
@@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options *opts)
 		oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
 	add_var(&script_snippet, "GIT_QUIET",
 		opts->flags & REBASE_NO_QUIET ? "" : "t");
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
 	add_var(&script_snippet, "verbose",
 		opts->flags & REBASE_VERBOSE ? "t" : "");
 	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	struct rebase_options options = {
 		.type = REBASE_UNSPECIFIED,
 		.flags = REBASE_NO_QUIET,
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
 		.allow_rerere_autoupdate  = -1,
 		.allow_empty_message = 1,
 		.git_format_patch_opt = STRBUF_INIT,
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		ACTION_EDIT_TODO,
 		ACTION_SHOW_CURRENT_PATCH,
 	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
 	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
 	struct string_list exec = STRING_LIST_INIT_NODUP;
 	const char *rebase_merges = NULL;
 	int fork_point = -1;
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
 			N_("do not show diffstat of what changed upstream"),
 			PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
 		OPT_BOOL(0, "signoff", &options.signoff,
 			 N_("add a Signed-off-by: line to each commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace", &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"), 0),
 		OPT_BIT('f', "force-rebase", &options.flags,
 			N_("cherry-pick all commits, even if unchanged"),
 			REBASE_FORCE),
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
 			N_("GPG-sign commits"),
 			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
 		OPT_BOOL(0, "autostash", &options.autostash,
 			 N_("automatically stash/stash pop before and after")),
 		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 			 N_("rebase all reachable commits up to the root(s)")),
 		OPT_END(),
 	};
+	int i;
 
 	/*
 	 * NEEDSWORK: Once the builtin rebase has been tested enough
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		    state_dir_base, cmd_live_rebase, buf.buf);
 	}
 
-	if (!(options.flags & REBASE_NO_QUIET))
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
 	}
 
-	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
 	if (options.keep_empty)
 		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
 	}
 
-	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item, "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
 	if (exec.nr) {
 		int i;
 
@@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		break;
 	}
 
-	if (options.git_am_opt.len) {
-		const char *p;
-
+	if (options.git_am_opts.argc) {
 		/* all am options except -q are compatible only with --am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
 
-		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
 			die(_("error: cannot combine interactive options "
 			      "(--interactive, --exec, --rebase-merges, "
 			      "--preserve-merges, --keep-empty, --root + "
 			      "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
 			die(_("error: cannot combine merge options (--merge, "
 			      "--strategy, --strategy-option) with am options "
 			      "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		if (options.type == REBASE_PRESERVE_MERGES)
 			die("cannot combine '--signoff' with "
 			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
 		options.flags |= REBASE_FORCE;
 	}
 
-- 
gitgitgadget

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

[Junio C Hamano]

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> However, there is a much better way (that I was unaware of, at the time
> when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV.
> It is intended for exactly this use case, where command-line options
> want to be parsed into a separate `argv_array`.
>
> Let's use this feature.
>
> Incidentally, this also allows us to address a bug discovered by Phillip
> Wood, where the built-in rebase failed to understand that the `-C`
> option takes an optional argument.

The resulting code does show what is going on more clearly.  I
wonder if we can do the same for -S as well?  It needs to be passed
to other backends, so I guess it is not such a good idea.

> -		.git_am_opt = STRBUF_INIT,
> +		.git_am_opts = ARGV_ARRAY_INIT,

Yes, this is much nicer.

> @@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>  		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
>  			N_("GPG-sign commits"),
>  			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
> -		OPT_STRING_LIST(0, "whitespace", &whitespace,
> -				N_("whitespace"), N_("passed to 'git apply'")),
> -		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
> -			    REBASE_AM),
>  		OPT_BOOL(0, "autostash", &options.autostash,
>  			 N_("automatically stash/stash pop before and after")),
>  		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
> @@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>  			 N_("rebase all reachable commits up to the root(s)")),
>  		OPT_END(),
>  	};
> +	int i;
>  
> +	for (i = 0; i < options.git_am_opts.argc; i++) {

Yes, this is very nice way to first collect and then inspect them
separately.

> +		const char *option = options.git_am_opts.argv[i];
> +		if (!strcmp(option, "--committer-date-is-author-date") ||
> +		    !strcmp(option, "--ignore-date") ||
> +		    !strcmp(option, "--whitespace=fix") ||
> +		    !strcmp(option, "--whitespace=strip"))
> +			options.flags |= REBASE_FORCE;
>  	}

Overall very nicely done.  Perhaps we'll see a test or two from
Philip?

Thanks.

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

[Phillip Wood]

Hi Johannes

Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems 
to break the error reporting

Running
   bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad

Gives
   git encountered an error while preparing the patches to replay
   these revisions:

 
67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c

   As a result, git cannot rebase them.

If I do

   bin/wrappers/git rebase @^^ -Cbad

I get no error, it just tells me that it does not need to rebase (which 
is true)

Best Wishes

Phillip


On 13/11/2018 12:38, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> Currently, we parse the options intended for `git am` as if we wanted to
> handle them in `git rebase`, and then reconstruct them painstakingly to
> define the `git_am_opt` variable.
> 
> However, there is a much better way (that I was unaware of, at the time
> when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV.
> It is intended for exactly this use case, where command-line options
> want to be parsed into a separate `argv_array`.
> 
> Let's use this feature.
> 
> Incidentally, this also allows us to address a bug discovered by Phillip
> Wood, where the built-in rebase failed to understand that the `-C`
> option takes an optional argument.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>   builtin/rebase.c | 98 +++++++++++++++++-------------------------------
>   1 file changed, 35 insertions(+), 63 deletions(-)
> 
> diff --git a/builtin/rebase.c b/builtin/rebase.c
> index 0ee06aa363..96ffa80b71 100644
> --- a/builtin/rebase.c
> +++ b/builtin/rebase.c
> @@ -87,7 +87,7 @@ struct rebase_options {
>   		REBASE_FORCE = 1<<3,
>   		REBASE_INTERACTIVE_EXPLICIT = 1<<4,
>   	} flags;
> -	struct strbuf git_am_opt;
> +	struct argv_array git_am_opts;
>   	const char *action;
>   	int signoff;
>   	int allow_rerere_autoupdate;
> @@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as resolved with\n"
>   static int run_specific_rebase(struct rebase_options *opts)
>   {
>   	const char *argv[] = { NULL, NULL };
> -	struct strbuf script_snippet = STRBUF_INIT;
> +	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
>   	int status;
>   	const char *backend, *backend_func;
>   
> @@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options *opts)
>   		oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
>   	add_var(&script_snippet, "GIT_QUIET",
>   		opts->flags & REBASE_NO_QUIET ? "" : "t");
> -	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
> +	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
> +	add_var(&script_snippet, "git_am_opt", buf.buf);
> +	strbuf_release(&buf);
>   	add_var(&script_snippet, "verbose",
>   		opts->flags & REBASE_VERBOSE ? "t" : "");
>   	add_var(&script_snippet, "diffstat",
> @@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   	struct rebase_options options = {
>   		.type = REBASE_UNSPECIFIED,
>   		.flags = REBASE_NO_QUIET,
> -		.git_am_opt = STRBUF_INIT,
> +		.git_am_opts = ARGV_ARRAY_INIT,
>   		.allow_rerere_autoupdate  = -1,
>   		.allow_empty_message = 1,
>   		.git_format_patch_opt = STRBUF_INIT,
> @@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   		ACTION_EDIT_TODO,
>   		ACTION_SHOW_CURRENT_PATCH,
>   	} action = NO_ACTION;
> -	int committer_date_is_author_date = 0;
> -	int ignore_date = 0;
> -	int ignore_whitespace = 0;
>   	const char *gpg_sign = NULL;
> -	int opt_c = -1;
> -	struct string_list whitespace = STRING_LIST_INIT_NODUP;
>   	struct string_list exec = STRING_LIST_INIT_NODUP;
>   	const char *rebase_merges = NULL;
>   	int fork_point = -1;
> @@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   		{OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
>   			N_("do not show diffstat of what changed upstream"),
>   			PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
> -		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
> -			 N_("passed to 'git apply'")),
>   		OPT_BOOL(0, "signoff", &options.signoff,
>   			 N_("add a Signed-off-by: line to each commit")),
> -		OPT_BOOL(0, "committer-date-is-author-date",
> -			 &committer_date_is_author_date,
> -			 N_("passed to 'git am'")),
> -		OPT_BOOL(0, "ignore-date", &ignore_date,
> -			 N_("passed to 'git am'")),
> +		OPT_PASSTHRU_ARGV(0, "ignore-whitespace", &options.git_am_opts,
> +				  NULL, N_("passed to 'git am'"),
> +				  PARSE_OPT_NOARG),
> +		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
> +				  &options.git_am_opts, NULL,
> +				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
> +		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts, NULL,
> +				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
> +		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
> +				  N_("passed to 'git apply'"), 0),
> +		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
> +				  N_("action"), N_("passed to 'git apply'"), 0),
>   		OPT_BIT('f', "force-rebase", &options.flags,
>   			N_("cherry-pick all commits, even if unchanged"),
>   			REBASE_FORCE),
> @@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
>   			N_("GPG-sign commits"),
>   			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
> -		OPT_STRING_LIST(0, "whitespace", &whitespace,
> -				N_("whitespace"), N_("passed to 'git apply'")),
> -		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
> -			    REBASE_AM),
>   		OPT_BOOL(0, "autostash", &options.autostash,
>   			 N_("automatically stash/stash pop before and after")),
>   		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
> @@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   			 N_("rebase all reachable commits up to the root(s)")),
>   		OPT_END(),
>   	};
> +	int i;
>   
>   	/*
>   	 * NEEDSWORK: Once the builtin rebase has been tested enough
> @@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   		    state_dir_base, cmd_live_rebase, buf.buf);
>   	}
>   
> -	if (!(options.flags & REBASE_NO_QUIET))
> -		strbuf_addstr(&options.git_am_opt, " -q");
> -
> -	if (committer_date_is_author_date) {
> -		strbuf_addstr(&options.git_am_opt,
> -			      " --committer-date-is-author-date");
> -		options.flags |= REBASE_FORCE;
> +	for (i = 0; i < options.git_am_opts.argc; i++) {
> +		const char *option = options.git_am_opts.argv[i];
> +		if (!strcmp(option, "--committer-date-is-author-date") ||
> +		    !strcmp(option, "--ignore-date") ||
> +		    !strcmp(option, "--whitespace=fix") ||
> +		    !strcmp(option, "--whitespace=strip"))
> +			options.flags |= REBASE_FORCE;
>   	}
>   
> -	if (ignore_whitespace)
> -		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
> -
> -	if (ignore_date) {
> -		strbuf_addstr(&options.git_am_opt, " --ignore-date");
> -		options.flags |= REBASE_FORCE;
> -	}
> +	if (!(options.flags & REBASE_NO_QUIET))
> +		argv_array_push(&options.git_am_opts, "-q");
>   
>   	if (options.keep_empty)
>   		imply_interactive(&options, "--keep-empty");
> @@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   		options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
>   	}
>   
> -	if (opt_c >= 0)
> -		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
> -
> -	if (whitespace.nr) {
> -		int i;
> -
> -		for (i = 0; i < whitespace.nr; i++) {
> -			const char *item = whitespace.items[i].string;
> -
> -			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
> -				    item);
> -
> -			if ((!strcmp(item, "fix")) || (!strcmp(item, "strip")))
> -				options.flags |= REBASE_FORCE;
> -		}
> -	}
> -
>   	if (exec.nr) {
>   		int i;
>   
> @@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   		break;
>   	}
>   
> -	if (options.git_am_opt.len) {
> -		const char *p;
> -
> +	if (options.git_am_opts.argc) {
>   		/* all am options except -q are compatible only with --am */
> -		strbuf_reset(&buf);
> -		strbuf_addbuf(&buf, &options.git_am_opt);
> -		strbuf_addch(&buf, ' ');
> -		while ((p = strstr(buf.buf, " -q ")))
> -			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
> -		strbuf_trim(&buf);
> +		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
> +			if (strcmp(options.git_am_opts.argv[i], "-q"))
> +				break;
>   
> -		if (is_interactive(&options) && buf.len)
> +		if (is_interactive(&options) && i >= 0)
>   			die(_("error: cannot combine interactive options "
>   			      "(--interactive, --exec, --rebase-merges, "
>   			      "--preserve-merges, --keep-empty, --root + "
>   			      "--onto) with am options (%s)"), buf.buf);
> -		if (options.type == REBASE_MERGE && buf.len)
> +		if (options.type == REBASE_MERGE && i >= 0)
>   			die(_("error: cannot combine merge options (--merge, "
>   			      "--strategy, --strategy-option) with am options "
>   			      "(%s)"), buf.buf);
> @@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   		if (options.type == REBASE_PRESERVE_MERGES)
>   			die("cannot combine '--signoff' with "
>   			    "'--preserve-merges'");
> -		strbuf_addstr(&options.git_am_opt, " --signoff");
> +		argv_array_push(&options.git_am_opts, "--signoff");
>   		options.flags |= REBASE_FORCE;
>   	}
>   
> 

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

[Johannes Schindelin]

Hi Phillip,

On Tue, 13 Nov 2018, Phillip Wood wrote:

> Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to
> break the error reporting
> 
> Running
>   bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad
> 
> Gives
>   git encountered an error while preparing the patches to replay
>   these revisions:
> 
> 
> 67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c
> 
>   As a result, git cannot rebase them.
> 
> If I do
> 
>   bin/wrappers/git rebase @^^ -Cbad
> 
> I get no error, it just tells me that it does not need to rebase (which is
> true)

Hmm. Isn't this the same behavior as with the scripted version? Also: are
we sure that we want to allow options to come *after* the `<upstream>`
argument?

Ciao,
Dscho

> Best Wishes
> 
> Phillip
> 
> 
> On 13/11/2018 12:38, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > 
> > Currently, we parse the options intended for `git am` as if we wanted to
> > handle them in `git rebase`, and then reconstruct them painstakingly to
> > define the `git_am_opt` variable.
> > 
> > However, there is a much better way (that I was unaware of, at the time
> > when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV.
> > It is intended for exactly this use case, where command-line options
> > want to be parsed into a separate `argv_array`.
> > 
> > Let's use this feature.
> > 
> > Incidentally, this also allows us to address a bug discovered by Phillip
> > Wood, where the built-in rebase failed to understand that the `-C`
> > option takes an optional argument.
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >   builtin/rebase.c | 98 +++++++++++++++++-------------------------------
> >   1 file changed, 35 insertions(+), 63 deletions(-)
> > 
> > diff --git a/builtin/rebase.c b/builtin/rebase.c
> > index 0ee06aa363..96ffa80b71 100644
> > --- a/builtin/rebase.c
> > +++ b/builtin/rebase.c
> > @@ -87,7 +87,7 @@ struct rebase_options {
> >     REBASE_FORCE = 1<<3,
> >     REBASE_INTERACTIVE_EXPLICIT = 1<<4,
> >   	} flags;
> > -	struct strbuf git_am_opt;
> > +	struct argv_array git_am_opts;
> >    const char *action;
> >    int signoff;
> >    int allow_rerere_autoupdate;
> > @@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as
> > resolved with\n"
> >   static int run_specific_rebase(struct rebase_options *opts)
> >   {
> >   	const char *argv[] = { NULL, NULL };
> > -	struct strbuf script_snippet = STRBUF_INIT;
> > +	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
> >    int status;
> >    const char *backend, *backend_func;
> >   @@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options
> > *opts)
> >    	oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
> >    add_var(&script_snippet, "GIT_QUIET",
> >   		opts->flags & REBASE_NO_QUIET ? "" : "t");
> > -	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
> > +	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
> > +	add_var(&script_snippet, "git_am_opt", buf.buf);
> > +	strbuf_release(&buf);
> >    add_var(&script_snippet, "verbose",
> >    	opts->flags & REBASE_VERBOSE ? "t" : "");
> >   	add_var(&script_snippet, "diffstat",
> > @@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char
> > *prefix)
> >    struct rebase_options options = {
> >     .type = REBASE_UNSPECIFIED,
> >     .flags = REBASE_NO_QUIET,
> > -		.git_am_opt = STRBUF_INIT,
> > +		.git_am_opts = ARGV_ARRAY_INIT,
> >     .allow_rerere_autoupdate  = -1,
> >     .allow_empty_message = 1,
> >     .git_format_patch_opt = STRBUF_INIT,
> > @@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const
> > char *prefix)
> >     ACTION_EDIT_TODO,
> >     ACTION_SHOW_CURRENT_PATCH,
> >   	} action = NO_ACTION;
> > -	int committer_date_is_author_date = 0;
> > -	int ignore_date = 0;
> > -	int ignore_whitespace = 0;
> >   	const char *gpg_sign = NULL;
> > -	int opt_c = -1;
> > -	struct string_list whitespace = STRING_LIST_INIT_NODUP;
> >    struct string_list exec = STRING_LIST_INIT_NODUP;
> >    const char *rebase_merges = NULL;
> >    int fork_point = -1;
> > @@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const
> > char *prefix)
> >     {OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
> >      N_("do not show diffstat of what changed upstream"),
> >      PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
> > -		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
> > -			 N_("passed to 'git apply'")),
> >     OPT_BOOL(0, "signoff", &options.signoff,
> >   			 N_("add a Signed-off-by: line to each commit")),
> > -		OPT_BOOL(0, "committer-date-is-author-date",
> > -			 &committer_date_is_author_date,
> > -			 N_("passed to 'git am'")),
> > -		OPT_BOOL(0, "ignore-date", &ignore_date,
> > -			 N_("passed to 'git am'")),
> > +		OPT_PASSTHRU_ARGV(0, "ignore-whitespace",
> > &options.git_am_opts,
> > +				  NULL, N_("passed to 'git am'"),
> > +				  PARSE_OPT_NOARG),
> > +		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
> > +				  &options.git_am_opts, NULL,
> > +				  N_("passed to 'git am'"),
> > PARSE_OPT_NOARG),
> > +		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts,
> > NULL,
> > +				  N_("passed to 'git am'"),
> > PARSE_OPT_NOARG),
> > +		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
> > +				  N_("passed to 'git apply'"), 0),
> > +		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
> > +				  N_("action"), N_("passed to 'git apply'"),
> > 0),
> >     OPT_BIT('f', "force-rebase", &options.flags,
> >      N_("cherry-pick all commits, even if unchanged"),
> >      REBASE_FORCE),
> > @@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const
> > char *prefix)
> >     { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
> >      N_("GPG-sign commits"),
> >      PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
> > -		OPT_STRING_LIST(0, "whitespace", &whitespace,
> > -				N_("whitespace"), N_("passed to 'git
> > apply'")),
> > -		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
> > -			    REBASE_AM),
> >     OPT_BOOL(0, "autostash", &options.autostash,
> >     	 N_("automatically stash/stash pop before and after")),
> >   		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
> > @@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char
> > *prefix)
> >     	 N_("rebase all reachable commits up to the root(s)")),
> >    	OPT_END(),
> >   	};
> > +	int i;
> >   
> >    /*
> >   	 * NEEDSWORK: Once the builtin rebase has been tested enough
> > @@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const
> > char *prefix)
> >    	    state_dir_base, cmd_live_rebase, buf.buf);
> >    }
> >   -	if (!(options.flags & REBASE_NO_QUIET))
> > -		strbuf_addstr(&options.git_am_opt, " -q");
> > -
> > -	if (committer_date_is_author_date) {
> > -		strbuf_addstr(&options.git_am_opt,
> > -			      " --committer-date-is-author-date");
> > -		options.flags |= REBASE_FORCE;
> > +	for (i = 0; i < options.git_am_opts.argc; i++) {
> > +		const char *option = options.git_am_opts.argv[i];
> > +		if (!strcmp(option, "--committer-date-is-author-date") ||
> > +		    !strcmp(option, "--ignore-date") ||
> > +		    !strcmp(option, "--whitespace=fix") ||
> > +		    !strcmp(option, "--whitespace=strip"))
> > +			options.flags |= REBASE_FORCE;
> >    }
> >   -	if (ignore_whitespace)
> > -		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
> > -
> > -	if (ignore_date) {
> > -		strbuf_addstr(&options.git_am_opt, " --ignore-date");
> > -		options.flags |= REBASE_FORCE;
> > -	}
> > +	if (!(options.flags & REBASE_NO_QUIET))
> > +		argv_array_push(&options.git_am_opts, "-q");
> >   
> >    if (options.keep_empty)
> >   		imply_interactive(&options, "--keep-empty");
> > @@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const
> > char *prefix)
> >    	options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
> >    }
> >   -	if (opt_c >= 0)
> > -		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
> > -
> > -	if (whitespace.nr) {
> > -		int i;
> > -
> > -		for (i = 0; i < whitespace.nr; i++) {
> > -			const char *item = whitespace.items[i].string;
> > -
> > -			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
> > -				    item);
> > -
> > -			if ((!strcmp(item, "fix")) || (!strcmp(item,
> > "strip")))
> > -				options.flags |= REBASE_FORCE;
> > -		}
> > -	}
> > -
> >    if (exec.nr) {
> >     int i;
> >   @@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv,
> > const char *prefix)
> >    	break;
> >    }
> >   -	if (options.git_am_opt.len) {
> > -		const char *p;
> > -
> > +	if (options.git_am_opts.argc) {
> >   		/* all am options except -q are compatible only with --am */
> > -		strbuf_reset(&buf);
> > -		strbuf_addbuf(&buf, &options.git_am_opt);
> > -		strbuf_addch(&buf, ' ');
> > -		while ((p = strstr(buf.buf, " -q ")))
> > -			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
> > -		strbuf_trim(&buf);
> > +		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
> > +			if (strcmp(options.git_am_opts.argv[i], "-q"))
> > +				break;
> >   -		if (is_interactive(&options) && buf.len)
> > +		if (is_interactive(&options) && i >= 0)
> >      die(_("error: cannot combine interactive options "
> >            "(--interactive, --exec, --rebase-merges, "
> >            "--preserve-merges, --keep-empty, --root + "
> >            "--onto) with am options (%s)"), buf.buf);
> > -		if (options.type == REBASE_MERGE && buf.len)
> > +		if (options.type == REBASE_MERGE && i >= 0)
> >      die(_("error: cannot combine merge options (--merge, "
> >            "--strategy, --strategy-option) with am options "
> >            "(%s)"), buf.buf);
> > @@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const
> > char *prefix)
> >     if (options.type == REBASE_PRESERVE_MERGES)
> >      die("cannot combine '--signoff' with "
> >   			    "'--preserve-merges'");
> > -		strbuf_addstr(&options.git_am_opt, " --signoff");
> > +		argv_array_push(&options.git_am_opts, "--signoff");
> >    	options.flags |= REBASE_FORCE;
> >    }
> >   
> > 
> 
> 

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

[Phillip Wood]

Hi Johannes

On 13/11/2018 19:21, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Tue, 13 Nov 2018, Phillip Wood wrote:
> 
>> Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to
>> break the error reporting
>>
>> Running
>>    bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad
>>
>> Gives
>>    git encountered an error while preparing the patches to replay
>>    these revisions:
>>
>>
>> 67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c
>>
>>    As a result, git cannot rebase them.
>>

git 2.19.1 gives

First, rewinding head to replay your work on top of it...
Applying: Ninth batch for 2.20
error: switch `C' expects a numerical value

So it has a clear message as to what the error is, this patch regresses 
that. It would be better if rebase detected the error before starting 
though.

>> If I do
>>
>>    bin/wrappers/git rebase @^^ -Cbad
>>
>> I get no error, it just tells me that it does not need to rebase (which is
>> true)
> 
> Hmm. Isn't this the same behavior as with the scripted version?

Ah you're right the script does not check if the option argument is valid.

> Also: are
> we sure that we want to allow options to come *after* the `<upstream>`
> argument?

Maybe not but the scripted version does. I'm not sure if that is a good 
idea or not.

Best Wishes

Phillip

> Ciao,
> Dscho
> 
>> Best Wishes
>>
>> Phillip
>>
>>
>> On 13/11/2018 12:38, Johannes Schindelin via GitGitGadget wrote:
>>> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>>>
>>> Currently, we parse the options intended for `git am` as if we wanted to
>>> handle them in `git rebase`, and then reconstruct them painstakingly to
>>> define the `git_am_opt` variable.
>>>
>>> However, there is a much better way (that I was unaware of, at the time
>>> when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV.
>>> It is intended for exactly this use case, where command-line options
>>> want to be parsed into a separate `argv_array`.
>>>
>>> Let's use this feature.
>>>
>>> Incidentally, this also allows us to address a bug discovered by Phillip
>>> Wood, where the built-in rebase failed to understand that the `-C`
>>> option takes an optional argument.
>>>
>>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>>> ---
>>>    builtin/rebase.c | 98 +++++++++++++++++-------------------------------
>>>    1 file changed, 35 insertions(+), 63 deletions(-)
>>>
>>> diff --git a/builtin/rebase.c b/builtin/rebase.c
>>> index 0ee06aa363..96ffa80b71 100644
>>> --- a/builtin/rebase.c
>>> +++ b/builtin/rebase.c
>>> @@ -87,7 +87,7 @@ struct rebase_options {
>>>      REBASE_FORCE = 1<<3,
>>>      REBASE_INTERACTIVE_EXPLICIT = 1<<4,
>>>    	} flags;
>>> -	struct strbuf git_am_opt;
>>> +	struct argv_array git_am_opts;
>>>     const char *action;
>>>     int signoff;
>>>     int allow_rerere_autoupdate;
>>> @@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as
>>> resolved with\n"
>>>    static int run_specific_rebase(struct rebase_options *opts)
>>>    {
>>>    	const char *argv[] = { NULL, NULL };
>>> -	struct strbuf script_snippet = STRBUF_INIT;
>>> +	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
>>>     int status;
>>>     const char *backend, *backend_func;
>>>    @@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options
>>> *opts)
>>>     	oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
>>>     add_var(&script_snippet, "GIT_QUIET",
>>>    		opts->flags & REBASE_NO_QUIET ? "" : "t");
>>> -	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
>>> +	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
>>> +	add_var(&script_snippet, "git_am_opt", buf.buf);
>>> +	strbuf_release(&buf);
>>>     add_var(&script_snippet, "verbose",
>>>     	opts->flags & REBASE_VERBOSE ? "t" : "");
>>>    	add_var(&script_snippet, "diffstat",
>>> @@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char
>>> *prefix)
>>>     struct rebase_options options = {
>>>      .type = REBASE_UNSPECIFIED,
>>>      .flags = REBASE_NO_QUIET,
>>> -		.git_am_opt = STRBUF_INIT,
>>> +		.git_am_opts = ARGV_ARRAY_INIT,
>>>      .allow_rerere_autoupdate  = -1,
>>>      .allow_empty_message = 1,
>>>      .git_format_patch_opt = STRBUF_INIT,
>>> @@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const
>>> char *prefix)
>>>      ACTION_EDIT_TODO,
>>>      ACTION_SHOW_CURRENT_PATCH,
>>>    	} action = NO_ACTION;
>>> -	int committer_date_is_author_date = 0;
>>> -	int ignore_date = 0;
>>> -	int ignore_whitespace = 0;
>>>    	const char *gpg_sign = NULL;
>>> -	int opt_c = -1;
>>> -	struct string_list whitespace = STRING_LIST_INIT_NODUP;
>>>     struct string_list exec = STRING_LIST_INIT_NODUP;
>>>     const char *rebase_merges = NULL;
>>>     int fork_point = -1;
>>> @@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const
>>> char *prefix)
>>>      {OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
>>>       N_("do not show diffstat of what changed upstream"),
>>>       PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
>>> -		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
>>> -			 N_("passed to 'git apply'")),
>>>      OPT_BOOL(0, "signoff", &options.signoff,
>>>    			 N_("add a Signed-off-by: line to each commit")),
>>> -		OPT_BOOL(0, "committer-date-is-author-date",
>>> -			 &committer_date_is_author_date,
>>> -			 N_("passed to 'git am'")),
>>> -		OPT_BOOL(0, "ignore-date", &ignore_date,
>>> -			 N_("passed to 'git am'")),
>>> +		OPT_PASSTHRU_ARGV(0, "ignore-whitespace",
>>> &options.git_am_opts,
>>> +				  NULL, N_("passed to 'git am'"),
>>> +				  PARSE_OPT_NOARG),
>>> +		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
>>> +				  &options.git_am_opts, NULL,
>>> +				  N_("passed to 'git am'"),
>>> PARSE_OPT_NOARG),
>>> +		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts,
>>> NULL,
>>> +				  N_("passed to 'git am'"),
>>> PARSE_OPT_NOARG),
>>> +		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
>>> +				  N_("passed to 'git apply'"), 0),
>>> +		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
>>> +				  N_("action"), N_("passed to 'git apply'"),
>>> 0),
>>>      OPT_BIT('f', "force-rebase", &options.flags,
>>>       N_("cherry-pick all commits, even if unchanged"),
>>>       REBASE_FORCE),
>>> @@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const
>>> char *prefix)
>>>      { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
>>>       N_("GPG-sign commits"),
>>>       PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
>>> -		OPT_STRING_LIST(0, "whitespace", &whitespace,
>>> -				N_("whitespace"), N_("passed to 'git
>>> apply'")),
>>> -		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
>>> -			    REBASE_AM),
>>>      OPT_BOOL(0, "autostash", &options.autostash,
>>>      	 N_("automatically stash/stash pop before and after")),
>>>    		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
>>> @@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char
>>> *prefix)
>>>      	 N_("rebase all reachable commits up to the root(s)")),
>>>     	OPT_END(),
>>>    	};
>>> +	int i;
>>>    
>>>     /*
>>>    	 * NEEDSWORK: Once the builtin rebase has been tested enough
>>> @@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const
>>> char *prefix)
>>>     	    state_dir_base, cmd_live_rebase, buf.buf);
>>>     }
>>>    -	if (!(options.flags & REBASE_NO_QUIET))
>>> -		strbuf_addstr(&options.git_am_opt, " -q");
>>> -
>>> -	if (committer_date_is_author_date) {
>>> -		strbuf_addstr(&options.git_am_opt,
>>> -			      " --committer-date-is-author-date");
>>> -		options.flags |= REBASE_FORCE;
>>> +	for (i = 0; i < options.git_am_opts.argc; i++) {
>>> +		const char *option = options.git_am_opts.argv[i];
>>> +		if (!strcmp(option, "--committer-date-is-author-date") ||
>>> +		    !strcmp(option, "--ignore-date") ||
>>> +		    !strcmp(option, "--whitespace=fix") ||
>>> +		    !strcmp(option, "--whitespace=strip"))
>>> +			options.flags |= REBASE_FORCE;
>>>     }
>>>    -	if (ignore_whitespace)
>>> -		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
>>> -
>>> -	if (ignore_date) {
>>> -		strbuf_addstr(&options.git_am_opt, " --ignore-date");
>>> -		options.flags |= REBASE_FORCE;
>>> -	}
>>> +	if (!(options.flags & REBASE_NO_QUIET))
>>> +		argv_array_push(&options.git_am_opts, "-q");
>>>    
>>>     if (options.keep_empty)
>>>    		imply_interactive(&options, "--keep-empty");
>>> @@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const
>>> char *prefix)
>>>     	options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
>>>     }
>>>    -	if (opt_c >= 0)
>>> -		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
>>> -
>>> -	if (whitespace.nr) {
>>> -		int i;
>>> -
>>> -		for (i = 0; i < whitespace.nr; i++) {
>>> -			const char *item = whitespace.items[i].string;
>>> -
>>> -			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
>>> -				    item);
>>> -
>>> -			if ((!strcmp(item, "fix")) || (!strcmp(item,
>>> "strip")))
>>> -				options.flags |= REBASE_FORCE;
>>> -		}
>>> -	}
>>> -
>>>     if (exec.nr) {
>>>      int i;
>>>    @@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv,
>>> const char *prefix)
>>>     	break;
>>>     }
>>>    -	if (options.git_am_opt.len) {
>>> -		const char *p;
>>> -
>>> +	if (options.git_am_opts.argc) {
>>>    		/* all am options except -q are compatible only with --am */
>>> -		strbuf_reset(&buf);
>>> -		strbuf_addbuf(&buf, &options.git_am_opt);
>>> -		strbuf_addch(&buf, ' ');
>>> -		while ((p = strstr(buf.buf, " -q ")))
>>> -			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
>>> -		strbuf_trim(&buf);
>>> +		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
>>> +			if (strcmp(options.git_am_opts.argv[i], "-q"))
>>> +				break;
>>>    -		if (is_interactive(&options) && buf.len)
>>> +		if (is_interactive(&options) && i >= 0)
>>>       die(_("error: cannot combine interactive options "
>>>             "(--interactive, --exec, --rebase-merges, "
>>>             "--preserve-merges, --keep-empty, --root + "
>>>             "--onto) with am options (%s)"), buf.buf);
>>> -		if (options.type == REBASE_MERGE && buf.len)
>>> +		if (options.type == REBASE_MERGE && i >= 0)
>>>       die(_("error: cannot combine merge options (--merge, "
>>>             "--strategy, --strategy-option) with am options "
>>>             "(%s)"), buf.buf);
>>> @@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const
>>> char *prefix)
>>>      if (options.type == REBASE_PRESERVE_MERGES)
>>>       die("cannot combine '--signoff' with "
>>>    			    "'--preserve-merges'");
>>> -		strbuf_addstr(&options.git_am_opt, " --signoff");
>>> +		argv_array_push(&options.git_am_opts, "--signoff");
>>>     	options.flags |= REBASE_FORCE;
>>>     }
>>>    
>>>
>>
>>

rebase-in-C stability for 2.20

[Ævar Arnfjörð Bjarmason]

On Tue, Nov 13 2018, Phillip Wood wrote:

> Hi Johannes
>
> On 13/11/2018 19:21, Johannes Schindelin wrote:
>> Hi Phillip,
>>
>> On Tue, 13 Nov 2018, Phillip Wood wrote:
>>
>>> Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to
>>> break the error reporting
>>>
>>> Running
>>>    bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad
>>>
>>> Gives
>>>    git encountered an error while preparing the patches to replay
>>>    these revisions:
>>>
>>>
>>> 67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c
>>>
>>>    As a result, git cannot rebase them.
>>>
>
> git 2.19.1 gives
>
> First, rewinding head to replay your work on top of it...
> Applying: Ninth batch for 2.20
> error: switch `C' expects a numerical value
>
> So it has a clear message as to what the error is, this patch
> regresses that. It would be better if rebase detected the error before
> starting though.
>
>>> If I do
>>>
>>>    bin/wrappers/git rebase @^^ -Cbad
>>>
>>> I get no error, it just tells me that it does not need to rebase (which is
>>> true)
>>
>> Hmm. Isn't this the same behavior as with the scripted version?
>
> Ah you're right the script does not check if the option argument is valid.
>
>> Also: are
>> we sure that we want to allow options to come *after* the `<upstream>`
>> argument?
>
> Maybe not but the scripted version does. I'm not sure if that is a
> good idea or not.

According to Junio's calendar we're now 2 days from 2.20-rc0. We have
the js/rebase-autostash-detach-fix bug I reported sitting in "pu" still,
and then this.

I see we still have rebase.useBuiltin in the code as an escape hatch,
but it's undocumented.

Given that we're still finding regressions bugs in the rebase-in-C
version should we be considering reverting 5541bd5b8f ("rebase: default
to using the builtin rebase", 2018-08-08)?

I love the feature, but fear that the current list of known regressions
serve as a canary for a larger list which we'd discover if we held off
for another major release (and would re-enable rebase.useBuiltin=true in
master right after 2.20 is out the door).

But maybe I'm being overly paranoid. What do those more familiar with
this think?

Re: rebase-in-C stability for 2.20

[Stefan Beller]

> But maybe I'm being overly paranoid. What do those more familiar with
> this think?

I am not too worried,
* as rebase is a main porcelain, that is even hard to use in a script.
  so any failures are not deep down in some automation,
  but when found exposed quickly (and hopefully reported).
* 5541bd5b8f was merged to next a month ago; internally we
   distribute the next branch to Googlers (on a weekly basis)
   and we have not had any bug reports regarding rebase.
   (Maybe our environment is too strict for the wide range
    of bugs reported)
* Johannes reported that the rebase is used in GfW
   https://public-inbox.org/git/nycvar.QRO.7.76.6.1808241320540.73@tvgsbejvaqbjf.bet/
   https://github.com/git-for-windows/git/pull/1800
   and from my cursory reading it is part of
   2.19.windows, which has a large user base.

> (and would re-enable rebase.useBuiltin=true in
> master right after 2.20 is out the door).

That would be fine with me as well, but I'd rather
document rebase.useBuiltin instead of flip-flopping
the switch around the release.

Have there been any fixes that are only in
the C version (has the shell version already bitrotted)?

Stefan

Re: rebase-in-C stability for 2.20

[Elijah Newren]

On Tue, Nov 13, 2018 at 1:52 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> According to Junio's calendar we're now 2 days from 2.20-rc0. We have
> the js/rebase-autostash-detach-fix bug I reported sitting in "pu" still,
> and then this.
>
> I see we still have rebase.useBuiltin in the code as an escape hatch,
> but it's undocumented.
>
> Given that we're still finding regressions bugs in the rebase-in-C
> version should we be considering reverting 5541bd5b8f ("rebase: default
> to using the builtin rebase", 2018-08-08)?

That date feels slightly misleading; it has a commit date of Oct 11,
and only merged to master less than two weeks ago.  Given how big the
two different rebase in C series were, I'd expect a couple small
things to fall out after it hit master, which is what appears to be
happening.

> I love the feature, but fear that the current list of known regressions
> serve as a canary for a larger list which we'd discover if we held off
> for another major release (and would re-enable rebase.useBuiltin=true in
> master right after 2.20 is out the door).
>
> But maybe I'm being overly paranoid. What do those more familiar with
> this think?

I probably don't qualify as more familar, but giving my $0.02
anyway...  I'm happy that setting rebase.useBuiltin=false by default
exists an escape hatch if things don't settle down as we get closer to
the release, but I'd rather wait until further into the RC cycle
before going that route, personally.

Re: rebase-in-C stability for 2.20

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> According to Junio's calendar we're now 2 days from 2.20-rc0. We have
> the js/rebase-autostash-detach-fix bug I reported sitting in "pu" still,
> and then this.
>
> I see we still have rebase.useBuiltin in the code as an escape hatch,
> but it's undocumented.
>
> Given that we're still finding regressions bugs in the rebase-in-C
> version should we be considering reverting 5541bd5b8f ("rebase: default
> to using the builtin rebase", 2018-08-08)?
>
> I love the feature, but fear that the current list of known regressions
> serve as a canary for a larger list which we'd discover if we held off
> for another major release (and would re-enable rebase.useBuiltin=true in
> master right after 2.20 is out the door).
>
> But maybe I'm being overly paranoid. What do those more familiar with
> this think?

I was hoping that having it early in GfW 2.19 would have smoked out
all the remaining issues, but it seems that those building from the
source and testing have usage patterns different enough to find more
issues.  This is a normal part of the development process, and
hopefully the remaining bugs are minor and can be flushed out in the
-rc testing period---this kind of thing is the whole reason why we
code-freeze and test rcs.

It unfortunately is too late to depend on the rebase.useBuiltin as
an escape hatch, as 'master', 'next', or anywhere else had the
"rebase and rebase -i in C" series merged without it set to false
and used in any seriousness.  Quite honestly, I think we can trust
the rest of the "rebase and rebase -i in C" series much more than
its "use the scripted one even though we have built-in one" part.

So if we have to seriously consider holding back, we may be better
off ripping the whole thing out.  I do not offhand know how involved
such a reversion would be, though, and I am *NOT* looking forward to
having do such a major surgery right before the release.

Re: [PATCH 0/1] rebase: understand -C again, refactor

[Jeff King]

On Tue, Nov 13, 2018 at 04:38:24AM -0800, Johannes Schindelin via GitGitGadget wrote:

> Phillip Wood reported a problem where the built-in rebase did not understand
> options like -C1, i.e. it did not expect the option argument.
> 
> While investigating how to address this best, I stumbled upon 
> OPT_PASSTHRU_ARGV (which I was so far happily unaware of).

I was unaware of it, too. Looking at the OPT_PASSTHRU and its ARGV
counterpart, I think the original intent was that you'd pass through
normal last-one-wins individual options with OPT_PASSTHRU, and then
list-like options with OPT_PASSTHRU_ARGV. But here you've used the
latter to pass sets of individual last-one-wins options.

That said, I think what you've done here is way simpler and more
readable than using a bunch of OPT_PASSTHRUs would have been. And even
if it was not the original intent of the ARGV variant, I can't see any
reason to avoid doing it this way.

-Peff

[PATCH 0/2] rebase.useBuiltin doc & test mode

[Ævar Arnfjörð Bjarmason]

On Wed, Nov 14 2018, Stefan Beller wrote:

>> But maybe I'm being overly paranoid. What do those more familiar with
>> this think?
>
> I am not too worried,
> * as rebase is a main porcelain, that is even hard to use in a script.
>   so any failures are not deep down in some automation,
>   but when found exposed quickly (and hopefully reported).
> * 5541bd5b8f was merged to next a month ago; internally we
>    distribute the next branch to Googlers (on a weekly basis)
>    and we have not had any bug reports regarding rebase.
>    (Maybe our environment is too strict for the wide range
>     of bugs reported)

I do the same at Booking.com (although at a more ad-hoc schedule) and
got the report whose fix is now sitting in "pu" noted upthread.

I fear that these sorts of corporate environments, both Google's and
Booking's, end up testing a relatively narrow featureset. Most people
have similar enough workflows, e.g. just using "git pull --rebase",
I'd be surprised if we have more than 2-3 internal users who ever use
the --onto option for example.

> * Johannes reported that the rebase is used in GfW
>    https://public-inbox.org/git/nycvar.QRO.7.76.6.1808241320540.73@tvgsbejvaqbjf.bet/
>    https://github.com/git-for-windows/git/pull/1800
>    and from my cursory reading it is part of
>    2.19.windows, which has a large user base.
>
>> (and would re-enable rebase.useBuiltin=true in
>> master right after 2.20 is out the door).
>
> That would be fine with me as well, but I'd rather
> document rebase.useBuiltin instead of flip-flopping
> the switch around the release.
>
> Have there been any fixes that are only in
> the C version (has the shell version already bitrotted)?

That's a good question, one which I don't think we knew the answer to
before the following patches. As it turns out no, we still run the
tests without failures with GIT_TEST_REBASE_USE_BUILTIN=false.

Ævar Arnfjörð Bjarmason (2):
  rebase doc: document rebase.useBuiltin
  tests: add a special setup where rebase.useBuiltin is off

 Documentation/config/rebase.txt | 14 ++++++++++++++
 builtin/rebase.c                |  5 ++++-
 t/README                        |  3 +++
 3 files changed, 21 insertions(+), 1 deletion(-)

-- 
2.19.1.1182.g4ecb1133ce

[PATCH 1/2] rebase doc: document rebase.useBuiltin

[Ævar Arnfjörð Bjarmason]

The rebase.useBuiltin variable introduced in 55071ea248 ("rebase:
start implementing it as a builtin", 2018-08-07) was turned on by
default in 5541bd5b8f ("rebase: default to using the builtin rebase",
2018-08-08), but had no documentation.

Let's document it so that users who run into any stability issues with
the C rewrite know there's an escape hatch[1], and make it clear that
needing to turn off builtin rebase means you've found a bug in git.

1. https://public-inbox.org/git/87y39w1wc2.fsf@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config/rebase.txt | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/Documentation/config/rebase.txt b/Documentation/config/rebase.txt
index 42e1ba7575..f079bf6b7e 100644
--- a/Documentation/config/rebase.txt
+++ b/Documentation/config/rebase.txt
@@ -1,3 +1,17 @@
+rebase.useBuiltin::
+	Set to `false` to use the legacy shellscript implementation of
+	linkgit:git-rebase[1]. Is `true` by default, which means use
+	the built-in rewrite of it in C.
++
+The C rewrite is first included with Git version 2.20. This option
+serves an an escape hatch to re-enable the legacy version in case any
+bugs are found in the rewrite. This option and the shellscript version
+of linkgit:git-rebase[1] will be removed in some future release.
++
+If you find some reason to set this option to `false` other than
+one-off testing you should report the behavior difference as a bug in
+git.
+
 rebase.stat::
 	Whether to show a diffstat of what changed upstream since the last
 	rebase. False by default.
-- 
2.19.1.1182.g4ecb1133ce

[PATCH 2/2] tests: add a special setup where rebase.useBuiltin is off

[Ævar Arnfjörð Bjarmason]

Add a GIT_TEST_REBASE_USE_BUILTIN=false test mode which is equivalent
to running with rebase.useBuiltin=false. This is needed to spot that
we're not introducing any regressions in the legacy rebase version
while we're carrying both it and the new builtin version.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/rebase.c | 5 ++++-
 t/README         | 3 +++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 0ee06aa363..68ad8c1149 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -48,7 +48,10 @@ static int use_builtin_rebase(void)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
 	struct strbuf out = STRBUF_INIT;
-	int ret;
+	int ret, env = git_env_bool("GIT_TEST_REBASE_USE_BUILTIN", -1);
+
+	if (env != -1)
+		return env;
 
 	argv_array_pushl(&cp.args,
 			 "config", "--bool", "rebase.usebuiltin", NULL);
diff --git a/t/README b/t/README
index 242497455f..c719e08414 100644
--- a/t/README
+++ b/t/README
@@ -348,6 +348,9 @@ GIT_TEST_MULTI_PACK_INDEX=<boolean>, when true, forces the multi-pack-
 index to be written after every 'git repack' command, and overrides the
 'core.multiPackIndex' setting to true.
 
+GIT_TEST_REBASE_USE_BUILTIN<boolean>, when false, disables the builtin
+version of git-rebase. See 'rebase.useBuiltin' in git-config(1).
+
 Naming Tests
 ------------
 
-- 
2.19.1.1182.g4ecb1133ce

Re: [PATCH 0/2] rebase.useBuiltin doc & test mode

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2659 bytes --]

Hi Ævar,

On Wed, 14 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> On Wed, Nov 14 2018, Stefan Beller wrote:
> 
> >> But maybe I'm being overly paranoid. What do those more familiar with
> >> this think?
> >
> > I am not too worried,
> > * as rebase is a main porcelain, that is even hard to use in a script.
> >   so any failures are not deep down in some automation,
> >   but when found exposed quickly (and hopefully reported).
> > * 5541bd5b8f was merged to next a month ago; internally we
> >    distribute the next branch to Googlers (on a weekly basis)
> >    and we have not had any bug reports regarding rebase.
> >    (Maybe our environment is too strict for the wide range
> >     of bugs reported)
> 
> I do the same at Booking.com (although at a more ad-hoc schedule) and
> got the report whose fix is now sitting in "pu" noted upthread.
> 
> I fear that these sorts of corporate environments, both Google's and
> Booking's, end up testing a relatively narrow featureset. Most people
> have similar enough workflows, e.g. just using "git pull --rebase",
> I'd be surprised if we have more than 2-3 internal users who ever use
> the --onto option for example.
> 
> > * Johannes reported that the rebase is used in GfW
> >    https://public-inbox.org/git/nycvar.QRO.7.76.6.1808241320540.73@tvgsbejvaqbjf.bet/
> >    https://github.com/git-for-windows/git/pull/1800
> >    and from my cursory reading it is part of
> >    2.19.windows, which has a large user base.
> >
> >> (and would re-enable rebase.useBuiltin=true in
> >> master right after 2.20 is out the door).
> >
> > That would be fine with me as well, but I'd rather
> > document rebase.useBuiltin instead of flip-flopping
> > the switch around the release.
> >
> > Have there been any fixes that are only in
> > the C version (has the shell version already bitrotted)?
> 
> That's a good question, one which I don't think we knew the answer to
> before the following patches.

I pay close attention to `git rebase` bug reports and patches (obviously),
and there have not been any changes going into the built-in rebase/rebase
-i that necessitated changes also in the scripted version.

Ciao,
Dscho

> As it turns out no, we still run the tests without failures with
> GIT_TEST_REBASE_USE_BUILTIN=false.
> 
> Ævar Arnfjörð Bjarmason (2):
>   rebase doc: document rebase.useBuiltin
>   tests: add a special setup where rebase.useBuiltin is off
> 
>  Documentation/config/rebase.txt | 14 ++++++++++++++
>  builtin/rebase.c                |  5 ++++-
>  t/README                        |  3 +++
>  3 files changed, 21 insertions(+), 1 deletion(-)
> 
> -- 
> 2.19.1.1182.g4ecb1133ce
> 
> 

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

[Johannes Schindelin]

Hi Phillip,

On Tue, 13 Nov 2018, Phillip Wood wrote:

> On 13/11/2018 19:21, Johannes Schindelin wrote:
> > Hi Phillip,
> > 
> > On Tue, 13 Nov 2018, Phillip Wood wrote:
> > 
> > > Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to
> > > break the error reporting
> > >
> > > Running
> > >    bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad
> > >
> > > Gives
> > >    git encountered an error while preparing the patches to replay
> > >    these revisions:
> > >
> > >
> > > 67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c
> > >
> > >    As a result, git cannot rebase them.
> > >
> 
> git 2.19.1 gives
> 
> First, rewinding head to replay your work on top of it...
> Applying: Ninth batch for 2.20
> error: switch `C' expects a numerical value
> 
> So it has a clear message as to what the error is, this patch regresses that.
> It would be better if rebase detected the error before starting though.

I agree that that would make most sense: why start something when you can
know that it will fail later...

Let me try to add that test case that Junio wanted, and some early error
handling.

> > > If I do
> > >
> > >    bin/wrappers/git rebase @^^ -Cbad
> > >
> > > I get no error, it just tells me that it does not need to rebase (which is
> > > true)
> > 
> > Hmm. Isn't this the same behavior as with the scripted version?
> 
> Ah you're right the script does not check if the option argument is valid.
> 
> > Also: are
> > we sure that we want to allow options to come *after* the `<upstream>`
> > argument?
> 
> Maybe not but the scripted version does. I'm not sure if that is a good idea
> or not.

That behavior was never documented, though, was it?

Ciao,
Dscho

> 
> Best Wishes
> 
> Phillip
> 
> > Ciao,
> > Dscho
> > 
> > > Best Wishes
> > >
> > > Phillip
> > >
> > >
> > > On 13/11/2018 12:38, Johannes Schindelin via GitGitGadget wrote:
> > > > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > > >
> > > > Currently, we parse the options intended for `git am` as if we wanted to
> > > > handle them in `git rebase`, and then reconstruct them painstakingly to
> > > > define the `git_am_opt` variable.
> > > >
> > > > However, there is a much better way (that I was unaware of, at the time
> > > > when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV.
> > > > It is intended for exactly this use case, where command-line options
> > > > want to be parsed into a separate `argv_array`.
> > > >
> > > > Let's use this feature.
> > > >
> > > > Incidentally, this also allows us to address a bug discovered by Phillip
> > > > Wood, where the built-in rebase failed to understand that the `-C`
> > > > option takes an optional argument.
> > > >
> > > > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > > > ---
> > > >    builtin/rebase.c | 98
> > > >    +++++++++++++++++-------------------------------
> > > >    1 file changed, 35 insertions(+), 63 deletions(-)
> > > >
> > > > diff --git a/builtin/rebase.c b/builtin/rebase.c
> > > > index 0ee06aa363..96ffa80b71 100644
> > > > --- a/builtin/rebase.c
> > > > +++ b/builtin/rebase.c
> > > > @@ -87,7 +87,7 @@ struct rebase_options {
> > > >      REBASE_FORCE = 1<<3,
> > > >      REBASE_INTERACTIVE_EXPLICIT = 1<<4,
> > > >    	} flags;
> > > > -	struct strbuf git_am_opt;
> > > > +	struct argv_array git_am_opts;
> > > >     const char *action;
> > > >     int signoff;
> > > >     int allow_rerere_autoupdate;
> > > > @@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as
> > > > resolved with\n"
> > > >    static int run_specific_rebase(struct rebase_options *opts)
> > > >    {
> > > >    	const char *argv[] = { NULL, NULL };
> > > > -	struct strbuf script_snippet = STRBUF_INIT;
> > > > +	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
> > > >     int status;
> > > >     const char *backend, *backend_func;
> > > >    @@ -433,7 +433,9 @@ static int run_specific_rebase(struct
> > > > rebase_options
> > > > *opts)
> > > >     	oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
> > > >     add_var(&script_snippet, "GIT_QUIET",
> > > >    		opts->flags & REBASE_NO_QUIET ? "" : "t");
> > > > -	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
> > > > +	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
> > > > +	add_var(&script_snippet, "git_am_opt", buf.buf);
> > > > +	strbuf_release(&buf);
> > > >     add_var(&script_snippet, "verbose",
> > > >     	opts->flags & REBASE_VERBOSE ? "t" : "");
> > > >    	add_var(&script_snippet, "diffstat",
> > > > @@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const
> > > > char
> > > > *prefix)
> > > >     struct rebase_options options = {
> > > >      .type = REBASE_UNSPECIFIED,
> > > >      .flags = REBASE_NO_QUIET,
> > > > -		.git_am_opt = STRBUF_INIT,
> > > > +		.git_am_opts = ARGV_ARRAY_INIT,
> > > >      .allow_rerere_autoupdate  = -1,
> > > >      .allow_empty_message = 1,
> > > >      .git_format_patch_opt = STRBUF_INIT,
> > > > @@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const
> > > > char *prefix)
> > > >      ACTION_EDIT_TODO,
> > > >      ACTION_SHOW_CURRENT_PATCH,
> > > >    	} action = NO_ACTION;
> > > > -	int committer_date_is_author_date = 0;
> > > > -	int ignore_date = 0;
> > > > -	int ignore_whitespace = 0;
> > > >    	const char *gpg_sign = NULL;
> > > > -	int opt_c = -1;
> > > > -	struct string_list whitespace = STRING_LIST_INIT_NODUP;
> > > >     struct string_list exec = STRING_LIST_INIT_NODUP;
> > > >     const char *rebase_merges = NULL;
> > > >     int fork_point = -1;
> > > > @@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const
> > > > char *prefix)
> > > >      {OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
> > > >       N_("do not show diffstat of what changed upstream"),
> > > >       PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
> > > > -		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
> > > > -			 N_("passed to 'git apply'")),
> > > >      OPT_BOOL(0, "signoff", &options.signoff,
> > > >    			 N_("add a Signed-off-by: line to each
> > > > commit")),
> > > > -		OPT_BOOL(0, "committer-date-is-author-date",
> > > > -			 &committer_date_is_author_date,
> > > > -			 N_("passed to 'git am'")),
> > > > -		OPT_BOOL(0, "ignore-date", &ignore_date,
> > > > -			 N_("passed to 'git am'")),
> > > > +		OPT_PASSTHRU_ARGV(0, "ignore-whitespace",
> > > > &options.git_am_opts,
> > > > +				  NULL, N_("passed to 'git am'"),
> > > > +				  PARSE_OPT_NOARG),
> > > > +		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
> > > > +				  &options.git_am_opts, NULL,
> > > > +				  N_("passed to 'git am'"),
> > > > PARSE_OPT_NOARG),
> > > > +		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts,
> > > > NULL,
> > > > +				  N_("passed to 'git am'"),
> > > > PARSE_OPT_NOARG),
> > > > +		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
> > > > +				  N_("passed to 'git apply'"), 0),
> > > > +		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
> > > > +				  N_("action"), N_("passed to 'git apply'"),
> > > > 0),
> > > >      OPT_BIT('f', "force-rebase", &options.flags,
> > > >       N_("cherry-pick all commits, even if unchanged"),
> > > >       REBASE_FORCE),
> > > > @@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const
> > > > char *prefix)
> > > >      { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
> > > >       N_("GPG-sign commits"),
> > > >       PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
> > > > -		OPT_STRING_LIST(0, "whitespace", &whitespace,
> > > > -				N_("whitespace"), N_("passed to 'git
> > > > apply'")),
> > > > -		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
> > > > -			    REBASE_AM),
> > > >      OPT_BOOL(0, "autostash", &options.autostash,
> > > >        	 N_("automatically stash/stash pop before and after")),
> > > >    		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
> > > > @@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const
> > > > char
> > > > *prefix)
> > > >      	 N_("rebase all reachable commits up to the root(s)")),
> > > >     	OPT_END(),
> > > >    	};
> > > > +	int i;
> > > >    
> > > >     /*
> > > >    	 * NEEDSWORK: Once the builtin rebase has been tested enough
> > > > @@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv,
> > > > const
> > > > char *prefix)
> > > >     	    state_dir_base, cmd_live_rebase, buf.buf);
> > > >    }
> > > >    -	if (!(options.flags & REBASE_NO_QUIET))
> > > > -		strbuf_addstr(&options.git_am_opt, " -q");
> > > > -
> > > > -	if (committer_date_is_author_date) {
> > > > -		strbuf_addstr(&options.git_am_opt,
> > > > -			      " --committer-date-is-author-date");
> > > > -		options.flags |= REBASE_FORCE;
> > > > +	for (i = 0; i < options.git_am_opts.argc; i++) {
> > > > +		const char *option = options.git_am_opts.argv[i];
> > > > +		if (!strcmp(option, "--committer-date-is-author-date") ||
> > > > +		    !strcmp(option, "--ignore-date") ||
> > > > +		    !strcmp(option, "--whitespace=fix") ||
> > > > +		    !strcmp(option, "--whitespace=strip"))
> > > > +			options.flags |= REBASE_FORCE;
> > > >    }
> > > >    -	if (ignore_whitespace)
> > > > -		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
> > > > -
> > > > -	if (ignore_date) {
> > > > -		strbuf_addstr(&options.git_am_opt, " --ignore-date");
> > > > -		options.flags |= REBASE_FORCE;
> > > > -	}
> > > > +	if (!(options.flags & REBASE_NO_QUIET))
> > > > +		argv_array_push(&options.git_am_opts, "-q");
> > > >    
> > > >     if (options.keep_empty)
> > > >    		imply_interactive(&options, "--keep-empty");
> > > > @@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const
> > > > char *prefix)
> > > >     	options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
> > > >    }
> > > >    -	if (opt_c >= 0)
> > > > -		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
> > > > -
> > > > -	if (whitespace.nr) {
> > > > -		int i;
> > > > -
> > > > -		for (i = 0; i < whitespace.nr; i++) {
> > > > -			const char *item = whitespace.items[i].string;
> > > > -
> > > > -			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
> > > > -				    item);
> > > > -
> > > > -			if ((!strcmp(item, "fix")) || (!strcmp(item,
> > > > "strip")))
> > > > -				options.flags |= REBASE_FORCE;
> > > > -		}
> > > > -	}
> > > > -
> > > >     if (exec.nr) {
> > > >      int i;
> > > >    @@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv,
> > > > const char *prefix)
> > > >     	break;
> > > >    }
> > > >    -	if (options.git_am_opt.len) {
> > > > -		const char *p;
> > > > -
> > > > +	if (options.git_am_opts.argc) {
> > > >    		/* all am options except -q are compatible only with
> > > > --am */
> > > > -		strbuf_reset(&buf);
> > > > -		strbuf_addbuf(&buf, &options.git_am_opt);
> > > > -		strbuf_addch(&buf, ' ');
> > > > -		while ((p = strstr(buf.buf, " -q ")))
> > > > -			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
> > > > -		strbuf_trim(&buf);
> > > > +		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
> > > > +			if (strcmp(options.git_am_opts.argv[i], "-q"))
> > > > +				break;
> > > >    -		if (is_interactive(&options) && buf.len)
> > > > +		if (is_interactive(&options) && i >= 0)
> > > >       die(_("error: cannot combine interactive options "
> > > >             "(--interactive, --exec, --rebase-merges, "
> > > >             "--preserve-merges, --keep-empty, --root + "
> > > >             "--onto) with am options (%s)"), buf.buf);
> > > > -		if (options.type == REBASE_MERGE && buf.len)
> > > > +		if (options.type == REBASE_MERGE && i >= 0)
> > > >       die(_("error: cannot combine merge options (--merge, "
> > > >             "--strategy, --strategy-option) with am options "
> > > >             "(%s)"), buf.buf);
> > > > @@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const
> > > > char *prefix)
> > > >      if (options.type == REBASE_PRESERVE_MERGES)
> > > >       die("cannot combine '--signoff' with "
> > > >    			    "'--preserve-merges'");
> > > > -		strbuf_addstr(&options.git_am_opt, " --signoff");
> > > > +		argv_array_push(&options.git_am_opts, "--signoff");
> > > >     	options.flags |= REBASE_FORCE;
> > > >     }
> > > >    
> > > >
> > >
> > >
> 
> 
> 

Re: [PATCH 0/1] rebase: understand -C again, refactor

[Johannes Schindelin]

Hi Peff,

On Wed, 14 Nov 2018, Jeff King wrote:

> On Tue, Nov 13, 2018 at 04:38:24AM -0800, Johannes Schindelin via GitGitGadget wrote:
> 
> > Phillip Wood reported a problem where the built-in rebase did not understand
> > options like -C1, i.e. it did not expect the option argument.
> > 
> > While investigating how to address this best, I stumbled upon 
> > OPT_PASSTHRU_ARGV (which I was so far happily unaware of).
> 
> I was unaware of it, too.

Thanks, that makes me feel better ;-)

> Looking at the OPT_PASSTHRU and its ARGV counterpart, I think the
> original intent was that you'd pass through normal last-one-wins
> individual options with OPT_PASSTHRU, and then list-like options with
> OPT_PASSTHRU_ARGV. But here you've used the latter to pass sets of
> individual last-one-wins options.
> 
> That said, I think what you've done here is way simpler and more
> readable than using a bunch of OPT_PASSTHRUs would have been. And even
> if it was not the original intent of the ARGV variant, I can't see any
> reason to avoid doing it this way.

Thank you, that makes me feel *even* better ;-)

Together with the extra-early error checking for incorrect -C and
--whitespace options, I think we're in a much better place now.

Ciao,
Dscho

[PATCH v2 0/2] rebase: understand -C again, refactor

[Johannes Schindelin via GitGitGadget]

Phillip Wood reported a problem where the built-in rebase did not understand
options like -C1, i.e. it did not expect the option argument.

While investigating how to address this best, I stumbled upon 
OPT_PASSTHRU_ARGV (which I was so far happily unaware of).

Instead of just fixing the -C<n> bug, I decided to simply convert all of the
options intended for git am (or, eventually, for git apply). This happens to
fix that bug, and does so much more: it simplifies the entire logic (and
removes more lines than it adds).

Change since v1:

 * Introduce early parameter validation for the options passed through to 
   git am.

Johannes Schindelin (2):
  rebase: really just passthru the `git am` options
  rebase: validate -C<n> and --whitespace=<mode> parameters early

 builtin/rebase.c          | 108 ++++++++++++++++----------------------
 t/t3406-rebase-message.sh |   7 +++
 2 files changed, 52 insertions(+), 63 deletions(-)


base-commit: 8858448bb49332d353febc078ce4a3abcc962efe
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-76%2Fdscho%2Frebase-Cn-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-76/dscho/rebase-Cn-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/76

Range-diff vs v1:

 1:  dc36a45068 = 1:  dc36a45068 rebase: really just passthru the `git am` options
 -:  ---------- > 2:  4c2ba52766 rebase: validate -C<n> and --whitespace=<mode> parameters early

-- 
gitgitgadget

[PATCH v2 1/2] rebase: really just passthru the `git am` options

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Currently, we parse the options intended for `git am` as if we wanted to
handle them in `git rebase`, and then reconstruct them painstakingly to
define the `git_am_opt` variable.

However, there is a much better way (that I was unaware of, at the time
when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV.
It is intended for exactly this use case, where command-line options
want to be parsed into a separate `argv_array`.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip
Wood, where the built-in rebase failed to understand that the `-C`
option takes an optional argument.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase.c | 98 +++++++++++++++++-------------------------------
 1 file changed, 35 insertions(+), 63 deletions(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 0ee06aa363..96ffa80b71 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -87,7 +87,7 @@ struct rebase_options {
 		REBASE_FORCE = 1<<3,
 		REBASE_INTERACTIVE_EXPLICIT = 1<<4,
 	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
 	const char *action;
 	int signoff;
 	int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as resolved with\n"
 static int run_specific_rebase(struct rebase_options *opts)
 {
 	const char *argv[] = { NULL, NULL };
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
 	int status;
 	const char *backend, *backend_func;
 
@@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options *opts)
 		oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
 	add_var(&script_snippet, "GIT_QUIET",
 		opts->flags & REBASE_NO_QUIET ? "" : "t");
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
 	add_var(&script_snippet, "verbose",
 		opts->flags & REBASE_VERBOSE ? "t" : "");
 	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	struct rebase_options options = {
 		.type = REBASE_UNSPECIFIED,
 		.flags = REBASE_NO_QUIET,
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
 		.allow_rerere_autoupdate  = -1,
 		.allow_empty_message = 1,
 		.git_format_patch_opt = STRBUF_INIT,
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		ACTION_EDIT_TODO,
 		ACTION_SHOW_CURRENT_PATCH,
 	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
 	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
 	struct string_list exec = STRING_LIST_INIT_NODUP;
 	const char *rebase_merges = NULL;
 	int fork_point = -1;
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
 			N_("do not show diffstat of what changed upstream"),
 			PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
 		OPT_BOOL(0, "signoff", &options.signoff,
 			 N_("add a Signed-off-by: line to each commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace", &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"), 0),
 		OPT_BIT('f', "force-rebase", &options.flags,
 			N_("cherry-pick all commits, even if unchanged"),
 			REBASE_FORCE),
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
 			N_("GPG-sign commits"),
 			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
 		OPT_BOOL(0, "autostash", &options.autostash,
 			 N_("automatically stash/stash pop before and after")),
 		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 			 N_("rebase all reachable commits up to the root(s)")),
 		OPT_END(),
 	};
+	int i;
 
 	/*
 	 * NEEDSWORK: Once the builtin rebase has been tested enough
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		    state_dir_base, cmd_live_rebase, buf.buf);
 	}
 
-	if (!(options.flags & REBASE_NO_QUIET))
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
 	}
 
-	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
 	if (options.keep_empty)
 		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
 	}
 
-	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item, "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
 	if (exec.nr) {
 		int i;
 
@@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		break;
 	}
 
-	if (options.git_am_opt.len) {
-		const char *p;
-
+	if (options.git_am_opts.argc) {
 		/* all am options except -q are compatible only with --am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
 
-		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
 			die(_("error: cannot combine interactive options "
 			      "(--interactive, --exec, --rebase-merges, "
 			      "--preserve-merges, --keep-empty, --root + "
 			      "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
 			die(_("error: cannot combine merge options (--merge, "
 			      "--strategy, --strategy-option) with am options "
 			      "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		if (options.type == REBASE_PRESERVE_MERGES)
 			die("cannot combine '--signoff' with "
 			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
 		options.flags |= REBASE_FORCE;
 	}
 
-- 
gitgitgadget

[PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

It is a good idea to error out early upon seeing, say, `-Cbad`, rather
than starting the rebase only to have the `--am` backend complain later.

Let's do this.

The only options accepting parameters which we pass through to `git am`
(which may, or may not, forward them to `git apply`) are `-C` and
`--whitespace`. The other options we pass through do not accept
parameters, so we do not have to validate them here.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/rebase.c          | 12 +++++++++++-
 t/t3406-rebase-message.sh |  7 +++++++
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 96ffa80b71..571cf899d5 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	for (i = 0; i < options.git_am_opts.argc; i++) {
-		const char *option = options.git_am_opts.argv[i];
+		const char *option = options.git_am_opts.argv[i], *p;
 		if (!strcmp(option, "--committer-date-is-author-date") ||
 		    !strcmp(option, "--ignore-date") ||
 		    !strcmp(option, "--whitespace=fix") ||
 		    !strcmp(option, "--whitespace=strip"))
 			options.flags |= REBASE_FORCE;
+		else if (skip_prefix(option, "-C", &p)) {
+			while (*p)
+				if (!isdigit(*(p++)))
+					die(_("switch `C' expects a "
+					      "numerical value"));
+		} else if (skip_prefix(option, "--whitespace=", &p)) {
+			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
+			    strcmp(p, "error") && strcmp(p, "error-all"))
+				die("Invalid whitespace option: '%s'", p);
+		}
 	}
 
 	if (!(options.flags & REBASE_NO_QUIET))
diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
index 0392e36d23..2c79eed4fe 100755
--- a/t/t3406-rebase-message.sh
+++ b/t/t3406-rebase-message.sh
@@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
 	test_i18ngrep "invalid-ref" err
 '
 
+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+	test_i18ngrep "numerical value" err &&
+	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+	test_i18ngrep "Invalid whitespace option" err
+'
+
 test_done
-- 
gitgitgadget

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

[Phillip Wood]

Hi Johannes

Thanks for doing this, I think this patch is good. I've not checked the 
first patch as I think it is the same as before judging from the 
covering letter.

Best Wishes

Phillip

On 14/11/2018 16:25, Johannes Schindelin via GitGitGadget wrote:
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> It is a good idea to error out early upon seeing, say, `-Cbad`, rather
> than starting the rebase only to have the `--am` backend complain later.
> 
> Let's do this.
> 
> The only options accepting parameters which we pass through to `git am`
> (which may, or may not, forward them to `git apply`) are `-C` and
> `--whitespace`. The other options we pass through do not accept
> parameters, so we do not have to validate them here.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>   builtin/rebase.c          | 12 +++++++++++-
>   t/t3406-rebase-message.sh |  7 +++++++
>   2 files changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/builtin/rebase.c b/builtin/rebase.c
> index 96ffa80b71..571cf899d5 100644
> --- a/builtin/rebase.c
> +++ b/builtin/rebase.c
> @@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>   	}
>   
>   	for (i = 0; i < options.git_am_opts.argc; i++) {
> -		const char *option = options.git_am_opts.argv[i];
> +		const char *option = options.git_am_opts.argv[i], *p;
>   		if (!strcmp(option, "--committer-date-is-author-date") ||
>   		    !strcmp(option, "--ignore-date") ||
>   		    !strcmp(option, "--whitespace=fix") ||
>   		    !strcmp(option, "--whitespace=strip"))
>   			options.flags |= REBASE_FORCE;
> +		else if (skip_prefix(option, "-C", &p)) {
> +			while (*p)
> +				if (!isdigit(*(p++)))
> +					die(_("switch `C' expects a "
> +					      "numerical value"));
> +		} else if (skip_prefix(option, "--whitespace=", &p)) {
> +			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
> +			    strcmp(p, "error") && strcmp(p, "error-all"))
> +				die("Invalid whitespace option: '%s'", p);
> +		}
>   	}
>   
>   	if (!(options.flags & REBASE_NO_QUIET))
> diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
> index 0392e36d23..2c79eed4fe 100755
> --- a/t/t3406-rebase-message.sh
> +++ b/t/t3406-rebase-message.sh
> @@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
>   	test_i18ngrep "invalid-ref" err
>   '
>   
> +test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
> +	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
> +	test_i18ngrep "numerical value" err &&
> +	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
> +	test_i18ngrep "Invalid whitespace option" err
> +'
> +
>   test_done
> 

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

[Johannes Schindelin]

Hi Phillip,

On Wed, 14 Nov 2018, Phillip Wood wrote:

> Thanks for doing this, I think this patch is good.

Thanks!

> I've not checked the first patch as I think it is the same as before
> judging from the covering letter.

Indeed, that's what the range-diff said. Sorry for not stating this
explicitly.

Thank you for your review,
Dscho

> 
> Best Wishes
> 
> Phillip
> 
> On 14/11/2018 16:25, Johannes Schindelin via GitGitGadget wrote:
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> > 
> > It is a good idea to error out early upon seeing, say, `-Cbad`, rather
> > than starting the rebase only to have the `--am` backend complain later.
> > 
> > Let's do this.
> > 
> > The only options accepting parameters which we pass through to `git am`
> > (which may, or may not, forward them to `git apply`) are `-C` and
> > `--whitespace`. The other options we pass through do not accept
> > parameters, so we do not have to validate them here.
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >   builtin/rebase.c          | 12 +++++++++++-
> >   t/t3406-rebase-message.sh |  7 +++++++
> >   2 files changed, 18 insertions(+), 1 deletion(-)
> > 
> > diff --git a/builtin/rebase.c b/builtin/rebase.c
> > index 96ffa80b71..571cf899d5 100644
> > --- a/builtin/rebase.c
> > +++ b/builtin/rebase.c
> > @@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv,
> > const char *prefix)
> >    }
> >   
> >   	for (i = 0; i < options.git_am_opts.argc; i++) {
> > -		const char *option = options.git_am_opts.argv[i];
> > +		const char *option = options.git_am_opts.argv[i], *p;
> >     if (!strcmp(option, "--committer-date-is-author-date") ||
> > !strcmp(option, "--ignore-date") ||
> > !strcmp(option, "--whitespace=fix") ||
> > !strcmp(option, "--whitespace=strip"))
> >   			options.flags |= REBASE_FORCE;
> > +		else if (skip_prefix(option, "-C", &p)) {
> > +			while (*p)
> > +				if (!isdigit(*(p++)))
> > +					die(_("switch `C' expects a "
> > +					      "numerical value"));
> > +		} else if (skip_prefix(option, "--whitespace=", &p)) {
> > +			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn")
> > &&
> > +			    strcmp(p, "error") && strcmp(p, "error-all"))
> > +				die("Invalid whitespace option: '%s'", p);
> > +		}
> >    }
> >   
> >   	if (!(options.flags & REBASE_NO_QUIET))
> > diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
> > index 0392e36d23..2c79eed4fe 100755
> > --- a/t/t3406-rebase-message.sh
> > +++ b/t/t3406-rebase-message.sh
> > @@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the
> > invalid ref' '
> >   	test_i18ngrep "invalid-ref" err
> >   '
> >   
> > +test_expect_success 'error out early upon -C<n> or --whitespace=<bad>'
> > '
> > +	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
> > +	test_i18ngrep "numerical value" err &&
> > +	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
> > +	test_i18ngrep "Invalid whitespace option" err
> > +'
> > +
> >   test_done
> > 
> 
> 

Git Test Coverage Report (v2.20.0-rc0)

[Derrick Stolee]

Here is a test coverage report for the uncovered lines introduced in 
v2.20.0-rc0 compared to v2.19.1.

Thanks,

-Stolee

[1] https://dev.azure.com/git/git/_build/results?buildId=263&view=logs

---


apply.c
eccb5a5f3d 4071) return get_oid_hex(p->old_oid_prefix, oid);
517fe807d6 4776) BUG_ON_OPT_NEG(unset);
735ca208c5 4830) return -1;

blame.c
a470beea39  113)  !strcmp(r->index->cache[-1 - pos]->name, path))
a470beea39  272) int pos = index_name_pos(r->index, path, len);
a470beea39  274) mode = r->index->cache[pos]->ce_mode;

builtin/add.c
d1664e73ad builtin/add.c 458) die(_("index file corrupt"));

builtin/am.c
2abf350385 1362) repo_init_revisions(the_repository, &rev_info, NULL);
fce5664805 2117) *opt_value = PATCH_FORMAT_UNKNOWN;

builtin/blame.c
517fe807d6 builtin/blame.c    759) BUG_ON_OPT_NEG(unset);

builtin/cat-file.c
98f425b453 builtin/cat-file.c  56) die("unable to stream %s to stdout", 
oid_to_hex(oid));
0eb8d3767c builtin/cat-file.c 609) return error(_("only one batch option 
may be specified"));

builtin/checkout.c
fa655d8411 builtin/checkout.c  539) return 0;
fa655d8411 builtin/checkout.c  953) return error(_("index file corrupt"));

builtin/difftool.c
4a7e27e957 441) if (oideq(&loid, &roid))

builtin/fast-export.c
4a7e27e957 builtin/fast-export.c  387) if (oideq(&ospec->oid, &spec->oid) &&

builtin/fetch.c
builtin/fsck.c
b29759d89a builtin/fsck.c 613) fprintf(stderr, "Checking %s link\n", 
head_ref_name);
b29759d89a builtin/fsck.c 618) return error("Invalid %s", head_ref_name);
454ea2e4d7 builtin/fsck.c 769) for (p = get_all_packs(the_repository); p;
66ec0390e7 builtin/fsck.c 888) midx_argv[2] = "--object-dir";
66ec0390e7 builtin/fsck.c 889) midx_argv[3] = alt->path;
66ec0390e7 builtin/fsck.c 890) if (run_command(&midx_verify))
66ec0390e7 builtin/fsck.c 891) errors_found |= ERROR_COMMIT_GRAPH;

builtin/gc.c
3029970275 builtin/gc.c 461) ret = error_errno(_("cannot stat '%s'"), 
gc_log_path);
3029970275 builtin/gc.c 470) ret = error_errno(_("cannot read '%s'"), 
gc_log_path);
fec2ed2187 builtin/gc.c 495) die(FAILED_RUN, pack_refs_cmd.argv[0]);
fec2ed2187 builtin/gc.c 498) die(FAILED_RUN, reflog.argv[0]);
3029970275 builtin/gc.c 585) exit(128);
fec2ed2187 builtin/gc.c 637) die(FAILED_RUN, repack.argv[0]);
fec2ed2187 builtin/gc.c 647) die(FAILED_RUN, prune.argv[0]);
fec2ed2187 builtin/gc.c 654) die(FAILED_RUN, prune_worktrees.argv[0]);
fec2ed2187 builtin/gc.c 658) die(FAILED_RUN, rerere.argv[0]);

builtin/grep.c
76e9bdc437 builtin/grep.c  424) grep_read_unlock();
fd6263fb73 builtin/grep.c 1051) warning(_("invalid option combination, 
ignoring --threads"));
fd6263fb73 builtin/grep.c 1057) die(_("invalid number of threads 
specified (%d)"), num_threads);

builtin/help.c
e6e76baaf4 builtin/help.c 429) if (!exclude_guides || alias[0] == '!') {
e6e76baaf4 builtin/help.c 430) printf_ln(_("'%s' is aliased to '%s'"), 
cmd, alias);
e6e76baaf4 builtin/help.c 431) free(alias);
e6e76baaf4 builtin/help.c 432) exit(0);
e6e76baaf4 builtin/help.c 441) fprintf_ln(stderr, _("'%s' is aliased to 
'%s'"), cmd, alias);
e6e76baaf4 builtin/help.c 442) count = split_cmdline(alias, &argv);
e6e76baaf4 builtin/help.c 443) if (count < 0)
e6e76baaf4 builtin/help.c 444) die(_("bad alias.%s string: %s"), cmd,
e6e76baaf4 builtin/help.c 446) free(argv);
e6e76baaf4 builtin/help.c 448) return alias;

builtin/log.c
517fe807d6 builtin/log.c 1196) BUG_ON_OPT_NEG(unset);
2e6fd71a52 builtin/log.c 1472) die(_("failed to infer range-diff ranges"));
ee6cbf712e builtin/log.c 1818) die(_("--interdiff requires 
--cover-letter or single patch"));
8631bf1cdd builtin/log.c 1828) else if (!rdiff_prev)
8631bf1cdd builtin/log.c 1829) die(_("--creation-factor requires 
--range-diff"));
40ce41604d builtin/log.c 1833) die(_("--range-diff requires 
--cover-letter or single patch"));

builtin/multi-pack-index.c
6d68e6a461 35) usage_with_options(builtin_multi_pack_index_usage,
6d68e6a461 39) die(_("too many arguments"));
6d68e6a461 48) die(_("unrecognized verb: %s"), argv[0]);

builtin/pack-objects.c
6a22d52126 builtin/pack-objects.c 1091) continue;
2fa233a554 builtin/pack-objects.c 1512) hashcpy(base_oid.hash, base_sha1);
2fa233a554 builtin/pack-objects.c 1513) if 
(!in_same_island(&delta->idx.oid, &base_oid))
2fa233a554 builtin/pack-objects.c 1514) return 0;
28b8a73080 builtin/pack-objects.c 2793) depth++;
108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, ent, 
depth);
454ea2e4d7 builtin/pack-objects.c 2981) p = get_all_packs(the_repository);

builtin/pack-redundant.c
454ea2e4d7 builtin/pack-redundant.c 580) struct packed_git *p = 
get_all_packs(the_repository);
454ea2e4d7 builtin/pack-redundant.c 595) struct packed_git *p = 
get_all_packs(the_repository);

builtin/pull.c
01a31f3bca 565) die(_("unable to access commit %s"),

builtin/rebase--interactive.c
53bbcfbde7 builtin/rebase--interactive2.c  24) return error(_("no HEAD?"));
53bbcfbde7 builtin/rebase--interactive2.c  51) return 
error_errno(_("could not create temporary %s"), path_state_dir());
53bbcfbde7 builtin/rebase--interactive2.c  57) return 
error_errno(_("could not mark as interactive"));
53bbcfbde7 builtin/rebase--interactive2.c  77) return -1;
53bbcfbde7 builtin/rebase--interactive2.c  81) return -1;
53bbcfbde7 builtin/rebase--interactive2.c  87) free(revisions);
53bbcfbde7 builtin/rebase--interactive2.c  88) free(shortrevisions);
53bbcfbde7 builtin/rebase--interactive2.c  90) return -1;
53bbcfbde7 builtin/rebase--interactive2.c  98) free(revisions);
53bbcfbde7 builtin/rebase--interactive2.c  99) free(shortrevisions);
53bbcfbde7 builtin/rebase--interactive2.c 101) return 
error_errno(_("could not open %s"), rebase_path_todo());
53bbcfbde7 builtin/rebase--interactive2.c 106) 
argv_array_push(&make_script_args, restrict_revision);
53bbcfbde7 builtin/rebase--interactive2.c 114) error(_("could not 
generate todo list"));
53bbcfbde7 builtin/rebase--interactive2.c 206) 
usage_with_options(builtin_rebase_interactive_usage, options);
53bbcfbde7 builtin/rebase--interactive2.c 220) 
warning(_("--[no-]rebase-cousins has no effect without "
0af129b2ed builtin/rebase--interactive2.c 226) die(_("a base commit must 
be provided with --upstream or --onto"));
34b47315d9 builtin/rebase--interactive.c  261) ret = rearrange_squash();
34b47315d9 builtin/rebase--interactive.c  262) break;
34b47315d9 builtin/rebase--interactive.c  264) ret = 
sequencer_add_exec_commands(cmd);
34b47315d9 builtin/rebase--interactive.c  265) break;

builtin/rebase.c
62c23938fa   55) return env;
55071ea248   65) strbuf_trim(&out);
55071ea248   66) ret = !strcmp("true", out.buf);
55071ea248   67) strbuf_release(&out);
002ee2fe68  119) die(_("%s requires an interactive rebase"), option);
f95736288a  152) return error_errno(_("could not read '%s'"), path);
f95736288a  166) return -1;
f95736288a  171) return error(_("could not get 'onto': '%s'"), buf.buf);
f95736288a  182) return -1;
f95736288a  183) } else if (read_one(state_dir_path("head", opts), &buf))
f95736288a  184) return -1;
f95736288a  186) return error(_("invalid orig-head: '%s'"), buf.buf);
f95736288a  190) return -1;
f95736288a  192) opts->flags &= ~REBASE_NO_QUIET;
73d51ed0a5  200) opts->signoff = 1;
73d51ed0a5  201) opts->flags |= REBASE_FORCE;
ead98c111b  208) return -1;
12026a412c  223) return -1;
ba1905a5fe  231) return -1;
ba1905a5fe  239) return -1;
6defce2b02  259) return error(_("Could not read '%s'"), path);
6defce2b02  277) res = error(_("Cannot store %s"), autostash.buf);
6defce2b02  281) return res;
bc24382c2b  379) argv_array_pushf(&child.args,
bc24382c2b  381) oid_to_hex(&opts->restrict_revision->object.oid));
ac7f467fef  515) struct strbuf dir = STRBUF_INIT;
6defce2b02  517) apply_autostash(opts);
ac7f467fef  518) strbuf_addstr(&dir, opts->state_dir);
ac7f467fef  519) remove_dir_recursively(&dir, 0);
ac7f467fef  520) strbuf_release(&dir);
ac7f467fef  521) die("Nothing to do");
3249c1251e  556) ret = -1;
3249c1251e  557) goto leave_reset_head;
bac2a1e36f  561) ret = error(_("could not determine HEAD revision"));
bac2a1e36f  562) goto leave_reset_head;
3249c1251e  580) ret = error(_("could not read index"));
3249c1251e  581) goto leave_reset_head;
bac2a1e36f  585) ret = error(_("failed to find tree of %s"), 
oid_to_hex(oid));
bac2a1e36f  586) goto leave_reset_head;
3249c1251e  590) ret = error(_("failed to find tree of %s"), 
oid_to_hex(oid));
3249c1251e  591) goto leave_reset_head;
ac7f467fef  603) ret = error(_("could not write index"));
3249c1251e  604) goto leave_reset_head;
ac7f467fef  621) } else if (old_orig)
ac7f467fef  622) delete_ref(NULL, "ORIG_HEAD", old_orig, 0);
bff014dac7  655) opts->flags &= !REBASE_DIFFSTAT;
9a48a615b4  689) return 1;
9a48a615b4  705) return 0;
55071ea248  916) const char *path = mkpath("%s/git-legacy-rebase",
55071ea248  919) if (sane_execvp(path, (char **)argv) < 0)
55071ea248  920) die_errno(_("could not exec %s"), path);
0eabf4b95c  938) die(_("It looks like 'git am' is in progress. Cannot 
rebase."));
f28d40d3a9  975) usage_with_options(builtin_rebase_usage,
f95736288a  995) die(_("Cannot read HEAD"));
f95736288a  999) die(_("could not read index"));
f95736288a 1013) exit(1);
122420c295 1026) die(_("could not discard worktree changes"));
122420c295 1029) exit(1);
5e5d96197c 1040) exit(1);
5e5d96197c 1044) die(_("could not move back to %s"),
5a61494539 1055) die(_("could not remove '%s'"), options.state_dir);
c54dacb50e 1074) const char *last_slash = strrchr(options.state_dir, '/');
c54dacb50e 1075) const char *state_dir_base =
c54dacb50e 1076) last_slash ? last_slash + 1 : options.state_dir;
c54dacb50e 1077) const char *cmd_live_rebase =
c54dacb50e 1079) strbuf_reset(&buf);
c54dacb50e 1080) strbuf_addf(&buf, "rm -fr \"%s\"", options.state_dir);
c54dacb50e 1081) die(_("It seems that there is already a %s directory, 
and\n"
3c3588c7d3 1138) else if (strcmp("no-rebase-cousins", rebase_merges))
3c3588c7d3 1139) die(_("Unknown mode: %s"), rebase_merges);
ba1905a5fe 1161) die(_("--strategy requires --merge or --interactive"));
cda614e489 1179) strbuf_addstr(&options.git_format_patch_opt, " 
--progress");
ac7f467fef 1188) options.state_dir = apply_dir();
ac7f467fef 1189) break;
ac7f467fef 1261) die(_("invalid upstream '%s'"), options.upstream_name);
9dba809a69 1267) die(_("Could not create new root commit"));
e65123a71d 1317) die(_("fatal: no such branch/commit '%s'"),
ac7f467fef 1325) die(_("No such ref: %s"), "HEAD");
ac7f467fef 1337) die(_("Could not resolve HEAD to a revision"));
e0333e5c63 1350) die(_("could not read index"));
6defce2b02 1377) die(_("Cannot autostash"));
6defce2b02 1380) die(_("Unexpected stash response: '%s'"),
6defce2b02 1386) die(_("Could not create directory for '%s'"),
6defce2b02 1392) die(_("could not reset --hard"));
e65123a71d 1436) ret = !!error(_("could not parse '%s'"),
e65123a71d 1438) goto cleanup;
e65123a71d 1447) ret = !!error(_("could not switch to "
1ed9c14ff2 1457)  resolve_ref_unsafe("HEAD", 0, NULL, &flag))
1ed9c14ff2 1458) puts(_("HEAD is up to date."));
9a48a615b4 1467)  resolve_ref_unsafe("HEAD", 0, NULL, &flag))
9a48a615b4 1468) puts(_("HEAD is up to date, rebase forced."));

builtin/reflog.c
c9ef0d95eb builtin/reflog.c 580) all_worktrees = 0;
c9ef0d95eb builtin/reflog.c 616) continue;

builtin/remote.c
5025425dff builtin/remote.c  864) return error(_("No such remote: 
'%s'"), name);

builtin/repack.c
16d75fa48d  48) use_delta_islands = git_config_bool(var, value);
16d75fa48d  49) return 0;
2f0c9e9a9b 239) die("repack: Expecting full hex object ID lines only 
from pack-objects.");
2f0c9e9a9b 411) die("repack: Expecting full hex object ID lines only 
from pack-objects.");

builtin/rerere.c
2373b65059 builtin/rerere.c  79) warning(_("'git rerere forget' without 
paths is deprecated"));
2373b65059 builtin/rerere.c 111) die(_("unable to generate diff for 
'%s'"), rerere_path(id, NULL));

builtin/rev-list.c
7c0fe330d5 builtin/rev-list.c 227) die("unexpected missing %s object '%s'",
7c0fe330d5 builtin/rev-list.c 228)     type_name(obj->type), 
oid_to_hex(&obj->oid));

builtin/show-branch.c
9001dc2a74 builtin/show-branch.c 430) if (get_oid(refname + ofs, &tmp) 
|| !oideq(&tmp, oid))
517fe807d6 builtin/show-branch.c 607) BUG_ON_OPT_NEG(unset);

builtin/show-ref.c
517fe807d6 builtin/show-ref.c 154) BUG_ON_OPT_NEG(unset);

builtin/submodule--helper.c
ee69b2a90c 1469) die(_("Invalid update mode '%s' for submodule path '%s'"),
ee69b2a90c 1473) die(_("Invalid update mode '%s' configured for 
submodule path '%s'"),
ee69b2a90c 1476) out->type = sub->update_strategy.type;
ee69b2a90c 1477) out->command = sub->update_strategy.command;
ee69b2a90c 1497) die("submodule--helper update-module-clone expects 
<just-cloned> <path> [<update>]");
e0a862fdaf 1648) url = sub->url;
74d4731da1 2057) die(_("could not get a repository handle for submodule 
'%s'"), path);

builtin/unpack-objects.c
4a7e27e957 builtin/unpack-objects.c 306) if (oideq(&info->base_oid, 
&obj_list[nr].oid) ||

builtin/update-index.c
4a7e27e957 builtin/update-index.c  672) if (oideq(&ce_2->oid, &ce_3->oid) &&

builtin/worktree.c
e5353bef55  60) error_errno(_("failed to delete '%s'"), sb.buf);
e19831c94f 251)     die(_("unable to re-add worktree '%s'"), path);
68a6b3a1bd 793) die(_("cannot move a locked working tree, lock reason: 
%s\nuse 'move -f -f' to override or unlock first"),
f4143101cb 906) die(_("cannot remove a locked working tree, lock reason: 
%s\nuse 'remove -f -f' to override or unlock first"),

bundle.c
2c8ee1f53c 267) error_errno(_("unable to dup bundle descriptor"));
2c8ee1f53c 268) child_process_clear(&pack_objects);
2c8ee1f53c 269) return -1;
2c8ee1f53c 478) rollback_lock_file(&lock);

cache-tree.c
combine-diff.c
0074c9110d  377) state->sline[state->nb-1].p_lno =
0074c9110d  378) xcalloc(state->num_parent, sizeof(unsigned long));

commit-graph.c
20fd6d5799   79) return 0;
6cc017431c  275) return 0;

commit-reach.c
5227c38566 134) return ret;
5227c38566 282) return 1;
5227c38566 314) return ret;
5227c38566 317) return ret;
1d614d41e5 395) return 0;
1d614d41e5 401) return 0;
1d614d41e5 405) return 0;
4fbcca4eff 538) return 1;
b67f6b26e3 559) continue;
b67f6b26e3 570) from->objects[i].item->flags |= assign_flag;
b67f6b26e3 571) continue;
b67f6b26e3 577) result = 0;
b67f6b26e3 578) goto cleanup;

config.c
c780b9cfe8 2303) return val;
c780b9cfe8 2306) if (is_bool)
c780b9cfe8 2307) return val ? 0 : 1;
c780b9cfe8 2309) return val;

date.c
c27cc94fad  904) tm->tm_mon = number-1;
c27cc94fad  908) else if (number > 69 && number < 100)
c27cc94fad  909) tm->tm_year = number;
c27cc94fad  910) else if (number < 38)
c27cc94fad  911) tm->tm_year = 100 + number;
c27cc94fad  952) pending_number(tm, num);

delta-islands.c
c8d521faf7  53) memcpy(b, old, size);
c8d521faf7  73) return 1;
c8d521faf7 118) return 0;
c8d521faf7 130) return 0;
c8d521faf7 187) b->refcount--;
c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);
c8d521faf7 202) continue;
c8d521faf7 212) obj = ((struct tag *)obj)->tagged;
c8d521faf7 213) if (obj) {
c8d521faf7 214) parse_object(the_repository, &obj->oid);
c8d521faf7 215) marks = create_or_get_island_marks(obj);
c8d521faf7 216) island_bitmap_set(marks, island_counter);
c8d521faf7 248) return;
c8d521faf7 268) progress_state = start_progress(_("Propagating island 
marks"), nr);
c8d521faf7 286) die(_("bad tree object %s"), oid_to_hex(&ent->idx.oid));
c8d521faf7 293) continue;
c8d521faf7 297) continue;
c8d521faf7 321) return config_error_nonbool(k);
c8d521faf7 330) die(_("failed to load island regex for '%s': %s"), k, 
re.buf);
c8d521faf7 386) warning(_("island regex from config has "
c8d521faf7 397) strbuf_addch(&island_name, '-');
c8d521faf7 433) continue;
c8d521faf7 436) list[dst] = list[src];

diff-lib.c
9001dc2a74 diff-lib.c 346)     (!oideq(oid, &old_entry->oid) || 
!oideq(&old_entry->oid, &new_entry->oid))) {

diff.c
b78ea5fc35 4130) add_external_diff_name(o->repo, &argv, other, two);

dir.c
8a2c174677  287) name = to_free = xmemdupz(name, namelen);
c46c406ae1 2282) trace_performance_leave("read directory %.*s", len, path);

entry.c
b878579ae7 402) static void mark_colliding_entries(const struct checkout 
*state,
b878579ae7 405) int i, trust_ino = check_stat;
b878579ae7 411) ce->ce_flags |= CE_MATCHED;
b878579ae7 413) for (i = 0; i < state->istate->cache_nr; i++) {
b878579ae7 414) struct cache_entry *dup = state->istate->cache[i];
b878579ae7 416) if (dup == ce)
b878579ae7 417) break;
b878579ae7 419) if (dup->ce_flags & (CE_MATCHED | CE_VALID | 
CE_SKIP_WORKTREE))
b878579ae7 420) continue;
b878579ae7 422) if ((trust_ino && dup->ce_stat_data.sd_ino == st->st_ino) ||
b878579ae7 423)     (!trust_ino && !fspathcmp(ce->name, dup->name))) {
b878579ae7 424) dup->ce_flags |= CE_MATCHED;
b878579ae7 425) break;
b878579ae7 428) }
b878579ae7 488) mark_colliding_entries(state, ce, &st);

fsck.c
fb8952077d  214) die_errno("Could not read '%s'", path);

git.c
a9a60b94cc 322) fprintf_ln(stderr, _("'%s' is aliased to '%s'"),
c6d75bc17a 735) string_list_clear(&cmd_list, 0);

gpg-interface.c
4de9394dcb 155) break;

help.c
26c7d06783 help.c         500) static int get_alias(const char *var, 
const char *value, void *data)
26c7d06783 help.c         502) struct string_list *list = data;
26c7d06783 help.c         504) if (skip_prefix(var, "alias.", &var))
26c7d06783 help.c         505) string_list_append(list, var)->util = 
xstrdup(value);
26c7d06783 help.c         507) return 0;
26c7d06783 help.c         530) printf("\n%s\n", _("Command aliases"));
26c7d06783 help.c         531) ALLOC_ARRAY(aliases, alias_list.nr + 1);
26c7d06783 help.c         532) for (i = 0; i < alias_list.nr; i++) {
26c7d06783 help.c         533) aliases[i].name = alias_list.items[i].string;
26c7d06783 help.c         534) aliases[i].help = alias_list.items[i].util;
26c7d06783 help.c         535) aliases[i].category = 1;
26c7d06783 help.c         537) aliases[alias_list.nr].name = NULL;
26c7d06783 help.c         538) print_command_list(aliases, 1, longest);
26c7d06783 help.c         539) free(aliases);

http.c
21084e84a4  316) free(http_ssl_backend);
21084e84a4  317) http_ssl_backend = xstrdup_or_null(value);
21084e84a4  318) return 0;
93aef7c79b  322) http_schannel_check_revoke = git_config_bool(var, value);
93aef7c79b  323) return 0;
b67d40adbb  327) http_schannel_use_ssl_cainfo = git_config_bool(var, value);
b67d40adbb  328) return 0;
93aef7c79b  833)     !http_schannel_check_revoke) {
93aef7c79b  835) curl_easy_setopt(result, CURLOPT_SSL_OPTIONS, 
CURLSSLOPT_NO_REVOKE);
b67d40adbb  883)     !http_schannel_use_ssl_cainfo) {
b67d40adbb  884) curl_easy_setopt(result, CURLOPT_CAINFO, NULL);

ident.c
501afcb8b0 172) strbuf_addstr(&git_default_email, email);
501afcb8b0 173) free((char *)email);

list-objects-filter-options.c
bc5975d24f  55) if (errbuf) {
bc5975d24f  56) strbuf_addstr(
bc5975d24f  60) return 1;
cc0b05a4cc  86) if (errbuf)

list-objects-filter.c
list-objects.c
f447a499db 200) ctx->show_object(obj, base->buf, ctx->show_data);

ll-merge.c
d64324cb60 380) marker_size = DEFAULT_CONFLICT_MARKER_SIZE;

log-tree.c
4a7e27e957 477) if (oideq(&parent->item->object.oid, oid))

mailinfo.c
3aa4d81f88  992) len--;
3aa4d81f88  998) handle_filter(mi, prev);
3aa4d81f88  999) strbuf_reset(prev);
3aa4d81f88 1090) handle_filter(mi, &prev);

midx.c
4d80560c54   58) error_errno(_("failed to read %s"), midx_name);
4d80560c54   59) goto cleanup_fail;
4d80560c54   65) error(_("multi-pack-index file %s is too small"), 
midx_name);
4d80560c54   66) goto cleanup_fail;
0d5b3a5ef7  146) die(_("multi-pack-index missing required OID lookup 
chunk"));
662148c435  148) die(_("multi-pack-index missing required object offsets 
chunk"));
4d80560c54  173) munmap(midx_map, midx_size);
4d80560c54  175) close(fd);
1dcd9f2043  184) return;
3715a6335c  266) return 0;
fe1ed56f5e  413) warning(_("failed to open pack-index '%s'"),
fe1ed56f5e  415) close_pack(packs->list[packs->nr]);
fe1ed56f5e  416) FREE_AND_NULL(packs->list[packs->nr]);
fe1ed56f5e  417) return;
a40498a126  490) return 1;
fe1ed56f5e  507) die(_("failed to locate object %d in packfile"), 
cur_object);
fc59e74844  769) die_errno(_("unable to create leading directories of %s"),
525e18c04b  943) die(_("failed to clear multi-pack-index at %s"), midx);
56ee7ff156  969) return 0;
cc6af73c02 1010) midx_report(_("failed to load pack-index for packfile %s"),
cc6af73c02 1011)     e.p->pack_name);
cc6af73c02 1012) break;

name-hash.c
2179045fd0 532) die(_("unable to create lazy_dir thread: %s"), 
strerror(err));
2179045fd0 554) die(_("unable to create lazy_name thread: %s"), 
strerror(err));
2179045fd0 560) die(_("unable to join lazy_name thread: %s"), 
strerror(err));

oidmap.c
cc00e5ce6b 11) return !oideq(&entry_->oid,

oidset.c
8b2f8cbcb1 29) kh_del_oid(&set->set, pos);
8b2f8cbcb1 30) return 1;

pack-bitmap.c
30cdc33fba 1130) return 0;

pack-objects.c
108f530385 172) REALLOC_ARRAY(pdata->tree_depth, pdata->nr_alloc);
fe0ac2fb7f 175) REALLOC_ARRAY(pdata->layer, pdata->nr_alloc);
108f530385 192) pdata->tree_depth[pdata->nr_objects - 1] = 0;
fe0ac2fb7f 195) pdata->layer[pdata->nr_objects - 1] = 0;

packfile.c
1127a98cce  117) return error("index file %s is too small", path);
1127a98cce  119) return error("empty data");
fe1ed56f5e  211) if (open_pack_index(p))
fe1ed56f5e  212) return 0;
fe1ed56f5e  213) level1_ofs = p->index_data;
17c35c8969  490) break;
17c35c8969  548) return 0;

preload-index.c
ae9af12287  63) struct progress_data *pd = p->progress;
ae9af12287  65) pthread_mutex_lock(&pd->mutex);
ae9af12287  66) pd->n += last_nr - nr;
ae9af12287  67) display_progress(pd->progress, pd->n);
ae9af12287  68) pthread_mutex_unlock(&pd->mutex);
ae9af12287  69) last_nr = nr;
ae9af12287  83) struct progress_data *pd = p->progress;
ae9af12287  85) pthread_mutex_lock(&pd->mutex);
ae9af12287  86) display_progress(pd->progress, pd->n + last_nr);
ae9af12287  87) pthread_mutex_unlock(&pd->mutex);
ae9af12287 118) pd.progress = start_delayed_progress(_("Refreshing 
index"), index->cache_nr);
ae9af12287 119) pthread_mutex_init(&pd.mutex, NULL);
ae9af12287 132) p->progress = &pd;
2179045fd0 137) die(_("unable to create threaded lstat: %s"), 
strerror(err));

read-cache.c
ae9af12287 1490) progress = start_delayed_progress(_("Refresh index"),
ae9af12287 1491)   istate->cache_nr);
ae9af12287 1539) display_progress(progress, i);
ae9af12287 1572) display_progress(progress, istate->cache_nr);
ae9af12287 1573) stop_progress(&progress);
252d079cbd 1784) const unsigned char *cp = (const unsigned char *)name;
252d079cbd 1788) strip_len = decode_varint(&cp);
77ff1127a4 1789) if (previous_ce) {
77ff1127a4 1790) previous_len = previous_ce->ce_namelen;
77ff1127a4 1791) if (previous_len < strip_len)
252d079cbd 1792) die(_("malformed name field in the index, near path '%s'"),
77ff1127a4 1793) previous_ce->name);
77ff1127a4 1794) copy_len = previous_len - strip_len;
252d079cbd 1796) name = (const char *)cp;
252d079cbd 1802) len += copy_len;
252d079cbd 1823) if (copy_len)
252d079cbd 1824) memcpy(ce->name, previous_ce->name, copy_len);
252d079cbd 1825) memcpy(ce->name + copy_len, name, len + 1 - copy_len);
252d079cbd 1826) *ent_size = (name - ((char *)ondisk)) + len + 1 - copy_len;
abb4bb8384 1959) munmap((void *)p->mmap, p->mmap_size);
abb4bb8384 1960) die(_("index file corrupt"));
77ff1127a4 2001) mem_pool_init(&istate->ce_mem_pool,
77ff1127a4 2039) static void *load_cache_entries_thread(void *_data)
77ff1127a4 2041) struct load_cache_entries_thread_data *p = _data;
77ff1127a4 2045) for (i = p->ieot_start; i < p->ieot_start + 
p->ieot_blocks; i++) {
77ff1127a4 2046) p->consumed += load_cache_entry_block(p->istate, 
p->ce_mem_pool,
77ff1127a4 2047) p->offset, p->ieot->entries[i].nr, p->mmap, 
p->ieot->entries[i].offset, NULL);
77ff1127a4 2048) p->offset += p->ieot->entries[i].nr;
77ff1127a4 2050) return NULL;
77ff1127a4 2053) static unsigned long load_cache_entries_threaded(struct 
index_state *istate, const char *mmap, size_t mmap_size,
77ff1127a4 2058) unsigned long consumed = 0;
77ff1127a4 2061) if (istate->name_hash_initialized)
77ff1127a4 2064) mem_pool_init(&istate->ce_mem_pool, 0);
77ff1127a4 2067) if (nr_threads > ieot->nr)
77ff1127a4 2068) nr_threads = ieot->nr;
77ff1127a4 2069) data = xcalloc(nr_threads, sizeof(*data));
77ff1127a4 2071) offset = ieot_start = 0;
77ff1127a4 2072) ieot_blocks = DIV_ROUND_UP(ieot->nr, nr_threads);
77ff1127a4 2073) for (i = 0; i < nr_threads; i++) {
77ff1127a4 2074) struct load_cache_entries_thread_data *p = &data[i];
77ff1127a4 2077) if (ieot_start + ieot_blocks > ieot->nr)
77ff1127a4 2078) ieot_blocks = ieot->nr - ieot_start;
77ff1127a4 2080) p->istate = istate;
77ff1127a4 2081) p->offset = offset;
77ff1127a4 2082) p->mmap = mmap;
77ff1127a4 2083) p->ieot = ieot;
77ff1127a4 2084) p->ieot_start = ieot_start;
77ff1127a4 2085) p->ieot_blocks = ieot_blocks;
77ff1127a4 2088) nr = 0;
77ff1127a4 2089) for (j = p->ieot_start; j < p->ieot_start + 
p->ieot_blocks; j++)
77ff1127a4 2090) nr += p->ieot->entries[j].nr;
77ff1127a4 2091) if (istate->version == 4) {
77ff1127a4 2092) mem_pool_init(&p->ce_mem_pool,
77ff1127a4 2095) mem_pool_init(&p->ce_mem_pool,
77ff1127a4 2099) err = pthread_create(&p->pthread, NULL, 
load_cache_entries_thread, p);
77ff1127a4 2100) if (err)
77ff1127a4 2101) die(_("unable to create load_cache_entries thread: 
%s"), strerror(err));
77ff1127a4 2104) for (j = 0; j < ieot_blocks; j++)
77ff1127a4 2105) offset += ieot->entries[ieot_start + j].nr;
77ff1127a4 2106) ieot_start += ieot_blocks;
77ff1127a4 2109) for (i = 0; i < nr_threads; i++) {
77ff1127a4 2110) struct load_cache_entries_thread_data *p = &data[i];
77ff1127a4 2112) err = pthread_join(p->pthread, NULL);
77ff1127a4 2113) if (err)
77ff1127a4 2114) die(_("unable to join load_cache_entries thread: %s"), 
strerror(err));
77ff1127a4 2115) mem_pool_combine(istate->ce_mem_pool, p->ce_mem_pool);
77ff1127a4 2116) consumed += p->consumed;
77ff1127a4 2119) free(data);
77ff1127a4 2121) return consumed;
abb4bb8384 2193) extension_offset = read_eoie_extension(mmap, mmap_size);
abb4bb8384 2194) if (extension_offset) {
abb4bb8384 2197) p.src_offset = extension_offset;
abb4bb8384 2198) err = pthread_create(&p.pthread, NULL, 
load_index_extensions, &p);
abb4bb8384 2199) if (err)
abb4bb8384 2200) die(_("unable to create load_index_extensions thread: 
%s"), strerror(err));
abb4bb8384 2202) nr_threads--;
77ff1127a4 2211) ieot = read_ieot_extension(mmap, mmap_size, 
extension_offset);
77ff1127a4 2214) src_offset += load_cache_entries_threaded(istate, mmap, 
mmap_size, src_offset, nr_threads, ieot);
77ff1127a4 2215) free(ieot);
abb4bb8384 2225) int ret = pthread_join(p.pthread, NULL);
abb4bb8384 2226) if (ret)
abb4bb8384 2227) die(_("unable to join load_index_extensions thread: 
%s"), strerror(ret));
3255089ada 2769) ieot_blocks = nr_threads;
77ff1127a4 2770) if (ieot_blocks > istate->cache_nr)
77ff1127a4 2771) ieot_blocks = istate->cache_nr;
3255089ada 2779) ieot = xcalloc(1, sizeof(struct index_entry_offset_table)
3255089ada 2780) + (ieot_blocks * sizeof(struct index_entry_offset)));
77ff1127a4 2781) ieot_entries = DIV_ROUND_UP(entries, ieot_blocks);
3255089ada 2787) free(ieot);
3b1d9e045e 2788) return -1;
3255089ada 2814) ieot->entries[ieot->nr].nr = nr;
3255089ada 2815) ieot->entries[ieot->nr].offset = offset;
3255089ada 2816) ieot->nr++;
3255089ada 2822) if (previous_name)
3255089ada 2823) previous_name->buf[0] = 0;
3255089ada 2824) nr = 0;
3255089ada 2825) offset = lseek(newfd, 0, SEEK_CUR);
3255089ada 2826) if (offset < 0) {
3255089ada 2827) free(ieot);
3255089ada 2828) return -1;
3255089ada 2830) offset += write_buffer_len;
3255089ada 2840) ieot->entries[ieot->nr].nr = nr;
3255089ada 2841) ieot->entries[ieot->nr].offset = offset;
3255089ada 2842) ieot->nr++;
3255089ada 2854) free(ieot);
3b1d9e045e 2855) return -1;
3255089ada 2868) struct strbuf sb = STRBUF_INIT;
3255089ada 2870) write_ieot_extension(&sb, ieot);
3255089ada 2871) err = write_index_ext_header(&c, &eoie_c, newfd, 
CACHE_EXT_INDEXENTRYOFFSETTABLE, sb.len) < 0
3255089ada 2872) || ce_write(&c, newfd, sb.buf, sb.len) < 0;
3255089ada 2873) strbuf_release(&sb);
3255089ada 2874) free(ieot);
3255089ada 2875) if (err)
3255089ada 2876) return -1;
3b1d9e045e 3363) static size_t read_eoie_extension(const char *mmap, 
size_t mmap_size)
3b1d9e045e 3381) if (mmap_size < sizeof(struct cache_header) + 
EOIE_SIZE_WITH_HEADER + the_hash_algo->rawsz)
3b1d9e045e 3382) return 0;
3b1d9e045e 3385) index = eoie = mmap + mmap_size - EOIE_SIZE_WITH_HEADER 
- the_hash_algo->rawsz;
3b1d9e045e 3386) if (CACHE_EXT(index) != CACHE_EXT_ENDOFINDEXENTRIES)
3b1d9e045e 3387) return 0;
3b1d9e045e 3388) index += sizeof(uint32_t);
3b1d9e045e 3391) extsize = get_be32(index);
3b1d9e045e 3392) if (extsize != EOIE_SIZE)
3b1d9e045e 3393) return 0;
3b1d9e045e 3394) index += sizeof(uint32_t);
3b1d9e045e 3400) offset = get_be32(index);
3b1d9e045e 3401) if (mmap + offset < mmap + sizeof(struct cache_header))
3b1d9e045e 3402) return 0;
3b1d9e045e 3403) if (mmap + offset >= eoie)
3b1d9e045e 3404) return 0;
3b1d9e045e 3405) index += sizeof(uint32_t);
3b1d9e045e 3416) src_offset = offset;
3b1d9e045e 3417) the_hash_algo->init_fn(&c);
3b1d9e045e 3418) while (src_offset < mmap_size - the_hash_algo->rawsz - 
EOIE_SIZE_WITH_HEADER) {
3b1d9e045e 3426) memcpy(&extsize, mmap + src_offset + 4, 4);
3b1d9e045e 3427) extsize = ntohl(extsize);
3b1d9e045e 3430) if (src_offset + 8 + extsize < src_offset)
3b1d9e045e 3431) return 0;
3b1d9e045e 3433) the_hash_algo->update_fn(&c, mmap + src_offset, 8);
3b1d9e045e 3435) src_offset += 8;
3b1d9e045e 3436) src_offset += extsize;
3b1d9e045e 3438) the_hash_algo->final_fn(hash, &c);
3b1d9e045e 3439) if (!hasheq(hash, (const unsigned char *)index))
3b1d9e045e 3440) return 0;
3b1d9e045e 3443) if (src_offset != mmap_size - the_hash_algo->rawsz - 
EOIE_SIZE_WITH_HEADER)
3b1d9e045e 3444) return 0;
3b1d9e045e 3446) return offset;
3255089ada 3465) static struct index_entry_offset_table 
*read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset)
3255089ada 3467)        const char *index = NULL;
3255089ada 3473)        if (!offset)
3255089ada 3474)        return NULL;
3255089ada 3475)        while (offset <= mmap_size - 
the_hash_algo->rawsz - 8) {
3255089ada 3476)        extsize = get_be32(mmap + offset + 4);
3255089ada 3477)        if (CACHE_EXT((mmap + offset)) == 
CACHE_EXT_INDEXENTRYOFFSETTABLE) {
3255089ada 3478)        index = mmap + offset + 4 + 4;
3255089ada 3479)        break;
3255089ada 3481)        offset += 8;
3255089ada 3482)        offset += extsize;
3255089ada 3484)        if (!index)
3255089ada 3485)        return NULL;
3255089ada 3488)        ext_version = get_be32(index);
3255089ada 3489)        if (ext_version != IEOT_VERSION) {
3255089ada 3490)        error("invalid IEOT version %d", ext_version);
3255089ada 3491)        return NULL;
3255089ada 3493)        index += sizeof(uint32_t);
3255089ada 3496)        nr = (extsize - sizeof(uint32_t)) / 
(sizeof(uint32_t) + sizeof(uint32_t));
3255089ada 3497)        if (!nr) {
3255089ada 3498)        error("invalid number of IEOT entries %d", nr);
3255089ada 3499)        return NULL;
3255089ada 3501)        ieot = xmalloc(sizeof(struct 
index_entry_offset_table)
3255089ada 3502)        + (nr * sizeof(struct index_entry_offset)));
3255089ada 3503)        ieot->nr = nr;
3255089ada 3504)        for (i = 0; i < nr; i++) {
3255089ada 3505)        ieot->entries[i].offset = get_be32(index);
3255089ada 3506)        index += sizeof(uint32_t);
3255089ada 3507)        ieot->entries[i].nr = get_be32(index);
3255089ada 3508)        index += sizeof(uint32_t);
3255089ada 3511)        return ieot;
3255089ada 3514) static void write_ieot_extension(struct strbuf *sb, 
struct index_entry_offset_table *ieot)
3255089ada 3520)        put_be32(&buffer, IEOT_VERSION);
3255089ada 3521)        strbuf_add(sb, &buffer, sizeof(uint32_t));
3255089ada 3524)        for (i = 0; i < ieot->nr; i++) {
3255089ada 3527)        put_be32(&buffer, ieot->entries[i].offset);
3255089ada 3528)        strbuf_add(sb, &buffer, sizeof(uint32_t));
3255089ada 3531)        put_be32(&buffer, ieot->entries[i].nr);
3255089ada 3532)        strbuf_add(sb, &buffer, sizeof(uint32_t));
3255089ada 3534) }

rebase-interactive.c
64a43cbd5d 62) return error_errno(_("could not read '%s'."), todo_file);
64a43cbd5d 66) strbuf_release(&buf);
64a43cbd5d 67) return -1;
a9f5476fbc 75) return error_errno(_("could not read '%s'."), todo_file);
a9f5476fbc 79) strbuf_release(&buf);
a9f5476fbc 80) return -1;
64a43cbd5d 86) return -1;

ref-filter.c
f0062d3b74 1039) v->s = xstrdup("");
f0062d3b74 1302) free((char *)to_free);
f0062d3b74 1303) return xstrdup("");
f0062d3b74 1340) free((char *)to_free);
f0062d3b74 1341) return xstrdup("");
f0062d3b74 1391) *s = xstrdup("=");
f0062d3b74 1393) *s = xstrdup("<");
f0062d3b74 1518) ref->symref = xstrdup("");
f0062d3b74 1587) v->s = xstrdup("");

refs.c
3a3b9d8cde  661) return 0;
4a6067cda5 1431) return 0;

refs/files-backend.c
refs/packed-backend.c
9001dc2a74 1163) } else if (!oideq(&update->old_oid, iter->oid)) {

refs/ref-cache.c
9001dc2a74 275) if (!oideq(&ref1->u.value.oid, &ref2->u.value.oid))

remote.c
85daa01f6b 1219) continue;
85daa01f6b 1225) continue;

rerere.c
2373b65059  217) die(_("corrupt MERGE_RR"));
2373b65059  226) die(_("corrupt MERGE_RR"));
2373b65059  229) die(_("corrupt MERGE_RR"));
2373b65059  264) die(_("unable to write rerere record"));
2373b65059  269) die(_("unable to write rerere record"));
4af32207bc  376) break;
4af32207bc  380) strbuf_addbuf(&two, &conflict);
c0f16f8e14  384) break;
c0f16f8e14  388) break;
c0f16f8e14  392) break;
2373b65059  480) return error_errno(_("could not open '%s'"), path);
2373b65059  485) error_errno(_("could not write '%s'"), output);
2373b65059  495) error(_("there were errors while writing '%s' (%s)"),
2373b65059  498) io.io.wrerror = error_errno(_("failed to flush '%s'"), 
path);
2373b65059  565) return error(_("index file corrupt"));
2373b65059  599) return error(_("index file corrupt"));
2373b65059  684) warning_errno(_("failed utime() on '%s'"),
2373b65059  690) return error_errno(_("could not open '%s'"), path);
2373b65059  692) error_errno(_("could not write '%s'"), path);
2373b65059  694) return error_errno(_("writing '%s' failed"), path);
2373b65059  720) die(_("unable to write new index file"));
2373b65059  803) die_errno(_("cannot unlink stray '%s'"), path);
2373b65059 1057) error(_("failed to update conflicted state in '%s'"), 
path);
2373b65059 1075) error(_("no remembered resolution for '%s'"), path);
2373b65059 1077) error_errno(_("cannot unlink '%s'"), filename);
2373b65059 1111) return error(_("index file corrupt"));
2373b65059 1199) die_errno(_("unable to open rr-cache directory"));

revision.c
2abf350385 1538) if (ce_path_match(istate, ce, &revs->prune_data, NULL)) {
2abf350385 1544) while ((i+1 < istate->cache_nr) &&
2abf350385 1545)        ce_same_name(ce, istate->cache[i+1]))
b45424181e 2942) return;
b45424181e 2945) return;
b45424181e 2951) c->object.flags |= UNINTERESTING;
b45424181e 2954) return;
b45424181e 2957) mark_parents_uninteresting(c);
b45424181e 2980) return;
b45424181e 2983) return;
b45424181e 3048) continue;
f0d9cc4196 3097) if (!revs->ignore_missing_links)
f0d9cc4196 3098) die("Failed to traverse parents of commit %s",
f0d9cc4196 3099)     oid_to_hex(&commit->object.oid));
b45424181e 3107) continue;
4a7e27e957 3473)     oideq(&p->item->object.oid, &commit->object.oid))

run-command.c
2179045fd0 1229) error(_("cannot create async thread: %s"), strerror(err));

send-pack.c
c0e40a2d66 207) close(fd[1]);

sequencer.c
bcd33ec25f  683) np = strchrnul(buf, '\n');
bcd33ec25f  684) return error(_("no key present in '%.*s'"),
bcd33ec25f  695) return error(_("unable to dequote value of '%s'"),
bcd33ec25f  737) goto finish;
bcd33ec25f  742) name_i = error(_("'GIT_AUTHOR_NAME' already given"));
bcd33ec25f  747) email_i = error(_("'GIT_AUTHOR_EMAIL' already given"));
bcd33ec25f  752) date_i = error(_("'GIT_AUTHOR_DATE' already given"));
bcd33ec25f  756) err = error(_("unknown variable '%s'"),
bcd33ec25f  761) error(_("missing 'GIT_AUTHOR_NAME'"));
bcd33ec25f  763) error(_("missing 'GIT_AUTHOR_EMAIL'"));
bcd33ec25f  765) error(_("missing 'GIT_AUTHOR_DATE'"));
65850686cf 2329) return;
65850686cf 2426) write_file(rebase_path_quiet(), "%s\n", quiet);
2c58483a59 3427) return error(_("could not checkout %s"), commit);
4df66c40b0 3441) return error(_("%s: not a valid OID"), orig_head);
71f82465b1 3461) fprintf(stderr, _("Stopped at HEAD\n"));
b97e187364 4827) return -1;
b97e187364 4830) return -1;
b97e187364 4836) return error_errno(_("could not read '%s'."), todo_file);
b97e187364 4839) todo_list_release(&todo_list);
b97e187364 4840) return error(_("unusable todo list: '%s'"), todo_file);
b97e187364 4859) todo_list_release(&todo_list);
b97e187364 4860) return -1;
b97e187364 4864) return error(_("could not copy '%s' to '%s'."), todo_file,
b97e187364 4868) return error(_("could not transform the todo list"));
b97e187364 4897) return error(_("could not transform the todo list"));
b97e187364 4900) return error(_("could not skip unnecessary pick 
commands"));
b97e187364 4906) return -1;

setup.c
58b284a2e9  413) return config_error_nonbool(var);

sha1-file.c
67947c34ae sha1-file.c 2225) if (!hasheq(expected_sha1, real_sha1)) {

sha1-name.c
8aac67a174 sha1-name.c  162) return;

split-index.c
e3d837989e 335) ce->ce_flags |= CE_UPDATE_IN_BASE;

strbuf.c
f95736288a  127) --sb->len;

submodule-config.c
bcbc780d14 739) return CONFIG_INVALID_KEY;
45f5ef3d77 754) warning(_("Could not update .gitmodules entry %s"), key);

trace.c
c46c406ae1 189) now = getnanotime();
c46c406ae1 190) perf_start_times[perf_indent] = now;
c46c406ae1 191) if (perf_indent + 1 < ARRAY_SIZE(perf_start_times))
c46c406ae1 192) perf_indent++;
c46c406ae1 195) return now;
c46c406ae1 211) if (perf_indent >= strlen(space))
c46c406ae1 214) strbuf_addf(&buf, ":%.*s ", perf_indent, space);
c46c406ae1 317) void trace_performance_leave_fl(const char *file, int line,
c46c406ae1 323) if (perf_indent)
c46c406ae1 324) perf_indent--;
c46c406ae1 326) if (!format) /* Allow callers to leave without tracing 
anything */
c46c406ae1 327) return;
c46c406ae1 329) since = perf_start_times[perf_indent];
c46c406ae1 330) va_start(ap, format);
c46c406ae1 331) trace_performance_vprintf_fl(file, line, nanos - since, 
format, ap);
c46c406ae1 332) va_end(ap);
c46c406ae1 477) trace_performance_leave("git command:%s", command_line.buf);
c46c406ae1 485) if (!command_line.len)
c46c406ae1 490) trace_performance_enter();

transport.c
unpack-trees.c
b878579ae7  360) string_list_append(&list, ce->name);
b878579ae7  361) ce->ce_flags &= ~CE_MATCHED;
b878579ae7  368) warning(_("the following paths have collided (e.g. 
case-sensitive paths\n"
b878579ae7  372) for (i = 0; i < list.nr; i++)
b878579ae7  373) fprintf(stderr, "  '%s'\n", list.items[i].string);
f1e11c6510  777) free(tree_ce);
b4da37380b  778) return rc;
b4da37380b  785) printf("Unpacked %d entries from %s to %s using 
cache-tree\n",
b4da37380b  787)        o->src_index->cache[pos]->name,
b4da37380b  788)        o->src_index->cache[pos + nr_entries - 1]->name);

upload-pack.c
1d1243fe63 1403) deepen(INFINITE_DEPTH, data->deepen_relative, 
&data->shallows,

worktree.c
3a3b9d8cde 495) return -1;
3a3b9d8cde 508) return -1;
3a3b9d8cde 517) return -1;
ab3e1f78ae 537) break;

wt-status.c
f3bd35fa0d  671) s->committable = 1;
73ba5d78b4 1958) if (s->state.rebase_in_progress ||
73ba5d78b4 1959)     s->state.rebase_interactive_in_progress)
73ba5d78b4 1960) branch_name = s->state.onto;
73ba5d78b4 1961) else if (s->state.detached_from)
73ba5d78b4 1962) branch_name = s->state.detached_from;

xdiff-interface.c
xdiff/xutils.c
611e42a598 405) return -1;

Commits introducing uncovered code:
Ævar Arnfjörð Bjarmason      62c23938f: tests: add a special setup where 
rebase.useBuiltin is off
Alban Gruin      0af129b2e: rebase--interactive2: rewrite the submodes 
of interactive rebase in C
Alban Gruin      2c58483a5: rebase -i: rewrite setup_reflog_action() in C
Alban Gruin      34b47315d: rebase -i: move rebase--helper modes to 
rebase--interactive
Alban Gruin      4df66c40b: rebase -i: rewrite checkout_onto() in C
Alban Gruin      53bbcfbde: rebase -i: implement the main part of 
interactive rebase as a builtin
Alban Gruin      64a43cbd5: rebase -i: rewrite the edit-todo 
functionality in C
Alban Gruin      65850686c: rebase -i: rewrite write_basic_state() in C
Alban Gruin      a9f5476fb: sequencer: refactor append_todo_help() to 
write its message to a buffer
Alban Gruin      b97e18736: rebase -i: rewrite complete_action() in C
Antonio Ospite      45f5ef3d7: submodule: factor out a 
config_set_in_gitmodules_file_gently function
Antonio Ospite      76e9bdc43: submodule: support reading .gitmodules 
when it's not in the working tree
Antonio Ospite      bcbc780d1: submodule: add a 
print_config_from_gitmodules() helper
Ben Peart      3255089ad: ieot: add Index Entry Offset Table (IEOT) 
extension
Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) extension
Ben Peart      77ff1127a: read-cache: load cache entries on worker threads
Ben Peart      abb4bb838: read-cache: load cache extensions on a worker 
thread
Ben Peart      c780b9cfe: config: add new index.threads config setting
Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing 
read_cache_preload()
Ben Peart      fa655d841: checkout: optimize "git checkout -b <new_branch>"
Brendan Forster      93aef7c79: http: add support for disabling SSL 
revocation checks in cURL
brian m. carlson      2f0c9e9a9: builtin/repack: replace hard-coded 
constants
brian m. carlson      eccb5a5f3: apply: rename new_sha1_prefix and 
old_sha1_prefix
Christian Couder      108f53038: pack-objects: move tree_depth into 
'struct packing_data'
Christian Couder      fe0ac2fb7: pack-objects: move 'layer' into 'struct 
packing_data'
Derrick Stolee      0d5b3a5ef: midx: write object ids in a chunk
Derrick Stolee      17c35c896: packfile: skip loading index if in 
multi-pack-index
Derrick Stolee      1d614d41e: commit-reach: move ref_newer from remote.c
Derrick Stolee      1dcd9f204: midx: close multi-pack-index on repack
Derrick Stolee      20fd6d579: commit-graph: not compatible with grafts
Derrick Stolee      3715a6335: midx: read objects from multi-pack-index
Derrick Stolee      454ea2e4d: treewide: use get_all_packs
Derrick Stolee      4d80560c5: multi-pack-index: load into memory
Derrick Stolee      4fbcca4ef: commit-reach: make can_all_from_reach... 
linear
Derrick Stolee      5227c3856: commit-reach: move walk methods from commit.c
Derrick Stolee      525e18c04: midx: clear midx on repack
Derrick Stolee      56ee7ff15: multi-pack-index: add 'verify' verb
Derrick Stolee      662148c43: midx: write object offsets
Derrick Stolee      66ec0390e: fsck: verify multi-pack-index
Derrick Stolee      6a22d5212: pack-objects: consider packs in 
multi-pack-index
Derrick Stolee      6cc017431: commit-reach: use can_all_from_reach
Derrick Stolee      6d68e6a46: multi-pack-index: provide more helpful 
usage info
Derrick Stolee      85daa01f6: remote: make add_missing_tags() linear
Derrick Stolee      8aac67a17: midx: use midx in abbreviation calculations
Derrick Stolee      a40498a12: midx: use existing midx when writing new one
Derrick Stolee      b45424181: revision.c: generation-based topo-order 
algorithm
Derrick Stolee      b67f6b26e: commit-reach: properly peel tags
Derrick Stolee      cc6af73c0: multi-pack-index: verify object offsets
Derrick Stolee      f0d9cc419: revision.c: begin refactoring 
--topo-order logic
Derrick Stolee      fc59e7484: midx: write header information to lockfile
Derrick Stolee      fe1ed56f5: midx: sort and deduplicate objects from 
packfiles
Duy Nguyen      b878579ae: clone: report duplicate entries on 
case-insensitive filesystems
Eric Sunshine      2e6fd71a5: format-patch: extend --range-diff to 
accept revision range
Eric Sunshine      40ce41604: format-patch: allow --range-diff to apply 
to a lone-patch
Eric Sunshine      68a6b3a1b: worktree: teach 'move' to override lock 
when --force given twice
Eric Sunshine      8631bf1cd: format-patch: add --creation-factor tweak 
for --range-diff
Eric Sunshine      e19831c94: worktree: teach 'add' to respect --force 
for registered but missing path
Eric Sunshine      e5353bef5: worktree: move delete_git_dir() earlier in 
file for upcoming new callers
Eric Sunshine      ee6cbf712: format-patch: allow --interdiff to apply 
to a lone-patch
Eric Sunshine      f4143101c: worktree: teach 'remove' to override lock 
when --force given twice
Jeff King      0074c9110: combine-diff: use an xdiff hunk callback
Jeff King      01a31f3bc: pull: handle --verify-signatures for unborn branch
Jeff King      0eb8d3767: cat-file: report an error on multiple --batch 
options
Jeff King      16d75fa48: repack: add delta-islands support
Jeff King      28b8a7308: pack-objects: add delta-islands support
Jeff King      2c8ee1f53: bundle: dup() output descriptor closer to 
point-of-use
Jeff King      2fa233a55: pack-objects: handle island check for 
"external" delta base
Jeff King      30cdc33fb: pack-bitmap: save "have" bitmap from walk
Jeff King      4a7e27e95: convert "oidcmp() == 0" to oideq()
Jeff King      517fe807d: assert NOARG/NONEG behavior of parse-options 
callbacks
Jeff King      611e42a59: xdiff: provide a separate emit callback for hunks
Jeff King      67947c34a: convert "hashcmp() != 0" to "!hasheq()"
Jeff King      735ca208c: apply: return -1 from option callback instead 
of calling exit(1)
Jeff King      8a2c17467: pathspec: handle non-terminated strings with 
:(attr)
Jeff King      9001dc2a7: convert "oidcmp() != 0" to "!oideq()"
Jeff King      98f425b45: cat-file: handle streaming failures consistently
Jeff King      c27cc94fa: approxidate: handle pending number for "specials"
Jeff King      c8d521faf: Add delta-islands.{c,h}
Jeff King      cc00e5ce6: convert hashmap comparison functions to oideq()
Jeff King      fce566480: am: handle --no-patch-format option
Johannes Schindelin      21084e84a: http: add support for selecting SSL 
backends at runtime
Johannes Schindelin      3249c1251: rebase: consolidate clean-up code 
before leaving reset_head()
Johannes Schindelin      501afcb8b: mingw: use domain information for 
default email
Johannes Schindelin      71f82465b: rebase -i: introduce the 'break' command
Johannes Schindelin      b67d40adb: http: when using Secure Channel, 
ignore sslCAInfo by default
Johannes Schindelin      bac2a1e36: built-in rebase: reinstate `checkout 
-q` behavior where appropriate
Johannes Schindelin      bc24382c2: builtin rebase: prepare for builtin 
rebase -i
Jonathan Nieder      302997027: gc: do not return error for prior errors 
in daemonized mode
Jonathan Nieder      fec2ed218: gc: exit with status 128 on failure
Jonathan Tan      1d1243fe6: upload-pack: make want_obj not global
Josh Steadmon      1127a98cc: fuzz: add fuzz testing for packfile indices.
Matthew DeVore      7c0fe330d: rev-list: handle missing tree objects 
properly
Matthew DeVore      bc5975d24: list-objects-filter: implement filter tree:0
Matthew DeVore      cc0b05a4c: list-objects-filter-options: do not 
over-strbuf_init
Matthew DeVore      f447a499d: list-objects: store common func args in 
struct
Michał Górny      4de9394dc: gpg-interface.c: obtain primary key 
fingerprint as well
Nguyễn Thái Ngọc Duy      2179045fd: Clean up pthread_create() error 
handling
Nguyễn Thái Ngọc Duy      252d079cb: read-cache.c: optimize reading 
index format v4
Nguyễn Thái Ngọc Duy      26c7d0678: help -a: improve and make --verbose 
default
Nguyễn Thái Ngọc Duy      2abf35038: revision.c: remove implicit 
dependency on the_index
Nguyễn Thái Ngọc Duy      3a3b9d8cd: refs: new ref types to make 
per-worktree refs visible to all worktrees
Nguyễn Thái Ngọc Duy      58b284a2e: worktree: add per-worktree config files
Nguyễn Thái Ngọc Duy      a470beea3: blame.c: rename "repo" argument to "r"
Nguyễn Thái Ngọc Duy      ab3e1f78a: revision.c: better error reporting 
on ref from different worktrees
Nguyễn Thái Ngọc Duy      ae9af1228: status: show progress bar if 
refreshing the index takes too long
Nguyễn Thái Ngọc Duy      b29759d89: fsck: check HEAD and reflog from 
other worktrees
Nguyễn Thái Ngọc Duy      b4da37380: unpack-trees: optimize walking same 
trees with cache-tree
Nguyễn Thái Ngọc Duy      b78ea5fc3: diff.c: reduce implicit dependency 
on the_index
Nguyễn Thái Ngọc Duy      c0e40a2d6: send-pack.c: move async's #ifdef 
NO_PTHREADS back to run-command.c
Nguyễn Thái Ngọc Duy      c46c406ae: trace.h: support nested performance 
tracing
Nguyễn Thái Ngọc Duy      c9ef0d95e: reflog expire: cover reflog from 
all worktrees
Nguyễn Thái Ngọc Duy      f1e11c651: unpack-trees: reduce malloc in 
cache-tree walk
Nguyễn Thái Ngọc Duy      fd6263fb7: grep: clean up num_threads handling
Olga Telezhnaya      f0062d3b7: ref-filter: free item->value and 
item->value->s
Phillip Wood      bcd33ec25: add read_author_script() to libgit
Pratik Karki      002ee2fe6: builtin rebase: support `keep-empty` option
Pratik Karki      0eabf4b95: builtin rebase: stop if `git am` is in progress
Pratik Karki      12026a412: builtin rebase: support `--gpg-sign` option
Pratik Karki      122420c29: builtin rebase: support --skip
Pratik Karki      1ed9c14ff: builtin rebase: support --force-rebase
Pratik Karki      3c3588c7d: builtin rebase: support 
--rebase-merges[=[no-]rebase-cousins]
Pratik Karki      55071ea24: rebase: start implementing it as a builtin
Pratik Karki      5a6149453: builtin rebase: support --quit
Pratik Karki      5e5d96197: builtin rebase: support --abort
Pratik Karki      6defce2b0: builtin rebase: support `--autostash` option
Pratik Karki      73d51ed0a: builtin rebase: support --signoff
Pratik Karki      9a48a615b: builtin rebase: try to fast forward when 
possible
Pratik Karki      9dba809a6: builtin rebase: support --root
Pratik Karki      ac7f467fe: builtin/rebase: support running "git rebase 
<upstream>"
Pratik Karki      ba1905a5f: builtin rebase: add support for custom 
merge strategies
Pratik Karki      bff014dac: builtin rebase: support the `verbose` and 
`diffstat` options
Pratik Karki      c54dacb50: builtin rebase: start a new rebase only if 
none is in progress
Pratik Karki      cda614e48: builtin rebase: show progress when 
connected to a terminal
Pratik Karki      e0333e5c6: builtin rebase: require a clean worktree
Pratik Karki      e65123a71: builtin rebase: support `git rebase 
<upstream> <switch-to>`
Pratik Karki      ead98c111: builtin rebase: support --rerere-autoupdate
Pratik Karki      f28d40d3a: builtin rebase: support --onto
Pratik Karki      f95736288: builtin rebase: support --continue
Rasmus Villemoes      a9a60b94c: git.c: handle_alias: prepend alias info 
when first argument is -h
Rasmus Villemoes      e6e76baaf: help: redirect to aliased commands for 
"git cmd --help"
René Scharfe      3aa4d81f8: mailinfo: support format=flowed
René Scharfe      8b2f8cbcb: oidset: use khash
René Scharfe      fb8952077: fsck: use strbuf_getline() to read skiplist 
file
Shulhan      5025425df: builtin/remote: quote remote name on error to 
display empty name
Stefan Beller      4a6067cda: refs.c: migrate internal ref iteration to 
pass thru repository argument
Stefan Beller      74d4731da: submodule--helper: replace 
connect-gitdir-workingtree by ensure-core-worktree
Stefan Beller      e0a862fda: submodule helper: convert relative URL to 
absolute URL if needed
Stefan Beller      ee69b2a90: submodule--helper: introduce new 
update-module-mode helper
Stephen P. Smith      73ba5d78b: roll wt_status_state into wt_status and 
populate in the collect phase
Stephen P. Smith      f3bd35fa0: wt-status.c: set the committable flag 
in the collect phase
SZEDER Gábor      e3d837989: split-index: don't compare cached data of 
entries already marked for split index
Thomas Gummerer      2373b6505: rerere: mark strings for translation
Thomas Gummerer      4af32207b: rerere: teach rerere to handle nested 
conflicts
Thomas Gummerer      c0f16f8e1: rerere: factor out handle_conflict function
Tim Schumacher      c6d75bc17: alias: add support for aliases of an alias
Torsten Bögershausen      d64324cb6: Make git_check_attr() a void function

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

[Ævar Arnfjörð Bjarmason]

On Wed, Nov 14 2018, Johannes Schindelin via GitGitGadget wrote:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> It is a good idea to error out early upon seeing, say, `-Cbad`, rather
> than starting the rebase only to have the `--am` backend complain later.
>
> Let's do this.
>
> The only options accepting parameters which we pass through to `git am`
> (which may, or may not, forward them to `git apply`) are `-C` and
> `--whitespace`. The other options we pass through do not accept
> parameters, so we do not have to validate them here.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  builtin/rebase.c          | 12 +++++++++++-
>  t/t3406-rebase-message.sh |  7 +++++++
>  2 files changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/builtin/rebase.c b/builtin/rebase.c
> index 96ffa80b71..571cf899d5 100644
> --- a/builtin/rebase.c
> +++ b/builtin/rebase.c
> @@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
>  	}
>
>  	for (i = 0; i < options.git_am_opts.argc; i++) {
> -		const char *option = options.git_am_opts.argv[i];
> +		const char *option = options.git_am_opts.argv[i], *p;
>  		if (!strcmp(option, "--committer-date-is-author-date") ||
>  		    !strcmp(option, "--ignore-date") ||
>  		    !strcmp(option, "--whitespace=fix") ||
>  		    !strcmp(option, "--whitespace=strip"))
>  			options.flags |= REBASE_FORCE;
> +		else if (skip_prefix(option, "-C", &p)) {
> +			while (*p)
> +				if (!isdigit(*(p++)))
> +					die(_("switch `C' expects a "
> +					      "numerical value"));
> +		} else if (skip_prefix(option, "--whitespace=", &p)) {
> +			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
> +			    strcmp(p, "error") && strcmp(p, "error-all"))
> +				die("Invalid whitespace option: '%s'", p);
> +		}
>  	}
>
>  	if (!(options.flags & REBASE_NO_QUIET))
> diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
> index 0392e36d23..2c79eed4fe 100755
> --- a/t/t3406-rebase-message.sh
> +++ b/t/t3406-rebase-message.sh
> @@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
>  	test_i18ngrep "invalid-ref" err
>  '
>
> +test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
> +	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
> +	test_i18ngrep "numerical value" err &&
> +	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
> +	test_i18ngrep "Invalid whitespace option" err
> +'
> +

The combination of this gitster/js/rebase-am-options and my
gitster/ab/rebase-in-c-escape-hatch breaks tests under
GIT_TEST_REBASE_USE_BUILTIN=false for obvious reasons. The C version is
now more strict.

Re: Git Test Coverage Report (v2.20.0-rc0)

[Derrick Stolee]

The test coverage reports started mid-way through this release cycle, so 
I thought it would be good to do a full review of the new uncovered code 
since the last release.

I eliminated most of the uncovered code due to the following cases:

1. Code was only moved or refactored.
2. Code was related to unusual error conditions (e.g. open_pack_index() 
fails)

The comments below are intended only to point out potential directions 
to improve test coverage. Some of it is for me to do!

Thanks,
-Stolee

On 11/18/2018 9:54 PM, Derrick Stolee wrote:
> 66ec0390e7 builtin/fsck.c 888) midx_argv[2] = "--object-dir";
> 66ec0390e7 builtin/fsck.c 889) midx_argv[3] = alt->path;
> 66ec0390e7 builtin/fsck.c 890) if (run_command(&midx_verify))
> 66ec0390e7 builtin/fsck.c 891) errors_found |= ERROR_COMMIT_GRAPH;
>

There are two things wrong here:

1. Not properly covering multi-pack-index fsck with alternates.
2. the ERROR_COMMIT_GRAPH flag when the multi-pack-index is being checked.

I'll submit a patch to fix this.

> 2fa233a554 builtin/pack-objects.c 1512) hashcpy(base_oid.hash, 
> base_sha1);
> 2fa233a554 builtin/pack-objects.c 1513) if 
> (!in_same_island(&delta->idx.oid, &base_oid))
> 2fa233a554 builtin/pack-objects.c 1514) return 0;

These lines are inside a block for the following if statement:

+       /*
+        * Otherwise, reachability bitmaps may tell us if the receiver 
has it,
+        * even if it was buried too deep in history to make it into the
+        * packing list.
+        */
+       if (thin && bitmap_has_sha1_in_uninteresting(bitmap_git, 
base_sha1)) {

Peff: is this difficult to test?

> 28b8a73080 builtin/pack-objects.c 2793) depth++;
> 108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, 
> ent, depth); 

This 'depth' variable is incremented as part of a for loop in this patch:

  static void show_object(struct object *obj, const char *name, void *data)
@@ -2686,6 +2706,19 @@ static void show_object(struct object *obj, const 
char *name, void *data)
         add_preferred_base_object(name);
         add_object_entry(&obj->oid, obj->type, name, 0);
         obj->flags |= OBJECT_ADDED;
+
+       if (use_delta_islands) {
+               const char *p;
+               unsigned depth = 0;
+               struct object_entry *ent;
+
+               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
+                       depth++;
+
+               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
+               if (ent && depth > ent->tree_depth)
+                       ent->tree_depth = depth;
+       }
  }

And that 'ent->tree_depth = depth;' line is replaced with the 
oe_set_tree_depth() call in the report.

Since depth is never incremented, we are not covering this block. Is it 
possible to test?

> builtin/repack.c
> 16d75fa48d  48) use_delta_islands = git_config_bool(var, value);
> 16d75fa48d  49) return 0;

This is a simple config option check for "repack.useDeltaIslands". The 
logic it enables is tested, so this is an OK gap, in my opinion.

 > builtin/submodule--helper.c
> ee69b2a90c 1476) out->type = sub->update_strategy.type;
> ee69b2a90c 1477) out->command = sub->update_strategy.command;

This block was introduced by this part of the patch:

+       } else if (sub->update_strategy.type != SM_UPDATE_UNSPECIFIED) {
+               trace_printf("loaded thing");
+               out->type = sub->update_strategy.type;
+               out->command = sub->update_strategy.command;

Which seems to be an important case, as the other SM_UPDATE_* types seem 
like interesting cases.

Stefan: what actions would trigger this block? Is it easy to test?

> delta-islands.c
> c8d521faf7  53) memcpy(b, old, size);

This memcpy happens when the 'old' island_bitmap is passed to 
island_bitmap_new(), but...

> c8d521faf7 187) b->refcount--;
> c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);

This block has the only non-NULL caller to island_bitmap_new().

> c8d521faf7 212) obj = ((struct tag *)obj)->tagged;
> c8d521faf7 213) if (obj) {
> c8d521faf7 214) parse_object(the_repository, &obj->oid);
> c8d521faf7 215) marks = create_or_get_island_marks(obj);
> c8d521faf7 216) island_bitmap_set(marks, island_counter);

It appears that this block would happen if we placed a tag in the delta 
island.

> c8d521faf7 397) strbuf_addch(&island_name, '-');

This block is inside the following patch:

+       if (matches[ARRAY_SIZE(matches) - 1].rm_so != -1)
+               warning(_("island regex from config has "
+                         "too many capture groups (max=%d)"),
+                       (int)ARRAY_SIZE(matches) - 2);
+
+       for (m = 1; m < ARRAY_SIZE(matches); m++) {
+               regmatch_t *match = &matches[m];
+
+               if (match->rm_so == -1)
+                       continue;
+
+               if (island_name.len)
+                       strbuf_addch(&island_name, '-');
+
+               strbuf_add(&island_name, refname + match->rm_so, 
match->rm_eo - match->rm_so);
+       }

This likely means that ARRAY_SIZE(matches) is never more than two.


> c8d521faf7 433) continue;
> c8d521faf7 436) list[dst] = list[src];

These blocks are inside the following nested loop in deduplicate_islands():

+       for (ref = 0; ref + 1 < island_count; ref++) {
+               for (src = ref + 1, dst = src; src < island_count; src++) {
+                       if (list[ref]->hash == list[src]->hash)
+                               continue;
+
+                       if (src != dst)
+                               list[dst] = list[src];
+
+                       dst++;
+               }
+               island_count = dst;
+       }

This means that our "deduplication" logic is never actually doing 
anything meaningful.

> entry.c
> b878579ae7 402) static void mark_colliding_entries(const struct 
> checkout *state,

(there is interesting logic in this method, but it is only enabled on 
case-insensitive filesystems. This run was done on a case-sensitive file 
system. Related changes happen in unpack-trees.c.)

> help.c
> 26c7d06783 help.c         500) static int get_alias(const char *var, 
> const char *value, void *data)
> 26c7d06783 help.c         502) struct string_list *list = data;
> 26c7d06783 help.c         504) if (skip_prefix(var, "alias.", &var))
> 26c7d06783 help.c         505) string_list_append(list, var)->util = 
> xstrdup(value);
> 26c7d06783 help.c         507) return 0;
> 26c7d06783 help.c         530) printf("\n%s\n", _("Command aliases"));
> 26c7d06783 help.c         531) ALLOC_ARRAY(aliases, alias_list.nr + 1);
> 26c7d06783 help.c         532) for (i = 0; i < alias_list.nr; i++) {
> 26c7d06783 help.c         533) aliases[i].name = 
> alias_list.items[i].string;
> 26c7d06783 help.c         534) aliases[i].help = 
> alias_list.items[i].util;
> 26c7d06783 help.c         535) aliases[i].category = 1;
> 26c7d06783 help.c         537) aliases[alias_list.nr].name = NULL;
> 26c7d06783 help.c         538) print_command_list(aliases, 1, longest);
> 26c7d06783 help.c         539) free(aliases);

This logic introduces alias help in 'git help -a'. This seems like a 
simple thing for adding a test to ensure that this works now and in the 
future.

>
> http.c
The code in here seems to be logic for Windows-specific SSL backends, so 
is not covered by this report.

> preload-index.c
> ae9af12287  63) struct progress_data *pd = p->progress;
> ae9af12287  65) pthread_mutex_lock(&pd->mutex);
> ae9af12287  66) pd->n += last_nr - nr;
> ae9af12287  67) display_progress(pd->progress, pd->n);
> ae9af12287  68) pthread_mutex_unlock(&pd->mutex);
> ae9af12287  69) last_nr = nr;
> ae9af12287  83) struct progress_data *pd = p->progress;
> ae9af12287  85) pthread_mutex_lock(&pd->mutex);
> ae9af12287  86) display_progress(pd->progress, pd->n + last_nr);
> ae9af12287  87) pthread_mutex_unlock(&pd->mutex);
> ae9af12287 118) pd.progress = start_delayed_progress(_("Refreshing 
> index"), index->cache_nr);
> ae9af12287 119) pthread_mutex_init(&pd.mutex, NULL);
> ae9af12287 132) p->progress = &pd;

There's a lot of stuff going on with showing progress on index writes. 
While the commit message states the progress doesn't show up for 3 
seconds, perhaps that can be tweaked to be in the millisecond range for 
a test?

> read-cache.c

(There's a lot of progress stuff here, too.)

There are a lot of lines introduced by the IEOT extension in these commits:

 > Ben Peart      3255089ad: ieot: add Index Entry Offset Table (IEOT) 
extension
 > Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) extension
 > Ben Peart      77ff1127a: read-cache: load cache entries on worker 
threads
 > Ben Peart      abb4bb838: read-cache: load cache extensions on a 
worker thread
 > Ben Peart      c780b9cfe: config: add new index.threads config setting
 > Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing 
read_cache_preload()
 > Ben Peart      fa655d841: checkout: optimize "git checkout -b 
<new_branch>"

> revision.c
> b45424181e 2951) c->object.flags |= UNINTERESTING;
> b45424181e 2957) mark_parents_uninteresting(c);

These blocks are currently unreachable because we do not use the new 
topo-order logic when there are UNINTERESTING commits. (This will be 
replaced after we have generation numbers v2.) I could force using this 
logic in a `git log --topo-order A..B` query when GIT_TEST_COMMIT_GRAPH 
is enabled.

Re: Git Test Coverage Report (v2.20.0-rc0)

[Jeff King]

On Mon, Nov 19, 2018 at 10:40:53AM -0500, Derrick Stolee wrote:

> > 2fa233a554 builtin/pack-objects.c 1512) hashcpy(base_oid.hash,
> > base_sha1);
> > 2fa233a554 builtin/pack-objects.c 1513) if
> > (!in_same_island(&delta->idx.oid, &base_oid))
> > 2fa233a554 builtin/pack-objects.c 1514) return 0;
> 
> These lines are inside a block for the following if statement:
> 
> +       /*
> +        * Otherwise, reachability bitmaps may tell us if the receiver has
> it,
> +        * even if it was buried too deep in history to make it into the
> +        * packing list.
> +        */
> +       if (thin && bitmap_has_sha1_in_uninteresting(bitmap_git, base_sha1))
> {
> 
> Peff: is this difficult to test?

A bit.

The two features (thin-packs and delta-islands) would not generally be
used together (one is for serving fetches, the other is for optimizing
on-disk packs to serve fetches). Something like:

 echo HEAD^ | git pack-objects --revs --thin --delta-islands

would probably exercise it (assuming there's a delta in HEAD^ against
something in HEAD), but you'd need to construct a very specific scenario
if you wanted to do any kind of checks no the output.

> > 28b8a73080 builtin/pack-objects.c 2793) depth++;
> > 108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, ent,
> > depth);
> 
> This 'depth' variable is incremented as part of a for loop in this patch:
> 
>  static void show_object(struct object *obj, const char *name, void *data)
> @@ -2686,6 +2706,19 @@ static void show_object(struct object *obj, const
> char *name, void *data)
>         add_preferred_base_object(name);
>         add_object_entry(&obj->oid, obj->type, name, 0);
>         obj->flags |= OBJECT_ADDED;
> +
> +       if (use_delta_islands) {
> +               const char *p;
> +               unsigned depth = 0;
> +               struct object_entry *ent;
> +
> +               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
> +                       depth++;
> +
> +               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
> +               if (ent && depth > ent->tree_depth)
> +                       ent->tree_depth = depth;
> +       }
>  }
> 
> And that 'ent->tree_depth = depth;' line is replaced with the
> oe_set_tree_depth() call in the report.
> 
> Since depth is never incremented, we are not covering this block. Is it
> possible to test?

Looks like t5320 only has single-level trees. We probably just need to
add a deeper tree. A more interesting case is when an object really is
found at multiple depths, but constructing a case that cared about that
would be quite complicated.

That said, there is much bigger problem with this code, which is that
108f530385 (pack-objects: move tree_depth into 'struct packing_data',
2018-08-16) is totally broken. It works on the trivial repository in the
test, but try this (especially under valgrind or ASan) on a real
repository:

  git repack -adi

which will crash immediately.  It doesn't correctly maintain the
invariant that if tree_depth is not NULL, it is the same size as
the main object array.

I'll see if I can come up with a fix.

-Peff

Re: Git Test Coverage Report (v2.20.0-rc0)

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 19 2018, Derrick Stolee wrote:

> Here is a test coverage report for the uncovered lines introduced in
> v2.20.0-rc0 compared to v2.19.1.

Thanks a lot for this.

> [...]
> builtin/rebase.c
> 62c23938fa 55) return env;
> [...]
> Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
> where rebase.useBuiltin is off

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

That would make it take at least 12x as long if you did them one at a
time. Probably just 2-3x as long in practice since most can be combined
in some way, and somewhat computationally infeasible if you're going to
try all possible combinations.

> [...]
> fsck.c
> fb8952077d 214) die_errno("Could not read '%s'", path);
>
> René Scharfe fb8952077: fsck: use strbuf_getline() to read
> skiplist file

The fault of a patch I submitted. Couldn't find a sane & portable way to
emulate cases where ferror() would trigger.

Re: Git Test Coverage Report (v2.20.0-rc0)

[Jeff King]

On Mon, Nov 19, 2018 at 11:21:38AM -0500, Jeff King wrote:

> > +       if (use_delta_islands) {
> > +               const char *p;
> > +               unsigned depth = 0;
> > +               struct object_entry *ent;
> > +
> > +               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
> > +                       depth++;
> > +
> > +               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
> > +               if (ent && depth > ent->tree_depth)
> > +                       ent->tree_depth = depth;
> > +       }
> >  }
> > 
> > And that 'ent->tree_depth = depth;' line is replaced with the
> > oe_set_tree_depth() call in the report.
> > 
> > Since depth is never incremented, we are not covering this block. Is it
> > possible to test?
> 
> Looks like t5320 only has single-level trees. We probably just need to
> add a deeper tree. A more interesting case is when an object really is
> found at multiple depths, but constructing a case that cared about that
> would be quite complicated.
> 
> That said, there is much bigger problem with this code, which is that
> 108f530385 (pack-objects: move tree_depth into 'struct packing_data',
> 2018-08-16) is totally broken. It works on the trivial repository in the
> test, but try this (especially under valgrind or ASan) on a real
> repository:
> 
>   git repack -adi
> 
> which will crash immediately.  It doesn't correctly maintain the
> invariant that if tree_depth is not NULL, it is the same size as
> the main object array.
> 
> I'll see if I can come up with a fix.

Just a quick update to prevent anyone else looking at it: I have a fix
for this (and another related issue with that commit).

There's an edge case in that depth computation that I think is
unhandled, as well. I _think_ it doesn't trigger very often, but I'm
running some experiments to verify it. That's S-L-O-W, so I probably
won't have results until tomorrow.

-Peff

Re: [PATCH] tests: add a special setup where rebase.useBuiltin is off (Re: Git Test Coverage Report (v2.20.0-rc0))

[Jonathan Nieder]

Ævar Arnfjörð Bjarmason wrote:
> On Mon, Nov 19 2018, Derrick Stolee wrote:

>> [...]
>> builtin/rebase.c
>> 62c23938fa 55) return env;
>> [...]
>> Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
>> where rebase.useBuiltin is off
>
> This one would be covered with
> GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
> the rest of the coverage would look different when passed through the various GIT_TEST_* options.

I wonder if we can do better for this kind of thing.

When I do routine development, I am not running tests with any
non-default flags.  So why should tests run with non-default flags
count toward coverage?  Is there a way to make the default test
settings dip their feet into some non-default configurations, without
running the full battery of tests and slowing tests down accordingly?
E.g. is there some kind of smoke test that rebase with
useBuiltin=false works at all that could run, even if I am not running
the full battery of rebase tests?

That's a bit of a non sequitor for this example, which is actual code
to handle GIT_TEST_REBASE_USE_BUILTIN, though.  For it, I wonder why
we need rebase.c to understand the envvar --- couldn't test-lib.sh
take care of setting rebase.useBuiltin to false when it's set?

Thanks,
Jonathan

Re: Git Test Coverage Report (v2.20.0-rc0)

[Ben Peart]

On 11/19/2018 10:40 AM, Derrick Stolee wrote:
> The test coverage reports started mid-way through this release cycle, so 
> I thought it would be good to do a full review of the new uncovered code 
> since the last release.
> 
> I eliminated most of the uncovered code due to the following cases:
> 
> 1. Code was only moved or refactored.
> 2. Code was related to unusual error conditions (e.g. open_pack_index() 
> fails)
> 
> The comments below are intended only to point out potential directions 
> to improve test coverage. Some of it is for me to do!
> 
> Thanks,
> -Stolee
> 
> On 11/18/2018 9:54 PM, Derrick Stolee wrote:
> 
> There are a lot of lines introduced by the IEOT extension in these commits:
> 
>  > Ben Peart      3255089ad: ieot: add Index Entry Offset Table (IEOT) 
> extension
>  > Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) extension
>  > Ben Peart      77ff1127a: read-cache: load cache entries on worker 
> threads
>  > Ben Peart      abb4bb838: read-cache: load cache extensions on a 
> worker thread
>  > Ben Peart      c780b9cfe: config: add new index.threads config setting
>  > Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing 
> read_cache_preload()
>  > Ben Peart      fa655d841: checkout: optimize "git checkout -b 
> <new_branch>"
> 

These should be hit if you run the test suite with 
GIT_TEST_INDEX_THREADS=2.  Without that, the indexes for the various 
tests are too small to trigger multi-threaded index reads/writes.

 From t/README:

GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading
of the index for the whole test suite by bypassing the default number of
cache entries and thread minimums. Setting this to 1 will make the
index loading single threaded.

Re: Git Test Coverage Report (v2.20.0-rc0)

[Derrick Stolee]

On 11/19/2018 1:33 PM, Ævar Arnfjörð Bjarmason wrote:
> On Mon, Nov 19 2018, Derrick Stolee wrote:
>
>> Here is a test coverage report for the uncovered lines introduced in
>> v2.20.0-rc0 compared to v2.19.1.
> Thanks a lot for this.
>
>> [...]
>> builtin/rebase.c
>> 62c23938fa 55) return env;
>> [...]
>> Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
>> where rebase.useBuiltin is off
> This one would be covered with
> GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
> the rest of the coverage would look different when passed through the various GIT_TEST_* options.

The coverage report has been using the following:

export GIT_TEST_MULTI_PACK_INDEX=1
export GIT_TEST_COMMIT_GRAPH=1
export GIT_TEST_INDEX_VERION=4
export GIT_TEST_SPLIT_INDEX=yes
export GIT_TEST_OE_SIZE=10
export GIT_TEST_OE_DELTA_SIZE=5

I need to add GIT_TEST_INDEX_THREADS=2 and GIT_TEST_REBASE_USE_BUILTIN=false

Thanks!
-Stolee

Re: Git Test Coverage Report (v2.20.0-rc0)

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 19 2018, Derrick Stolee wrote:

> On 11/19/2018 1:33 PM, Ævar Arnfjörð Bjarmason wrote:
>> On Mon, Nov 19 2018, Derrick Stolee wrote:
>>
>>> Here is a test coverage report for the uncovered lines introduced in
>>> v2.20.0-rc0 compared to v2.19.1.
>> Thanks a lot for this.
>>
>>> [...]
>>> builtin/rebase.c
>>> 62c23938fa 55) return env;
>>> [...]
>>> Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
>>> where rebase.useBuiltin is off
>> This one would be covered with
>> GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
>> the rest of the coverage would look different when passed through the various GIT_TEST_* options.
>
> The coverage report has been using the following:
>
> export GIT_TEST_MULTI_PACK_INDEX=1
> export GIT_TEST_COMMIT_GRAPH=1
> export GIT_TEST_INDEX_VERION=4
> export GIT_TEST_SPLIT_INDEX=yes
> export GIT_TEST_OE_SIZE=10
> export GIT_TEST_OE_DELTA_SIZE=5
>
> I need to add GIT_TEST_INDEX_THREADS=2 and GIT_TEST_REBASE_USE_BUILTIN=false

...although note you'll need to also test without
GIT_TEST_REBASE_USE_BUILTIN=false, otherwise a lot of the new C code
won't have coverage.

Re: Git Test Coverage Report (v2.20.0-rc0)

[Derrick Stolee]

On 11/19/2018 2:39 PM, Ævar Arnfjörð Bjarmason wrote:
> On Mon, Nov 19 2018, Derrick Stolee wrote:
>
>> The coverage report has been using the following:
>>
>> export GIT_TEST_MULTI_PACK_INDEX=1
>> export GIT_TEST_COMMIT_GRAPH=1
>> export GIT_TEST_INDEX_VERION=4
>> export GIT_TEST_SPLIT_INDEX=yes
>> export GIT_TEST_OE_SIZE=10
>> export GIT_TEST_OE_DELTA_SIZE=5
>>
>> I need to add GIT_TEST_INDEX_THREADS=2 and GIT_TEST_REBASE_USE_BUILTIN=false
> ...although note you'll need to also test without
> GIT_TEST_REBASE_USE_BUILTIN=false, otherwise a lot of the new C code
> won't have coverage.

Sorry for lack of clarity: I first run 'make coverage-test' with no 
GIT_TEST_* variables, then run the test suite again with the optional 
variables.

Thanks,
-Stolee

Re: [PATCH] tests: add a special setup where rebase.useBuiltin is off (Re: Git Test Coverage Report (v2.20.0-rc0))

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 19 2018, Jonathan Nieder wrote:

> Ævar Arnfjörð Bjarmason wrote:
>> On Mon, Nov 19 2018, Derrick Stolee wrote:
>
>>> [...]
>>> builtin/rebase.c
>>> 62c23938fa 55) return env;
>>> [...]
>>> Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
>>> where rebase.useBuiltin is off
>>
>> This one would be covered with
>> GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
>> the rest of the coverage would look different when passed through the various GIT_TEST_* options.
>
> I wonder if we can do better for this kind of thing.
>
> When I do routine development, I am not running tests with any
> non-default flags.  So why should tests run with non-default flags
> count toward coverage?  Is there a way to make the default test
> settings dip their feet into some non-default configurations, without
> running the full battery of tests and slowing tests down accordingly?
> E.g. is there some kind of smoke test that rebase with
> useBuiltin=false works at all that could run, even if I am not running
> the full battery of rebase tests?

Yeah, definitely. Just pointing out that it would smoke out coverage we
don't have at all v.s. cases where we just don't have coverage with the
default tests without any special modes.

Derrick: I think it would be useful to produce some delta report showing
covered lines without any GIT_TEST_* variables v.s. when they're set. As
Jonathan points out those should ideally be tested with the normal test
suite, leaving GIT_TEST_* just for stress testing to find new bugs.

> That's a bit of a non sequitor for this example, which is actual code
> to handle GIT_TEST_REBASE_USE_BUILTIN, though.  For it, I wonder why
> we need rebase.c to understand the envvar --- couldn't test-lib.sh
> take care of setting rebase.useBuiltin to false when it's set?

I guess the test-lib.sh could pass down things like
"GIT_CONFIG_PARAMETERS='x.y=z'". Maybe that's a better way to do it.

Re: Git Test Coverage Report (v2.20.0-rc0)

[Derrick Stolee]

On 11/19/2018 2:00 PM, Ben Peart wrote:
>
>
> On 11/19/2018 10:40 AM, Derrick Stolee wrote:
>>
>> There are a lot of lines introduced by the IEOT extension in these 
>> commits:
>>
>>  > Ben Peart      3255089ad: ieot: add Index Entry Offset Table 
>> (IEOT) extension
>>  > Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) 
>> extension
>>  > Ben Peart      77ff1127a: read-cache: load cache entries on worker 
>> threads
>>  > Ben Peart      abb4bb838: read-cache: load cache extensions on a 
>> worker thread
>>  > Ben Peart      c780b9cfe: config: add new index.threads config 
>> setting
>>  > Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing 
>> read_cache_preload()
>>  > Ben Peart      fa655d841: checkout: optimize "git checkout -b 
>> <new_branch>"
>>
>
> These should be hit if you run the test suite with 
> GIT_TEST_INDEX_THREADS=2.  Without that, the indexes for the various 
> tests are too small to trigger multi-threaded index reads/writes.
>
> From t/README:
>
> GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading
> of the index for the whole test suite by bypassing the default number of
> cache entries and thread minimums. Setting this to 1 will make the
> index loading single threaded.

I updated my build to add GIT_TEST_INDEX_THREADS=2 and I still see lots 
of uncovered stuff, including that load_cache_entries_threaded() is 
never run.

I added the following diff to my repo and ran the test suite manually 
with GIT_TEST_INDEX_THREADS=2 and it didn't fail:

diff --git a/read-cache.c b/read-cache.c
index 4ca81286c0..36502586a2 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -2057,6 +2057,9 @@ static unsigned long 
load_cache_entries_threaded(struct index_state *istate, con
         struct load_cache_entries_thread_data *data;
         unsigned long consumed = 0;

+       fprintf(stderr, "load_cache_entries_threaded\n");
+       exit(1);
+
         /* a little sanity checking */
         if (istate->name_hash_initialized)
                 BUG("the name hash isn't thread safe");

Am I missing something? Is there another variable I should add?

When I look for where the GIT_TEST_INDEX_THREADS variable is checked, I 
see that the calls to git_config_get_index_threads() are followed by a 
check for NO_PTHREADS (setting the number of threads to 1 again). Is it 
possible that my compiler environment is not allowing me to even compile 
with threads?

Thanks,
-Stolee

Re: Git Test Coverage Report (v2.20.0-rc0)

[Derrick Stolee]

On 11/19/2018 1:33 PM, Ævar Arnfjörð Bjarmason wrote:
> On Mon, Nov 19 2018, Derrick Stolee wrote:
>
>> [...]
>> builtin/rebase.c
>> 62c23938fa 55) return env;
>> [...]
>> Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
>> where rebase.useBuiltin is off
> This one would be covered with
> GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
> the rest of the coverage would look different when passed through the various GIT_TEST_* options.
>

Thanks for pointing out this GIT_TEST_* variable to me. I had been 
running builds with some of them enabled, but didn't know about this one.

Unfortunately, t3406-rebase-message.sh fails with 
GIT_TEST_REBASE_USE_BUILTIN=false and it bisects to 4520c2337: Merge 
branch 'ab/rebase-in-c-escape-hatch'.

The issue is that the commit 04519d72 "rebase: validate -C<n> and 
--whitespace=<mode> parameters early" introduced the following test that 
cares about error messages:

+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+       test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+       test_i18ngrep "numerical value" err &&
+       test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+       test_i18ngrep "Invalid whitespace option" err
+'

The merge commit then was the first place where this test could run with 
that variable.

What's the correct fix here? Force the builtin rebase in this test? 
Unify the error message in the non-builtin case?

Thanks,
-Stolee

Re: Git Test Coverage Report (v2.20.0-rc0)

[Ævar Arnfjörð Bjarmason]

On Mon, Nov 19 2018, Derrick Stolee wrote:

> On 11/19/2018 1:33 PM, Ævar Arnfjörð Bjarmason wrote:
>> On Mon, Nov 19 2018, Derrick Stolee wrote:
>>
>>> [...]
>>> builtin/rebase.c
>>> 62c23938fa 55) return env;
>>> [...]
>>> Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
>>> where rebase.useBuiltin is off
>> This one would be covered with
>> GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
>> the rest of the coverage would look different when passed through the various GIT_TEST_* options.
>>
>
> Thanks for pointing out this GIT_TEST_* variable to me. I had been
> running builds with some of them enabled, but didn't know about this
> one.
>
> Unfortunately, t3406-rebase-message.sh fails with
> GIT_TEST_REBASE_USE_BUILTIN=false and it bisects to 4520c2337: Merge
> branch 'ab/rebase-in-c-escape-hatch'.
>
> The issue is that the commit 04519d72 "rebase: validate -C<n> and
> --whitespace=<mode> parameters early" introduced the following test
> that cares about error messages:
>
> +test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
> + test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
> + test_i18ngrep "numerical value" err &&
> + test_must_fail git rebase --whitespace=bad HEAD 2>err &&
> + test_i18ngrep "Invalid whitespace option" err
> +'
>
> The merge commit then was the first place where this test could run
> with that variable.

Yup. Sorry should have mentioned that, it's broken in master. Reported
it earlier today:
https://public-inbox.org/git/874lcd1bub.fsf@evledraar.gmail.com/

> What's the correct fix here? Force the builtin rebase in this test?
> Unify the error message in the non-builtin case?

Probably to just force the builtin, unless Johannes wants to also fix
the bug for the shellscript version. I don't know if for 2.20 we're
trying to maintain 100% compatibility.

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3043 bytes --]

Hi Ævar,

On Mon, 19 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> 
> On Wed, Nov 14 2018, Johannes Schindelin via GitGitGadget wrote:
> 
> > From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >
> > It is a good idea to error out early upon seeing, say, `-Cbad`, rather
> > than starting the rebase only to have the `--am` backend complain later.
> >
> > Let's do this.
> >
> > The only options accepting parameters which we pass through to `git am`
> > (which may, or may not, forward them to `git apply`) are `-C` and
> > `--whitespace`. The other options we pass through do not accept
> > parameters, so we do not have to validate them here.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  builtin/rebase.c          | 12 +++++++++++-
> >  t/t3406-rebase-message.sh |  7 +++++++
> >  2 files changed, 18 insertions(+), 1 deletion(-)
> >
> > diff --git a/builtin/rebase.c b/builtin/rebase.c
> > index 96ffa80b71..571cf899d5 100644
> > --- a/builtin/rebase.c
> > +++ b/builtin/rebase.c
> > @@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
> >  	}
> >
> >  	for (i = 0; i < options.git_am_opts.argc; i++) {
> > -		const char *option = options.git_am_opts.argv[i];
> > +		const char *option = options.git_am_opts.argv[i], *p;
> >  		if (!strcmp(option, "--committer-date-is-author-date") ||
> >  		    !strcmp(option, "--ignore-date") ||
> >  		    !strcmp(option, "--whitespace=fix") ||
> >  		    !strcmp(option, "--whitespace=strip"))
> >  			options.flags |= REBASE_FORCE;
> > +		else if (skip_prefix(option, "-C", &p)) {
> > +			while (*p)
> > +				if (!isdigit(*(p++)))
> > +					die(_("switch `C' expects a "
> > +					      "numerical value"));
> > +		} else if (skip_prefix(option, "--whitespace=", &p)) {
> > +			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
> > +			    strcmp(p, "error") && strcmp(p, "error-all"))
> > +				die("Invalid whitespace option: '%s'", p);
> > +		}
> >  	}
> >
> >  	if (!(options.flags & REBASE_NO_QUIET))
> > diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
> > index 0392e36d23..2c79eed4fe 100755
> > --- a/t/t3406-rebase-message.sh
> > +++ b/t/t3406-rebase-message.sh
> > @@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
> >  	test_i18ngrep "invalid-ref" err
> >  '
> >
> > +test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
> > +	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
> > +	test_i18ngrep "numerical value" err &&
> > +	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
> > +	test_i18ngrep "Invalid whitespace option" err
> > +'
> > +
> 
> The combination of this gitster/js/rebase-am-options and my
> gitster/ab/rebase-in-c-escape-hatch breaks tests under
> GIT_TEST_REBASE_USE_BUILTIN=false for obvious reasons. The C version is
> now more strict.

Maybe you can concoct a prereq for this test? Something like

test_lazy_prereq BUILTIN_REBASE '
	test true = "${GIT_TEST_REBASE_USE_BUILTIN:-true}"
'

Ciao,
Dscho

Re: Git Test Coverage Report (v2.20.0-rc0)

[Jeff King]

On Mon, Nov 19, 2018 at 10:40:53AM -0500, Derrick Stolee wrote:

> > 28b8a73080 builtin/pack-objects.c 2793) depth++;
> > 108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, ent,
> > depth);
> 
> This 'depth' variable is incremented as part of a for loop in this patch:
> 
>  static void show_object(struct object *obj, const char *name, void *data)
> @@ -2686,6 +2706,19 @@ static void show_object(struct object *obj, const
> char *name, void *data)
>         add_preferred_base_object(name);
>         add_object_entry(&obj->oid, obj->type, name, 0);
>         obj->flags |= OBJECT_ADDED;
> +
> +       if (use_delta_islands) {
> +               const char *p;
> +               unsigned depth = 0;
> +               struct object_entry *ent;
> +
> +               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
> +                       depth++;
> +
> +               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
> +               if (ent && depth > ent->tree_depth)
> +                       ent->tree_depth = depth;
> +       }
>  }
> 
> And that 'ent->tree_depth = depth;' line is replaced with the
> oe_set_tree_depth() call in the report.
> 
> Since depth is never incremented, we are not covering this block. Is it
> possible to test?

This should be covered by the fix in:

  https://public-inbox.org/git/20181120095053.GC22742@sigill.intra.peff.net/

because now entries at the top-level are depth "1". The "depth++" line
is still never executed in our test suite. I'm not sure how much that
matters.

> > delta-islands.c
> > c8d521faf7  53) memcpy(b, old, size);
> 
> This memcpy happens when the 'old' island_bitmap is passed to
> island_bitmap_new(), but...
> 
> > c8d521faf7 187) b->refcount--;
> > c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);
> 
> This block has the only non-NULL caller to island_bitmap_new().

This is another case where it triggers a lot for a reasonably-sized
repo, but it's hard to construct a small case. This code implements a
copy-on-write of the bitmap, which means the same objects have to be
accessible from two different paths through the reachability graph, each
with different island marks. And then a test would I guess make sure
that the correct subsets of objects never become deltas, which gets
complicated.

And I think that's a pattern with the delta-island code. What we really
care about most is that if we throw a real fork-network repository at
it, it produces faster clones with fewer un-reusable deltas. So I think
a much more interesting approach here would be perf tests. But:

  - we'd want to count those as coverage, and that likely makes your
    coverage tests prohibitively expensive

  - it requires a specialized repo to demonstrate, which most people
    aren't going to have handy

> > c8d521faf7 212) obj = ((struct tag *)obj)->tagged;
> > c8d521faf7 213) if (obj) {
> > c8d521faf7 214) parse_object(the_repository, &obj->oid);
> > c8d521faf7 215) marks = create_or_get_island_marks(obj);
> > c8d521faf7 216) island_bitmap_set(marks, island_counter);
> 
> It appears that this block would happen if we placed a tag in the delta
> island.

Yep. Again, exercised by real repositories.

I'm not sure how far we want to go in the blind pursuit of coverage.
Certainly we could add a tag to the repo in t5320, and this code would
get executed. But verifying that it's doing the right thing is much
harder (and is more easily done with a perf test).

> > c8d521faf7 397) strbuf_addch(&island_name, '-');
> 
> This block is inside the following patch:
> [...]

Yeah, this triggers if your regex has more than one capture group (and
likewise, we almost certainly don't run the "you have too many groups"
warning).

> > c8d521faf7 433) continue;
> > c8d521faf7 436) list[dst] = list[src];
> 
> These blocks are inside the following nested loop in deduplicate_islands():
> 
> +       for (ref = 0; ref + 1 < island_count; ref++) {
> +               for (src = ref + 1, dst = src; src < island_count; src++) {
> +                       if (list[ref]->hash == list[src]->hash)
> +                               continue;
> +
> +                       if (src != dst)
> +                               list[dst] = list[src];
> +
> +                       dst++;
> +               }
> +               island_count = dst;
> +       }
> 
> This means that our "deduplication" logic is never actually doing anything
> meaningful.

Sorry, I don't even remember what this code is trying to do. The island
code is 5+ years old, and just recently ported to upstream Git by
Christian. And that's perhaps part of my laziness in the above tests; it
would be a significant effort to re-figure out all these corner cases.
It's a big part of why I hadn't been sending the patches upstream
myself.

-Peff

[PATCH] rebase: mark a test as failing with rebase.useBuiltin=false

[Ævar Arnfjörð Bjarmason]

Mark a test added in 04519d7201 ("rebase: validate -C<n> and
--whitespace=<mode> parameters early", 2018-11-14) as only succeeding
with the builtin version of rebase. It would be nice if the
shellscript version had the same fix, but it's on its way out, and the
author is not interested in fixing it[1].

This makes the entire test suite pass again with the
GIT_TEST_REBASE_USE_BUILTIN=false mode added in my 62c23938fa ("tests:
add a special setup where rebase.useBuiltin is off", 2018-11-14).

1. https://public-inbox.org/git/nycvar.QRO.7.76.6.1811201157170.41@tvgsbejvaqbjf.bet/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---

On Tue, Nov 20 2018, Johannes Schindelin wrote:

> [...]
> Maybe you can concoct a prereq for this test? Something like
>
> test_lazy_prereq BUILTIN_REBASE '
>       test true = "${GIT_TEST_REBASE_USE_BUILTIN:-true}"
> '

It's better to just mark the test as needing the prereq turned
off. E.g. this is what we do for the split index tests & now for the
gettext tests. That way we always run the test, but just indicate that
it relies on GIT_TEST_REBASE_USE_BUILTIN being unset.

 t/t3406-rebase-message.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
index 38bd876cab..77e5bbb3d5 100755
--- a/t/t3406-rebase-message.sh
+++ b/t/t3406-rebase-message.sh
@@ -84,7 +84,8 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
 	test_i18ngrep "invalid-ref" err
 '
 
-test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+test_expect_success 'builtin rebase: error out early upon -C<n> or --whitespace=<bad>' '
+	sane_unset GIT_TEST_REBASE_USE_BUILTIN &&
 	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
 	test_i18ngrep "numerical value" err &&
 	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
-- 
2.20.0.rc0.387.gc7a69e6b6c

Re: Git Test Coverage Report (v2.20.0-rc0)

[Derrick Stolee]

On 11/20/2018 6:34 AM, Jeff King wrote:
> On Mon, Nov 19, 2018 at 10:40:53AM -0500, Derrick Stolee wrote:
>
>
> Since depth is never incremented, we are not covering this block. Is it
> possible to test?
> This should be covered by the fix in:
>
>    https://public-inbox.org/git/20181120095053.GC22742@sigill.intra.peff.net/
>
> because now entries at the top-level are depth "1". The "depth++" line
> is still never executed in our test suite. I'm not sure how much that
> matters.

Thanks! I'll go take a look at your patch.

>>> delta-islands.c
>>> c8d521faf7  53) memcpy(b, old, size);
>> This memcpy happens when the 'old' island_bitmap is passed to
>> island_bitmap_new(), but...
>>
>>> c8d521faf7 187) b->refcount--;
>>> c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);
>> This block has the only non-NULL caller to island_bitmap_new().
> This is another case where it triggers a lot for a reasonably-sized
> repo, but it's hard to construct a small case. This code implements a
> copy-on-write of the bitmap, which means the same objects have to be
> accessible from two different paths through the reachability graph, each
> with different island marks. And then a test would I guess make sure
> that the correct subsets of objects never become deltas, which gets
> complicated.
>
> And I think that's a pattern with the delta-island code. What we really
> care about most is that if we throw a real fork-network repository at
> it, it produces faster clones with fewer un-reusable deltas. So I think
> a much more interesting approach here would be perf tests. But:
>
>    - we'd want to count those as coverage, and that likely makes your
>      coverage tests prohibitively expensive
>
>    - it requires a specialized repo to demonstrate, which most people
>      aren't going to have handy

Do you have regularly-running tests that check this in your 
infrastructure? As long as someone would notice if this code starts 
failing, that would be enough.

>>> c8d521faf7 212) obj = ((struct tag *)obj)->tagged;
>>> c8d521faf7 213) if (obj) {
>>> c8d521faf7 214) parse_object(the_repository, &obj->oid);
>>> c8d521faf7 215) marks = create_or_get_island_marks(obj);
>>> c8d521faf7 216) island_bitmap_set(marks, island_counter);
>> It appears that this block would happen if we placed a tag in the delta
>> island.
> Yep. Again, exercised by real repositories.
>
> I'm not sure how far we want to go in the blind pursuit of coverage.
> Certainly we could add a tag to the repo in t5320, and this code would
> get executed. But verifying that it's doing the right thing is much
> harder (and is more easily done with a perf test).

Blind coverage goals are definitely not worth the effort. My goal here 
was to re-check all of the new code (since last release) that is not 
covered, because it's easier to hide a bug there.

>
>>> c8d521faf7 397) strbuf_addch(&island_name, '-');
>> This block is inside the following patch:
>> [...]
> Yeah, this triggers if your regex has more than one capture group (and
> likewise, we almost certainly don't run the "you have too many groups"
> warning).

Did you know that regexes are notoriously under-tested [1]? When looking 
at this code, I didn't even know regexes were involved (but I didn't 
look enough at the context).

[1] https://dl.acm.org/citation.cfm?id=3236072

>
>>> c8d521faf7 433) continue;
>>> c8d521faf7 436) list[dst] = list[src];
>> These blocks are inside the following nested loop in deduplicate_islands():
>>
>> +       for (ref = 0; ref + 1 < island_count; ref++) {
>> +               for (src = ref + 1, dst = src; src < island_count; src++) {
>> +                       if (list[ref]->hash == list[src]->hash)
>> +                               continue;
>> +
>> +                       if (src != dst)
>> +                               list[dst] = list[src];
>> +
>> +                       dst++;
>> +               }
>> +               island_count = dst;
>> +       }
>>
>> This means that our "deduplication" logic is never actually doing anything
>> meaningful.
> Sorry, I don't even remember what this code is trying to do. The island
> code is 5+ years old, and just recently ported to upstream Git by
> Christian. And that's perhaps part of my laziness in the above tests; it
> would be a significant effort to re-figure out all these corner cases.
> It's a big part of why I hadn't been sending the patches upstream
> myself.

Sure. Hopefully pointing out these blocks gives us more motivation to 
manually inspect them and avoid silly bugs.

Thanks,
-Stolee

Re: Git Test Coverage Report (v2.20.0-rc0)

[Jeff King]

On Tue, Nov 20, 2018 at 07:17:46AM -0500, Derrick Stolee wrote:

> > And I think that's a pattern with the delta-island code. What we really
> > care about most is that if we throw a real fork-network repository at
> > it, it produces faster clones with fewer un-reusable deltas. So I think
> > a much more interesting approach here would be perf tests. But:
> > 
> >    - we'd want to count those as coverage, and that likely makes your
> >      coverage tests prohibitively expensive
> > 
> >    - it requires a specialized repo to demonstrate, which most people
> >      aren't going to have handy
> 
> Do you have regularly-running tests that check this in your infrastructure?
> As long as someone would notice if this code starts failing, that would be
> enough.

We do integration tests, and this feature gets exercised quite a bit in
production at GitHub. But none of that caught the breakage I fixed this
morning for the simple fact that we don't keep up with upstream master
in real-time. We're running the delta-island code I wrote years ago, and
the bug was introduced by the upstreaming. Probably in a month or three
I'll merge v2.20 into our fork, and we would have noticed then.

I've had dreams of following upstream more closely, but there are two
blockers:

  - we have enough custom bits that the merges are time-consuming (and
    introduce the possibility of funny interactions or regressions). I'd
    like to reduce our overall diff from upstream, but it's slow going
    and time is finite. (And ironically, it's made harder in some ways
    by the lag, because many of our topics are backports of things I
    send upstream).

  - deploying the ".0" release from master can be scary. For instance, I
    was really happy to have a fix for 9ac3f0e5b3 (pack-objects: fix
    performance issues on packing large deltas, 2018-07-22) before
    putting the bug into production.

> > Yeah, this triggers if your regex has more than one capture group (and
> > likewise, we almost certainly don't run the "you have too many groups"
> > warning).
> 
> Did you know that regexes are notoriously under-tested [1]? When looking at
> this code, I didn't even know regexes were involved (but I didn't look
> enough at the context).
> 
> [1] https://dl.acm.org/citation.cfm?id=3236072

Heh. Well, fortunately we are compiling regexes from the user's config,
so it's not Git's fault if the regexes are bad. ;)

(Of course on our production servers I'm on the hook for both sides!)

-Peff

Re: [PATCH] rebase: mark a test as failing with rebase.useBuiltin=false

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2428 bytes --]

Hi Ævar,

On Tue, 20 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> Mark a test added in 04519d7201 ("rebase: validate -C<n> and
> --whitespace=<mode> parameters early", 2018-11-14) as only succeeding
> with the builtin version of rebase. It would be nice if the
> shellscript version had the same fix, but it's on its way out, and the
> author is not interested in fixing it[1].

It's not that I am not interested in fixing it. It's more like I hoped
that I could work on Git for Windows v2.20.0-rc0 and rely on you to fix
this bug.

Now that Git for Windows v2.20.0-rc0 is out (see
https://github.com/git-for-windows/git/releases/tag/v2.20.0-rc0.windows.1),
expect a patch soon (see https://github.com/gitgitgadget/git/pull/86).

Ciao,
Dscho

> 
> This makes the entire test suite pass again with the
> GIT_TEST_REBASE_USE_BUILTIN=false mode added in my 62c23938fa ("tests:
> add a special setup where rebase.useBuiltin is off", 2018-11-14).
> 
> 1. https://public-inbox.org/git/nycvar.QRO.7.76.6.1811201157170.41@tvgsbejvaqbjf.bet/
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
> 
> On Tue, Nov 20 2018, Johannes Schindelin wrote:
> 
> > [...]
> > Maybe you can concoct a prereq for this test? Something like
> >
> > test_lazy_prereq BUILTIN_REBASE '
> >       test true = "${GIT_TEST_REBASE_USE_BUILTIN:-true}"
> > '
> 
> It's better to just mark the test as needing the prereq turned
> off. E.g. this is what we do for the split index tests & now for the
> gettext tests. That way we always run the test, but just indicate that
> it relies on GIT_TEST_REBASE_USE_BUILTIN being unset.
> 
>  t/t3406-rebase-message.sh | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/t/t3406-rebase-message.sh b/t/t3406-rebase-message.sh
> index 38bd876cab..77e5bbb3d5 100755
> --- a/t/t3406-rebase-message.sh
> +++ b/t/t3406-rebase-message.sh
> @@ -84,7 +84,8 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
>  	test_i18ngrep "invalid-ref" err
>  '
>  
> -test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
> +test_expect_success 'builtin rebase: error out early upon -C<n> or --whitespace=<bad>' '
> +	sane_unset GIT_TEST_REBASE_USE_BUILTIN &&
>  	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
>  	test_i18ngrep "numerical value" err &&
>  	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
> -- 
> 2.20.0.rc0.387.gc7a69e6b6c
> 
> 

Re: Git Test Coverage Report (v2.20.0-rc0)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

Hi Stolee,

On Mon, 19 Nov 2018, Derrick Stolee wrote:

> On 11/19/2018 1:33 PM, Ævar Arnfjörð Bjarmason wrote:
> > On Mon, Nov 19 2018, Derrick Stolee wrote:
> >
> > > [...]
> > > builtin/rebase.c
> > > 62c23938fa 55) return env;
> > > [...]
> > > Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup
> > > where rebase.useBuiltin is off
> > This one would be covered with
> > GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
> > the rest of the coverage would look different when passed through the
> > various GIT_TEST_* options.
> >
> 
> Thanks for pointing out this GIT_TEST_* variable to me. I had been running
> builds with some of them enabled, but didn't know about this one.
> 
> Unfortunately, t3406-rebase-message.sh fails with
> GIT_TEST_REBASE_USE_BUILTIN=false and it bisects to 4520c2337: Merge branch
> 'ab/rebase-in-c-escape-hatch'.
> 
> The issue is that the commit 04519d72 "rebase: validate -C<n> and
> --whitespace=<mode> parameters early" introduced the following test that cares
> about error messages:
> 
> +test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
> +       test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
> +       test_i18ngrep "numerical value" err &&
> +       test_must_fail git rebase --whitespace=bad HEAD 2>err &&
> +       test_i18ngrep "Invalid whitespace option" err
> +'
> 
> The merge commit then was the first place where this test could run with that
> variable.
> 
> What's the correct fix here? Force the builtin rebase in this test? Unify the
> error message in the non-builtin case?

I am obviously biased, but my take is that the correct fix is in
https://public-inbox.org/git/pull.86.git.gitgitgadget@gmail.com

Ciao,
Dscho

[ANNOUNCE] Git v2.20.0-rc1

[Junio C Hamano]

A release candidate Git v2.20.0-rc1 is now available for testing
at the usual places.  It is comprised of 915 non-merge commits
since v2.19.0, contributed by 73 people, 24 of which are new faces.

The tarballs are found at:

    https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the
'v2.20.0-rc1' tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.19.0 are as follows.
Welcome to the Git development community!

  Aaron Lindsay, Alexander Pyhalov, Anton Serbulov, Brendan
  Forster, Carlo Marcelo Arenas Belón, Daniels Umanovskis, David
  Zych, Đoàn Trần Công Danh, Frederick Eaton, James Knight,
  Jann Horn, Joshua Watt, Loo Rong Jie, Lucas De Marchi, Matthew
  DeVore, Mihir Mehta, Nickolai Belakovski, Roger Strain, Sam
  McKelvie, Saulius Gurklys, Shulhan, Steven Fernandez, Strain,
  Roger L, and Tim Schumacher.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Ævar Arnfjörð Bjarmason, Alban Gruin, Andreas Gruenbacher,
  Andreas Heiduk, Antonio Ospite, Ben Peart, Brandon Williams,
  brian m. carlson, Christian Couder, Christian Hesse, Denton Liu,
  Derrick Stolee, Elijah Newren, Eric Sunshine, Jeff Hostetler,
  Jeff King, Johannes Schindelin, Johannes Sixt, Jonathan Nieder,
  Jonathan Tan, Josh Steadmon, Junio C Hamano, Karsten Blees,
  Luke Diamand, Martin Ågren, Max Kirillov, Michael Witten,
  Michał Górny, Nguyễn Thái Ngọc Duy, Noam Postavsky,
  Olga Telezhnaya, Phillip Wood, Pratik Karki, Rafael Ascensão,
  Ralf Thielow, Ramsay Jones, Rasmus Villemoes, René Scharfe,
  Sebastian Staudt, Stefan Beller, Stephen P. Smith, Steve Hoelzer,
  SZEDER Gábor, Tao Qingyun, Taylor Blau, Thomas Gummerer,
  Todd Zullinger, Torsten Bögershausen, and Uwe Kleine-König.

----------------------------------------------------------------

Git 2.20 Release Notes (draft)
==============================

Backward Compatibility Notes
----------------------------

 * "git branch -l <foo>" used to be a way to ask a reflog to be
   created while creating a new branch, but that is no longer the
   case.  It is a short-hand for "git branch --list <foo>" now.

 * "git push" into refs/tags/* hierarchy is rejected without getting
   forced, but "git fetch" (misguidedly) used the "fast forwarding"
   rule used for the refs/heads/* hierarchy; this has been corrected,
   which means some fetches of tags that did not fail with older
   version of Git will fail without "--force" with this version.

 * "git help -a" now gives verbose output (same as "git help -av").
   Those who want the old output may say "git help --no-verbose -a"..

 * "git cpn --help", when "cpn" is an alias to, say, "cherry-pick -n",
   reported only the alias expansion of "cpn" in earlier versions of
   Git.  It now runs "git cherry-pick --help" to show the manual page
   of the command, while sending the alias expansion to the standard
   error stream.

 * "git send-email" learned to grab address-looking string on any
   trailer whose name ends with "-by". This is a backward-incompatible
   change.  Adding "--suppress-cc=misc-by" on the command line, or
   setting sendemail.suppresscc configuration variable to "misc-by",
   can be used to disable this behaviour.


Updates since v2.19
-------------------

UI, Workflows & Features

 * Running "git clone" against a project that contain two files with
   pathnames that differ only in cases on a case insensitive
   filesystem would result in one of the files lost because the
   underlying filesystem is incapable of holding both at the same
   time.  An attempt is made to detect such a case and warn.

 * "git checkout -b newbranch [HEAD]" should not have to do as much as
   checking out a commit different from HEAD.  An attempt is made to
   optimize this special case.

 * "git rev-list --stdin </dev/null" used to be an error; it now shows
   no output without an error.  "git rev-list --stdin --default HEAD"
   still falls back to the given default when nothing is given on the
   standard input.

 * Lift code from GitHub to restrict delta computation so that an
   object that exists in one fork is not made into a delta against
   another object that does not appear in the same forked repository.

 * "git format-patch" learned new "--interdiff" and "--range-diff"
   options to explain the difference between this version and the
   previous attempt in the cover letter (or after the three-dashes as
   a comment).

 * "git mailinfo" used in "git am" learned to make a best-effort
   recovery of a patch corrupted by MUA that sends text/plain with
   format=flawed option.
   (merge 3aa4d81f88 rs/mailinfo-format-flowed later to maint).

 * The rules used by "git push" and "git fetch" to determine if a ref
   can or cannot be updated were inconsistent; specifically, fetching
   to update existing tags were allowed even though tags are supposed
   to be unmoving anchoring points.  "git fetch" was taught to forbid
   updates to existing tags without the "--force" option.

 * "git multi-pack-index" learned to detect corruption in the .midx
   file it uses, and this feature has been integrated into "git fsck".

 * Generation of (experimental) commit-graph files have so far been
   fairly silent, even though it takes noticeable amount of time in a
   meaningfully large repository.  The users will now see progress
   output.

 * The minimum version of Windows supported by Windows port of Git is
   now set to Vista.

 * The completion script (in contrib/) learned to complete a handful of
   options "git stash list" command takes.

 * The completion script (in contrib/) learned that "git fetch
   --multiple" only takes remote names as arguments and no refspecs.

 * "git status" learns to show progress bar when refreshing the index
   takes a long time.
   (merge ae9af12287 nd/status-refresh-progress later to maint).

 * "git help -a" and "git help -av" give different pieces of
   information, and generally the "verbose" version is more friendly
   to the new users.  "git help -a" by default now uses the more
   verbose output (with "--no-verbose", you can go back to the
   original).  Also "git help -av" now lists aliases and external
   commands, which it did not used to.

 * Unlike "grep", "git grep" by default recurses to the whole tree.
   The command learned "git grep --recursive" option, so that "git
   grep --no-recursive" can serve as a synonym to setting the
   max-depth to 0.

 * When pushing into a repository that borrows its objects from an
   alternate object store, "git receive-pack" that responds to the
   push request on the other side lists the tips of refs in the
   alternate to reduce the amount of objects transferred.  This
   sometimes is detrimental when the number of refs in the alternate
   is absurdly large, in which case the bandwidth saved in potentially
   fewer objects transferred is wasted in excessively large ref
   advertisement.  The alternate refs that are advertised are now
   configurable with a pair of configuration variables.

 * "git cmd --help" when "cmd" is aliased used to only say "cmd is
   aliased to ...".  Now it shows that to the standard error stream
   and runs "git $cmd --help" where $cmd is the first word of the
   alias expansion.

 * The documentation of "git gc" has been updated to mention that it
   is no longer limited to "pruning away crufts" but also updates
   ancillary files like commit-graph as a part of repository
   optimization.

 * "git p4 unshelve" improvements.

 * The logic to select the default user name and e-mail on Windows has
   been improved.
   (merge 501afcb8b0 js/mingw-default-ident later to maint).

 * The "rev-list --filter" feature learned to exclude all trees via
   "tree:0" filter.

 * "git send-email" learned to grab address-looking string on any
   trailer whose name ends with "-by"; --suppress-cc=misc-by on the
   command line, or setting sendemail.suppresscc configuration
   variable to "misc-by", can be used to disable this behaviour.

 * Developer builds now uses -Wunused-function compilation option.

 * One of our CI tests to run with "unusual/experimental/random"
   settings now also uses commit-graph and midx.

 * "git mergetool" learned to take the "--[no-]gui" option, just like
   "git difftool" does.

 * "git rebase -i" learned a new insn, 'break', that the user can
   insert in the to-do list.  Upon hitting it, the command returns
   control back to the user.

 * New "--pretty=format:" placeholders %GF and %GP that show the GPG
   key fingerprints have been invented.

 * On platforms with recent cURL library, http.sslBackend configuration
   variable can be used to choose a different SSL backend at runtime.
   The Windows port uses this mechanism to switch between OpenSSL and
   Secure Channel while talking over the HTTPS protocol.

 * "git send-email" learned to disable SMTP authentication via the
   "--smtp-auth=none" option, even when the smtp username is given
   (which turns the authentication on by default).

 * A fourth class of configuration files (in addition to the
   traditional "system wide", "per user in the $HOME directory" and
   "per repository in the $GIT_DIR/config") has been introduced so
   that different worktrees that share the same repository (hence the
   same $GIT_DIR/config file) can use different customization.

 * A pattern with '**' that does not have a slash on either side used
   to be an invalid one, but the code now treats such double-asterisks
   the same way as two normal asterisks that happen to be adjacent to
   each other.
   (merge e5bbe09e88 nd/wildmatch-double-asterisk later to maint).

 * The "--no-patch" option, which can be used to get a high-level
   overview without the actual line-by-line patch difference shown, of
   the "range-diff" command was earlier broken, which has been
   corrected.

 * The recently merged "rebase in C" has an escape hatch to use the
   scripted version when necessary, but it hasn't been documented,
   which has been corrected.


Performance, Internal Implementation, Development Support etc.

 * When there are too many packfiles in a repository (which is not
   recommended), looking up an object in these would require
   consulting many pack .idx files; a new mechanism to have a single
   file that consolidates all of these .idx files is introduced.

 * "git submodule update" is getting rewritten piece-by-piece into C.

 * The code for computing history reachability has been shuffled,
   obtained a bunch of new tests to cover them, and then being
   improved.

 * The unpack_trees() API used in checking out a branch and merging
   walks one or more trees along with the index.  When the cache-tree
   in the index tells us that we are walking a tree whose flattened
   contents is known (i.e. matches a span in the index), as linearly
   scanning a span in the index is much more efficient than having to
   open tree objects recursively and listing their entries, the walk
   can be optimized, which has been done.

 * When creating a thin pack, which allows objects to be made into a
   delta against another object that is not in the resulting pack but
   is known to be present on the receiving end, the code learned to
   take advantage of the reachability bitmap; this allows the server
   to send a delta against a base beyond the "boundary" commit.

 * spatch transformation to replace boolean uses of !hashcmp() to
   newly introduced oideq() is added, and applied, to regain
   performance lost due to support of multiple hash algorithms.

 * Fix a bug in which the same path could be registered under multiple
   worktree entries if the path was missing (for instance, was removed
   manually).  Also, as a convenience, expand the number of cases in
   which --force is applicable.

 * Split Documentation/config.txt for easier maintenance.
   (merge 6014363f0b nd/config-split later to maint).

 * Test helper binaries clean-up.
   (merge c9a1f4161f nd/test-tool later to maint).

 * Various tests have been updated to make it easier to swap the
   hash function used for object identification.
   (merge ae0c89d41b bc/hash-independent-tests later to maint).

 * Update fsck.skipList implementation and documentation.
   (merge 371a655074 ab/fsck-skiplist later to maint).

 * An alias that expands to another alias has so far been forbidden,
   but now it is allowed to create such an alias.

 * Various test scripts have been updated for style and also correct
   handling of exit status of various commands.

 * "gc --auto" ended up calling exit(-1) upon error, which has been
   corrected to use exit(1).  Also the error reporting behaviour when
   daemonized has been updated to exit with zero status when stopping
   due to a previously discovered error (which implies there is no
   point running gc to improve the situation); we used to exit with
   failure in such a case.

 * Various codepaths in the core-ish part learned to work on an
   arbitrary in-core index structure, not necessarily the default
   instance "the_index".
   (merge b3c7eef9b0 nd/the-index later to maint).

 * Code clean-up in the internal machinery used by "git status" and
   "git commit --dry-run".
   (merge 73ba5d78b4 ss/wt-status-committable later to maint).

 * Some environment variables that control the runtime options of Git
   used during tests are getting renamed for consistency.
   (merge 4231d1ba99 bp/rename-test-env-var later to maint).

 * A pair of new extensions to the index file have been introduced.
   They allow the index file to be read in parallel for performance.

 * The oidset API was built on top of the oidmap API which in turn is
   on the hashmap API.  Replace the implementation to build on top of
   the khash API and gain performance.

 * Over some transports, fetching objects with an exact commit object
   name can be done without first seeing the ref advertisements.  The
   code has been optimized to exploit this.

 * In a partial clone that will lazily be hydrated from the
   originating repository, we generally want to avoid "does this
   object exist (locally)?" on objects that we deliberately omitted
   when we created the clone.  The cache-tree codepath (which is used
   to write a tree object out of the index) however insisted that the
   object exists, even for paths that are outside of the partial
   checkout area.  The code has been updated to avoid such a check.

 * To help developers, an EditorConfig file that attempts to follow
   the project convention has been added.
   (merge b548d698a0 bc/editorconfig later to maint).

 * The result of coverage test can be combined with "git blame" to
   check the test coverage of code introduced recently with a new
   'coverage-diff' tool (in contrib/).
   (merge 783faedd65 ds/coverage-diff later to maint).

 * An experiment to fuzz test a few areas, hopefully we can gain more
   coverage to various areas.

 * More codepaths are moving away from hardcoded hash sizes.

 * The way the Windows port figures out the current directory has been
   improved.

 * The way DLLs are loaded on the Windows port has been improved.

 * Some tests have been reorganized and renamed; "ls t/" now gives a
   better overview of what is tested for these scripts than before.

 * "git rebase" and "git rebase -i" have been reimplemented in C.

 * Windows port learned to use nano-second resolution file timestamps.

 * The overly large Documentation/config.txt file have been split into
   million little pieces.  This potentially allows each individual piece
   included into the manual page of the command it affects more easily.

 * Replace three string-list instances used as look-up tables in "git
   fetch" with hashmaps.

 * Unify code to read the author-script used in "git am" and the
   commands that use the sequencer machinery, e.g. "git rebase -i".

 * In preparation to the day when we can deprecate and remove the
   "rebase -p", make sure we can skip and later remove tests for
   it.

 * The history traversal used to implement the tag-following has been
   optimized by introducing a new helper.

 * The helper function to refresh the cached stat information in the
   in-core index has learned to perform the lstat() part of the
   operation in parallel on multi-core platforms.

 * The code to traverse objects for reachability, used to decide what
   objects are unreferenced and expendable, have been taught to also
   consider per-worktree refs of other worktrees as starting points to
   prevent data loss.

 * "git add" needs to internally run "diff-files" equivalent, and the
   codepath learned the same optimization as "diff-files" has to run
   lstat(2) in parallel to find which paths have been updated in the
   working tree.

 * The procedure to install dependencies before testing at Travis CI
   is getting revamped for both simplicity and flexibility, taking
   advantage of the recent move to the vm-based environment.

 * The support for format-patch (and send-email) by the command-line
   completion script (in contrib/) has been simplified a bit.

 * The revision walker machinery learned to take advantage of the
   commit generation numbers stored in the commit-graph file.

 * The codebase has been cleaned up to reduce "#ifndef NO_PTHREADS".

 * The way -lcurl library gets linked has been simplified by taking
   advantage of the fact that we can just ask curl-config command how.

 * Various functions have been audited for "-Wunused-parameter" warnings
   and bugs in them got fixed.

 * A sanity check for start-up sequence has been added in the config
   API codepath.

 * The build procedure to link for fuzzing test has been made
   customizable with a new Makefile variable.

 * The way "git rebase" parses and forwards the command line options
   meant for underlying "git am" has been revamped, which fixed for
   options with parameters that were not passed correctly.

 * Our testing framework uses a special i18n "poisoned localization"
   feature to find messages that ought to stay constant but are
   incorrectly marked to be translated.  This feature has been made
   into a runtime option (it used to be a compile-time option).

 * "git push" used to check ambiguities between object-names and
   refnames while processing the list of refs' old and new values,
   which was unnecessary (as it knew that it is feeding raw object
   names).  This has been optimized out.

 * The xcurl_off_t() helper function is used to cast size_t to
   curl_off_t, but some compilers gave warnings against the code to
   ensure the casting is done without wraparound, when size_t is
   narrower than curl_off_t.  This warning has been squelched.

 * Code preparation to replace ulong vars with size_t vars where
   appropriate continues.

 * The "test installed Git" mode of our test suite has been updated to
   work better.

 * A coding convention around the Coccinelle semantic patches to have
   two classes to ease code migration process has been proposed and
   its support has been added to the Makefile.


Fixes since v2.19
-----------------

 * "git interpret-trailers" and its underlying machinery had a buggy
   code that attempted to ignore patch text after commit log message,
   which triggered in various codepaths that will always get the log
   message alone and never get such an input.
   (merge 66e83d9b41 jk/trailer-fixes later to maint).

 * Malformed or crafted data in packstream can make our code attempt
   to read or write past the allocated buffer and abort, instead of
   reporting an error, which has been fixed.

 * "git rebase -i" did not clear the state files correctly when a run
   of "squash/fixup" is aborted and then the user manually amended the
   commit instead, which has been corrected.
   (merge 10d2f35436 js/rebase-i-autosquash-fix later to maint).

 * When fsmonitor is in use, after operation on submodules updates
   .gitmodules, we lost track of the fact that we did so and relied on
   stale fsmonitor data.
   (merge 43f1180814 bp/mv-submodules-with-fsmonitor later to maint).

 * Fix for a long-standing bug that leaves the index file corrupt when
   it shrinks during a partial commit.
   (merge 6c003d6ffb jk/reopen-tempfile-truncate later to maint).

 * Further fix for O_APPEND emulation on Windows
   (merge eeaf7ddac7 js/mingw-o-append later to maint).

 * A corner case bugfix in "git rerere" code.
   (merge ad2bf0d9b4 en/rerere-multi-stage-1-fix later to maint).

 * "git add ':(attr:foo)'" is not supported and is supposed to be
   rejected while the command line arguments are parsed, but we fail
   to reject such a command line upfront.
   (merge 84d938b732 nd/attr-pathspec-fix later to maint).

 * Recent update broke the reachability algorithm when refs (e.g.
   tags) that point at objects that are not commit were involved,
   which has been fixed.

 * "git rebase" etc. in Git 2.19 fails to abort when given an empty
   commit log message as result of editing, which has been corrected.
   (merge a3ec9eaf38 en/sequencer-empty-edit-result-aborts later to maint).

 * The code to backfill objects in lazily cloned repository did not
   work correctly, which has been corrected.
   (merge e68302011c jt/lazy-object-fetch-fix later to maint).

 * Update error messages given by "git remote" and make them consistent.
   (merge 5025425dff ms/remote-error-message-update later to maint).

 * "git update-ref" learned to make both "--no-deref" and "--stdin"
   work at the same time.
   (merge d345e9fbe7 en/update-ref-no-deref-stdin later to maint).

 * Recently added "range-diff" had a corner-case bug to cause it
   segfault, which has been corrected.
   (merge e467a90c7a tg/range-diff-corner-case-fix later to maint).

 * The recently introduced commit-graph auxiliary data is incompatible
   with mechanisms such as replace & grafts that "breaks" immutable
   nature of the object reference relationship.  Disable optimizations
   based on its use (and updating existing commit-graph) when these
   incompatible features are in use in the repository.
   (merge 829a321569 ds/commit-graph-with-grafts later to maint).

 * The mailmap file update.
   (merge 255eb03edf jn/mailmap-update later to maint).

 * The code in "git status" sometimes hit an assertion failure.  This
   was caused by a structure that was reused without cleaning the data
   used for the first run, which has been corrected.
   (merge 3e73cc62c0 en/status-multiple-renames-to-the-same-target-fix later to maint).

 * "git fetch $repo $object" in a partial clone did not correctly
   fetch the asked-for object that is referenced by an object in
   promisor packfile, which has been fixed.

 * A corner-case bugfix.
   (merge c5cbb27cb5 sm/show-superproject-while-conflicted later to maint).

 * Various fixes to "diff --color-moved-ws".

 * A partial clone that is configured to lazily fetch missing objects
   will on-demand issue a "git fetch" request to the originating
   repository to fill not-yet-obtained objects.  The request has been
   optimized for requesting a tree object (and not the leaf blob
   objects contained in it) by telling the originating repository that
   no blobs are needed.
   (merge 4c7f9567ea jt/non-blob-lazy-fetch later to maint).

 * The codepath to support the experimental split-index mode had
   remaining "racily clean" issues fixed.
   (merge 4c490f3d32 sg/split-index-racefix later to maint).

 * "git log --graph" showing an octopus merge sometimes miscounted the
   number of display columns it is consuming to show the merge and its
   parent commits, which has been corrected.
   (merge 04005834ed np/log-graph-octopus-fix later to maint).

 * "git range-diff" did not work well when the compared ranges had
   changes in submodules and the "--submodule=log" was used.

 * The implementation of run_command() API on the UNIX platforms had a
   bug that caused a command not on $PATH to be found in the current
   directory.
   (merge f67b980771 jk/run-command-notdot later to maint).

 * A mutex used in "git pack-objects" were not correctly initialized
   and this caused "git repack" to dump core on Windows.
   (merge 34204c8166 js/pack-objects-mutex-init-fix later to maint).

 * Under certain circumstances, "git diff D:/a/b/c D:/a/b/d" on
   Windows would strip initial parts from the paths because they
   were not recognized as absolute, which has been corrected.
   (merge ffd04e92e2 js/diff-notice-has-drive-prefix later to maint).

 * The receive.denyCurrentBranch=updateInstead codepath kicked in even
   when the push should have been rejected due to other reasons, such
   as it does not fast-forward or the update-hook rejects it, which
   has been corrected.
   (merge b072a25fad jc/receive-deny-current-branch-fix later to maint).

 * The logic to determine the archive type "git archive" uses did not
   correctly kick in for "git archive --remote", which has been
   corrected.

 * "git repack" in a shallow clone did not correctly update the
   shallow points in the repository, leading to a repository that
   does not pass fsck.
   (merge 5dcfbf564c js/shallow-and-fetch-prune later to maint).

 * Some codepaths failed to form a proper URL when .gitmodules record
   the URL to a submodule repository as relative to the repository of
   superproject, which has been corrected.
   (merge e0a862fdaf sb/submodule-url-to-absolute later to maint).

 * "git fetch" over protocol v2 into a shallow repository failed to
   fetch full history behind a new tip of history that was diverged
   before the cut-off point of the history that was previously fetched
   shallowly.

 * The command line completion machinery (in contrib/) has been
   updated to allow the completion script to tweak the list of options
   that are reported by the parse-options machinery correctly.
   (merge 276b49ff34 nd/completion-negation later to maint).

 * Operations on promisor objects make sense in the context of only a
   small subset of the commands that internally use the revisions
   machinery, but the "--exclude-promisor-objects" option were taken
   and led to nonsense results by commands like "log", to which it
   didn't make much sense.  This has been corrected.
   (merge 669b1d2aae md/exclude-promisor-objects-fix later to maint).

 * The "container" mode of TravisCI is going away.  Our .travis.yml
   file is getting prepared for the transition.
   (merge 32ee384be8 ss/travis-ci-force-vm-mode later to maint).

 * Our test scripts can now take the '-V' option as a synonym for the
   '--verbose-log' option.
   (merge a5f52c6dab sg/test-verbose-log later to maint).

 * A regression in Git 2.12 era made "git fsck" fall into an infinite
   loop while processing truncated loose objects.
   (merge 18ad13e5b2 jk/detect-truncated-zlib-input later to maint).

 * "git ls-remote $there foo" was broken by recent update for the
   protocol v2 and stopped showing refs that match 'foo' that are not
   refs/{heads,tags}/foo, which has been fixed.
   (merge 6a139cdd74 jk/proto-v2-ref-prefix-fix later to maint).

 * Additional comment on a tricky piece of code to help developers.
   (merge 0afbe3e806 jk/stream-pack-non-delta-clarification later to maint).

 * A couple of tests used to leave the repository in a state that is
   deliberately corrupt, which have been corrected.
   (merge aa984dbe5e ab/pack-tests-cleanup later to maint).

 * The submodule support has been updated to read from the blob at
   HEAD:.gitmodules when the .gitmodules file is missing from the
   working tree.
   (merge 2b1257e463 ao/submodule-wo-gitmodules-checked-out later to maint).

 * "git fetch" was a bit loose in parsing responses from the other side
   when talking over the protocol v2.

 * "git rev-parse --exclude=* --branches --branches"  (i.e. first
   saying "add only things that do not match '*' out of all branches"
   and then adding all branches, without any exclusion this time")
   worked as expected, but "--exclude=* --all --all" did not work the
   same way, which has been fixed.
   (merge 5221048092 ag/rev-parse-all-exclude-fix later to maint).

 * "git send-email --transfer-encoding=..." in recent versions of Git
   sometimes produced an empty "Content-Transfer-Encoding:" header,
   which has been corrected.
   (merge 3c88e46f1a al/send-email-auto-cte-fixup later to maint).

 * The interface into "xdiff" library used to discover the offset and
   size of a generated patch hunk by first formatting it into the
   textual hunk header "@@ -n,m +k,l @@" and then parsing the numbers
   out.  A new interface has been introduced to allow callers a more
   direct access to them.
   (merge 5eade0746e jk/xdiff-interface later to maint).

 * Pathspec matching against a tree object were buggy when negative
   pathspec elements were involved, which has been fixed.
   (merge b7845cebc0 nd/tree-walk-path-exclusion later to maint).

 * "git merge" and "git pull" that merges into an unborn branch used
   to completely ignore "--verify-signatures", which has been
   corrected.
   (merge 01a31f3bca jk/verify-sig-merge-into-void later to maint).

 * "git rebase --autostash" did not correctly re-attach the HEAD at times.

 * "rev-parse --exclude=<pattern> --branches=<pattern>" etc. did not
   quite work, which has been corrected.
   (merge 9ab9b5df0e ra/rev-parse-exclude-glob later to maint).

 * When editing a patch in a "git add -i" session, a hunk could be
   made to no-op.  The "git apply" program used to reject a patch with
   such a no-op hunk to catch user mistakes, but it is now updated to
   explicitly allow a no-op hunk in an edited patch.
   (merge 22cb3835b9 js/apply-recount-allow-noop later to maint).

 * The URL to an MSDN page in a comment has been updated.
   (merge 2ef2ae2917 js/mingw-msdn-url later to maint).

 * "git ls-remote --sort=<thing>" can feed an object that is not yet
   available into the comparison machinery and segfault, which has
   been corrected to check such a request upfront and reject it.

 * When "git bundle" aborts due to an empty commit ranges
   (i.e. resulting in an empty pack), it left a file descriptor to an
   lockfile open, which resulted in leftover lockfile on Windows where
   you cannot remove a file with an open file descriptor.  This has
   been corrected.
   (merge 2c8ee1f53c jk/close-duped-fd-before-unlock-for-bundle later to maint).

 * "git format-patch --stat=<width>" can be used to specify the width
   used by the diffstat (shown in the cover letter).
   (merge 284aeb7e60 nd/format-patch-cover-letter-stat-width later to maint).

 * The way .git/index and .git/sharedindex* files were initially
   created gave these files different perm bits until they were
   adjusted for shared repository settings.  This was made consistent.
   (merge c9d6c78870 cc/shared-index-permbits later to maint).

 * Code cleanup, docfix, build fix, etc.
   (merge 96a7501aad ts/doc-build-manpage-xsl-quietly later to maint).
   (merge b9b07efdb2 tg/conflict-marker-size later to maint).
   (merge fa0aeea770 sg/doc-trace-appends later to maint).
   (merge d64324cb60 tb/void-check-attr later to maint).
   (merge c3b9bc94b9 en/double-semicolon-fix later to maint).
   (merge 79336116f5 sg/t3701-tighten-trace later to maint).
   (merge 801fa63a90 jk/dev-build-format-security later to maint).
   (merge 0597dd62ba sb/string-list-remove-unused later to maint).
   (merge db2d36fad8 bw/protocol-v2 later to maint).
   (merge 456d7cd3a9 sg/split-index-test later to maint).
   (merge 7b6057c852 tq/refs-internal-comment-fix later to maint).
   (merge 29e8dc50ad tg/t5551-with-curl-7.61.1 later to maint).
   (merge 55f6bce2c9 fe/doc-updates later to maint).
   (merge 7987d2232d jk/check-everything-connected-is-long-gone later to maint).
   (merge 4ba3c9be47 dz/credential-doc-url-matching-rules later to maint).
   (merge 4c399442f7 ma/commit-graph-docs later to maint).
   (merge fc0503b04e ma/t1400-undebug-test later to maint).
   (merge e56b53553a nd/packobjectshook-doc-fix later to maint).
   (merge c56170a0c4 ma/mailing-list-address-in-git-help later to maint).
   (merge 6e8fc70fce rs/sequencer-oidset-insert-avoids-dups later to maint).
   (merge ad0b8f9575 mw/doc-typofixes later to maint).
   (merge d9f079ad1a jc/how-to-document-api later to maint).
   (merge b1492bf315 ma/t7005-bash-workaround later to maint).
   (merge ac1f98a0df du/rev-parse-is-plumbing later to maint).
   (merge ca8ed443a5 mm/doc-no-dashed-git later to maint).
   (merge ce366a8144 du/get-tar-commit-id-is-plumbing later to maint).
   (merge 61018fe9e0 du/cherry-is-plumbing later to maint).
   (merge c7e5fe79b9 sb/strbuf-h-update later to maint).
   (merge 8d2008196b tq/branch-create-wo-branch-get later to maint).
   (merge 2e3c894f4b tq/branch-style-fix later to maint).
   (merge c5d844af9c sg/doc-show-branch-typofix later to maint).
   (merge 081d91618b ah/doc-updates later to maint).
   (merge b84c783882 jc/cocci-preincr later to maint).
   (merge 5e495f8122 uk/merge-subtree-doc-update later to maint).
   (merge aaaa881822 jk/uploadpack-packobjectshook-fix later to maint).
   (merge 3063477445 tb/char-may-be-unsigned later to maint).
   (merge 8c64bc9420 sg/test-rebase-editor-fix later to maint).
   (merge 71571cd7d6 ma/sequencer-do-reset-saner-loop-termination later to maint).
   (merge 9a4cb8781e cb/notes-freeing-always-null-fix later to maint).

----------------------------------------------------------------

Changes since v2.19.0 are as follows:

Aaron Lindsay (1):
      send-email: avoid empty transfer encoding header

Alban Gruin (21):
      sequencer: make three functions and an enum from sequencer.c public
      rebase -i: rewrite append_todo_help() in C
      editor: add a function to launch the sequence editor
      rebase -i: rewrite the edit-todo functionality in C
      sequencer: add a new function to silence a command, except if it fails
      rebase -i: rewrite setup_reflog_action() in C
      rebase -i: rewrite checkout_onto() in C
      sequencer: refactor append_todo_help() to write its message to a buffer
      sequencer: change the way skip_unnecessary_picks() returns its result
      t3404: todo list with commented-out commands only aborts
      rebase -i: rewrite complete_action() in C
      rebase -i: remove unused modes and functions
      rebase -i: implement the logic to initialize $revisions in C
      rebase -i: rewrite the rest of init_revisions_and_shortrevisions() in C
      rebase -i: rewrite write_basic_state() in C
      rebase -i: rewrite init_basic_state() in C
      rebase -i: implement the main part of interactive rebase as a builtin
      rebase--interactive2: rewrite the submodes of interactive rebase in C
      rebase -i: remove git-rebase--interactive.sh
      rebase -i: move rebase--helper modes to rebase--interactive
      p3400: replace calls to `git checkout -b' by `git checkout -B'

Alexander Pyhalov (1):
      t7005-editor: quote filename to fix whitespace-issue

Andreas Gruenbacher (1):
      rev-parse: clear --exclude list after 'git rev-parse --all'

Andreas Heiduk (6):
      doc: clarify boundaries of 'git worktree list --porcelain'
      doc: fix ASCII art tab spacing
      doc: fix inappropriate monospace formatting
      doc: fix descripion for 'git tag --format'
      doc: fix indentation of listing blocks in gitweb.conf.txt
      doc: fix formatting in git-update-ref

Anton Serbulov (1):
      mingw: fix getcwd when the parent directory cannot be queried

Antonio Ospite (10):
      submodule: add a print_config_from_gitmodules() helper
      submodule: factor out a config_set_in_gitmodules_file_gently function
      t7411: merge tests 5 and 6
      t7411: be nicer to future tests and really clean things up
      submodule--helper: add a new 'config' subcommand
      submodule: use the 'submodule--helper config' command
      t7506: clean up .gitmodules properly before setting up new scenario
      submodule: add a helper to check if it is safe to write to .gitmodules
      submodule: support reading .gitmodules when it's not in the working tree
      t/helper: add test-submodule-nested-repo-config

Ben Peart (19):
      checkout: optimize "git checkout -b <new_branch>"
      git-mv: allow submodules and fsmonitor to work together
      t/README: correct spelling of "uncommon"
      preload-index: use git_env_bool() not getenv() for customization
      fsmonitor: update GIT_TEST_FSMONITOR support
      read-cache: update TEST_GIT_INDEX_VERSION support
      preload-index: update GIT_FORCE_PRELOAD_TEST support
      read-cache: clean up casting and byte decoding
      eoie: add End of Index Entry (EOIE) extension
      config: add new index.threads config setting
      read-cache: load cache extensions on a worker thread
      ieot: add Index Entry Offset Table (IEOT) extension
      read-cache: load cache entries on worker threads
      reset: don't compute unstaged changes after reset when --quiet
      reset: add new reset.quiet config setting
      reset: warn when refresh_index() takes more than 2 seconds
      speed up refresh_index() by utilizing preload_index()
      add: speed up cmd_add() by utilizing read_cache_preload()
      refresh_index: remove unnecessary calls to preload_index()

Brandon Williams (1):
      config: document value 2 for protocol.version

Brendan Forster (1):
      http: add support for disabling SSL revocation checks in cURL

Carlo Marcelo Arenas Belón (8):
      unpack-trees: avoid dead store for struct progress
      multi-pack-index: avoid dead store for struct progress
      read-cache: use of memory after it is freed
      commit-slabs: move MAYBE_UNUSED out
      khash: silence -Wunused-function for delta-islands
      compat: make sure git_mmap is not expected to write
      sequencer: cleanup for gcc warning in non developer mode
      builtin/notes: remove unnecessary free

Christian Couder (3):
      pack-objects: refactor code into compute_layer_order()
      pack-objects: move tree_depth into 'struct packing_data'
      pack-objects: move 'layer' into 'struct packing_data'

Christian Hesse (2):
      subtree: add build targets 'man' and 'html'
      subtree: make install targets depend on build targets

Daniels Umanovskis (3):
      doc: move git-rev-parse from porcelain to plumbing
      doc: move git-get-tar-commit-id to plumbing
      doc: move git-cherry to plumbing

David Zych (1):
      doc: clarify gitcredentials path component matching

Denton Liu (3):
      mergetool: accept -g/--[no-]gui as arguments
      completion: support `git mergetool --[no-]gui`
      doc: document diff/merge.guitool config keys

Derrick Stolee (93):
      multi-pack-index: add design document
      multi-pack-index: add format details
      multi-pack-index: add builtin
      multi-pack-index: add 'write' verb
      midx: write header information to lockfile
      multi-pack-index: load into memory
      t5319: expand test data
      packfile: generalize pack directory list
      multi-pack-index: read packfile list
      multi-pack-index: write pack names in chunk
      midx: read pack names into array
      midx: sort and deduplicate objects from packfiles
      midx: write object ids in a chunk
      midx: write object id fanout chunk
      midx: write object offsets
      config: create core.multiPackIndex setting
      midx: read objects from multi-pack-index
      midx: use midx in abbreviation calculations
      midx: use existing midx when writing new one
      midx: use midx in approximate_object_count
      midx: prevent duplicate packfile loads
      packfile: skip loading index if in multi-pack-index
      midx: clear midx on repack
      commit-reach: move walk methods from commit.c
      commit.h: remove method declarations
      commit-reach: move ref_newer from remote.c
      commit-reach: move commit_contains from ref-filter
      upload-pack: make reachable() more generic
      upload-pack: refactor ok_to_give_up()
      upload-pack: generalize commit date cutoff
      commit-reach: move can_all_from_reach_with_flags
      test-reach: create new test tool for ref_newer
      test-reach: test in_merge_bases
      test-reach: test is_descendant_of
      test-reach: test get_merge_bases_many
      test-reach: test reduce_heads
      test-reach: test can_all_from_reach_with_flags
      test-reach: test commit_contains
      commit-reach: replace ref_newer logic
      commit-reach: make can_all_from_reach... linear
      commit-reach: use can_all_from_reach
      multi-pack-index: provide more helpful usage info
      multi-pack-index: store local property
      midx: mark bad packed objects
      midx: stop reporting garbage
      midx: fix bug that skips midx with alternates
      packfile: add all_packs list
      treewide: use get_all_packs
      midx: test a few commands that use get_all_packs
      pack-objects: consider packs in multi-pack-index
      commit-graph: update design document
      test-repository: properly init repo
      commit-graph: not compatible with replace objects
      commit-graph: not compatible with grafts
      commit-graph: not compatible with uninitialized repo
      commit-graph: close_commit_graph before shallow walk
      commit-graph: define GIT_TEST_COMMIT_GRAPH
      t3206-range-diff.sh: cover single-patch case
      t5318: use test_oid for HASH_LEN
      multi-pack-index: add 'verify' verb
      multi-pack-index: verify bad header
      multi-pack-index: verify corrupt chunk lookup table
      multi-pack-index: verify packname order
      multi-pack-index: verify missing pack
      multi-pack-index: verify oid fanout order
      multi-pack-index: verify oid lookup order
      multi-pack-index: fix 32-bit vs 64-bit size check
      multi-pack-index: verify object offsets
      multi-pack-index: report progress during 'verify'
      fsck: verify multi-pack-index
      commit-reach: properly peel tags
      commit-reach: fix memory and flag leaks
      commit-reach: cleanups in can_all_from_reach...
      commit-graph: clean up leaked memory during write
      commit-graph: reduce initial oid allocation
      midx: fix broken free() in close_midx()
      contrib: add coverage-diff script
      ci: add optional test variables
      commit-reach: fix first-parent heuristic
      midx: close multi-pack-index on repack
      multi-pack-index: define GIT_TEST_MULTI_PACK_INDEX
      packfile: close multi-pack-index in close_all_packs
      prio-queue: add 'peek' operation
      test-reach: add run_three_modes method
      test-reach: add rev-list tests
      revision.c: begin refactoring --topo-order logic
      commit/revisions: bookkeeping before refactoring
      revision.c: generation-based topo-order algorithm
      t6012: make rev-list tests more interesting
      commit-reach: implement get_reachable_subset
      test-reach: test get_reachable_subset
      remote: make add_missing_tags() linear
      pack-objects: ignore ambiguous object warnings

Elijah Newren (14):
      Remove superfluous trailing semicolons
      t4200: demonstrate rerere segfault on specially crafted merge
      rerere: avoid buffer overrun
      update-ref: fix type of update_flags variable to match its usage
      update-ref: allow --no-deref with --stdin
      sequencer: fix --allow-empty-message behavior, make it smarter
      merge-recursive: set paths correctly when three-way merging content
      merge-recursive: avoid wrapper function when unnecessary and wasteful
      merge-recursive: remove final remaining caller of merge_file_one()
      merge-recursive: rename merge_file_1() and merge_content()
      commit: fix erroneous BUG, 'multiple renames on the same target? how?'
      merge-recursive: improve auto-merging messages with path collisions
      merge-recursive: avoid showing conflicts with merge branch before HEAD
      fsck: move fsck_head_link() to get_default_heads() to avoid some globals

Eric Sunshine (26):
      format-patch: allow additional generated content in make_cover_letter()
      format-patch: add --interdiff option to embed diff in cover letter
      format-patch: teach --interdiff to respect -v/--reroll-count
      interdiff: teach show_interdiff() to indent interdiff
      log-tree: show_log: make commentary block delimiting reusable
      format-patch: allow --interdiff to apply to a lone-patch
      range-diff: respect diff_option.file rather than assuming 'stdout'
      range-diff: publish default creation factor
      range-diff: relieve callers of low-level configuration burden
      format-patch: add --range-diff option to embed diff in cover letter
      format-patch: extend --range-diff to accept revision range
      format-patch: teach --range-diff to respect -v/--reroll-count
      format-patch: add --creation-factor tweak for --range-diff
      format-patch: allow --range-diff to apply to a lone-patch
      worktree: don't die() in library function find_worktree()
      worktree: move delete_git_dir() earlier in file for upcoming new callers
      worktree: generalize delete_git_dir() to reduce code duplication
      worktree: prepare for more checks of whether path can become worktree
      worktree: disallow adding same path multiple times
      worktree: teach 'add' to respect --force for registered but missing path
      worktree: teach 'move' to override lock when --force given twice
      worktree: teach 'remove' to override lock when --force given twice
      worktree: delete .git/worktrees if empty after 'remove'
      doc-diff: fix non-portable 'man' invocation
      doc-diff: add --clean mode to remove temporary working gunk
      doc/Makefile: drop doc-diff worktree and temporary files on "make clean"

Frederick Eaton (3):
      git-archimport.1: specify what kind of Arch we're talking about
      git-column.1: clarify initial description, provide examples
      git-describe.1: clarify that "human readable" is also git-readable

James Knight (1):
      build: link with curl-defined linker flags

Jann Horn (2):
      patch-delta: fix oob read
      patch-delta: consistently report corruption

Jeff Hostetler (2):
      t0051: test GIT_TRACE to a windows named pipe
      mingw: fix mingw_open_append to work with named pipes

Jeff King (97):
      branch: make "-l" a synonym for "--list"
      Add delta-islands.{c,h}
      pack-objects: add delta-islands support
      repack: add delta-islands support
      t5320: tests for delta islands
      t/perf: factor boilerplate out of test_perf
      t/perf: factor out percent calculations
      t/perf: add infrastructure for measuring sizes
      t/perf: add perf tests for fetches from a bitmapped server
      pack-bitmap: save "have" bitmap from walk
      pack-objects: reuse on-disk deltas for thin "have" objects
      SubmittingPatches: mention doc-diff
      rev-list: make empty --stdin not an error
      trailer: use size_t for string offsets
      trailer: use size_t for iterating trailer list
      trailer: pass process_trailer_opts to trailer_info_get()
      interpret-trailers: tighten check for "---" patch boundary
      interpret-trailers: allow suppressing "---" divider
      pretty, ref-filter: format %(trailers) with no_divider option
      sequencer: ignore "---" divider when parsing trailers
      append_signoff: use size_t for string offsets
      coccinelle: use <...> for function exclusion
      introduce hasheq() and oideq()
      convert "oidcmp() == 0" to oideq()
      convert "hashcmp() == 0" to hasheq()
      convert "oidcmp() != 0" to "!oideq()"
      convert "hashcmp() != 0" to "!hasheq()"
      convert hashmap comparison functions to oideq()
      read-cache: use oideq() in ce_compare functions
      show_dirstat: simplify same-content check
      doc-diff: always use oids inside worktree
      test-delta: read input into a heap buffer
      t5303: test some corrupt deltas
      patch-delta: handle truncated copy parameters
      t5303: use printf to generate delta bases
      doc/git-branch: remove obsolete "-l" references
      bitmap_has_sha1_in_uninteresting(): drop BUG check
      t5310: test delta reuse with bitmaps
      traverse_bitmap_commit_list(): don't free result
      pack-bitmap: drop "loaded" flag
      reopen_tempfile(): truncate opened file
      doc-diff: force worktree add
      config.mak.dev: add -Wformat-security
      pack-objects: handle island check for "external" delta base
      receive-pack: update comment with check_everything_connected
      submodule--helper: use "--" to signal end of clone options
      submodule-config: ban submodule urls that start with dash
      submodule-config: ban submodule paths that start with a dash
      fsck: detect submodule urls starting with dash
      fsck: detect submodule paths starting with dash
      more oideq/hasheq conversions
      transport: drop refnames from for_each_alternate_ref
      test-tool: show tool list on error
      config.mak.dev: enable -Wunused-function
      run-command: mark path lookup errors with ENOENT
      t5410: use longer path for sample script
      upload-pack: fix broken if/else chain in config callback
      t1450: check large blob in trailing-garbage test
      check_stream_sha1(): handle input underflow
      cat-file: handle streaming failures consistently
      ls-remote: do not send ref prefixes for patterns
      ls-remote: pass heads/tags prefixes to transport
      read_istream_pack_non_delta(): document input handling
      xdiff: provide a separate emit callback for hunks
      xdiff-interface: provide a separate consume callback for hunks
      rev-list: handle flags for --indexed-objects
      approxidate: handle pending number for "specials"
      pathspec: handle non-terminated strings with :(attr)
      diff: avoid generating unused hunk header lines
      diff: discard hunk headers for patch-ids earlier
      diff: use hunk callback for word-diff
      combine-diff: use an xdiff hunk callback
      diff: convert --check to use a hunk callback
      range-diff: use a hunk callback
      xdiff-interface: drop parse_hunk_header()
      apply: mark include/exclude options as NONEG
      am: handle --no-patch-format option
      ls-files: mark exclude options as NONEG
      pack-objects: mark index-version option as NONEG
      cat-file: mark batch options with NONEG
      status: mark --find-renames option with NONEG
      format-patch: mark "--no-numbered" option with NONEG
      show-branch: mark --reflog option as NONEG
      tag: mark "--message" option with NONEG
      cat-file: report an error on multiple --batch options
      apply: return -1 from option callback instead of calling exit(1)
      parse-options: drop OPT_DATE()
      assert NOARG/NONEG behavior of parse-options callbacks
      midx: double-check large object write loop
      merge: extract verify_merge_signature() helper
      merge: handle --verify-signatures for unborn branch
      pull: handle --verify-signatures for unborn branch
      approxidate: fix NULL dereference in date_time()
      bundle: dup() output descriptor closer to point-of-use
      pack-objects: fix tree_depth and layer invariants
      pack-objects: zero-initialize tree_depth/layer arrays
      pack-objects: fix off-by-one in delta-island tree-depth computation

Johannes Schindelin (62):
      rebase -i --autosquash: demonstrate a problem skipping the last squash
      rebase -i: be careful to wrap up fixup/squash chains
      compat/poll: prepare for targeting Windows Vista
      mingw: set _WIN32_WINNT explicitly for Git for Windows
      mingw: bump the minimum Windows version to Vista
      builtin rebase: prepare for builtin rebase -i
      rebase -i: clarify what happens on a failed `exec`
      rebase -i: introduce the 'break' command
      getpwuid(mingw): initialize the structure only once
      getpwuid(mingw): provide a better default for the user name
      mingw: use domain information for default email
      http: add support for selecting SSL backends at runtime
      pack-objects: fix typo 'detla' -> 'delta'
      pack-objects (mingw): demonstrate a segmentation fault with large deltas
      pack-objects (mingw): initialize `packing_data` mutex in the correct spot
      rebase (autostash): avoid duplicate call to state_dir_path()
      rebase (autostash): store the full OID in <state-dir>/autostash
      rebase (autostash): use an explicit OID to apply the stash
      mingw: factor out code to set stat() data
      rebase --autostash: demonstrate a problem with dirty submodules
      rebase --autostash: fix issue with dirty submodules
      mingw: load system libraries the recommended way
      mingw: ensure `getcwd()` reports the correct case
      repack: point out a bug handling stale shallow info
      shallow: offer to prune only non-existing entries
      repack -ad: prune the list of shallow commits
      http: when using Secure Channel, ignore sslCAInfo by default
      t7800: fix quoting
      mingw: reencode environment variables on the fly (UTF-16 <-> UTF-8)
      config: rename `dummy` parameter to `cb` in git_default_config()
      config: allow for platform-specific core.* config settings
      config: move Windows-specific config settings into compat/mingw.c
      mingw: unset PERL5LIB by default
      mingw: fix isatty() after dup2()
      t3404: decouple some test cases from outcomes of previous test cases
      t3418: decouple test cases from a previous `rebase -p` test case
      tests: optionally skip `git rebase -p` tests
      Windows: force-recompile git.res for differing architectures
      built-in rebase: demonstrate regression with --autostash
      built-in rebase --autostash: leave the current branch alone if possible
      Update .mailmap
      rebase -r: demonstrate bug with conflicting merges
      rebase -r: do not write MERGE_HEAD unless needed
      rebase -i: include MERGE_HEAD into files to clean up
      built-in rebase --skip/--abort: clean up stale .git/<name> files
      status: rebase and merge can be in progress at the same time
      apply --recount: allow "no-op hunks"
      rebase: consolidate clean-up code before leaving reset_head()
      rebase: prepare reset_head() for more flags
      built-in rebase: reinstate `checkout -q` behavior where appropriate
      tests: fix GIT_TEST_INSTALLED's PATH to include t/helper/
      tests: respect GIT_TEST_INSTALLED when initializing repositories
      t/lib-gettext: test installed git-sh-i18n if GIT_TEST_INSTALLED is set
      mingw: use `CreateHardLink()` directly
      rebase: really just passthru the `git am` options
      rebase: validate -C<n> and --whitespace=<mode> parameters early
      config: report a bug if git_dir exists without commondir
      tests: do not require Git to be built when testing an installed Git
      tests: explicitly use `git.exe` on Windows
      mingw: replace an obsolete link with the superseding one
      legacy-rebase: backport -C<n> and --whitespace=<option> checks
      rebase: warn about the correct tree's OID

Johannes Sixt (3):
      diff: don't attempt to strip prefix from absolute Windows paths
      rebase -i: recognize short commands without arguments
      t3404-rebase-interactive: test abbreviated commands

Jonathan Nieder (9):
      gc: improve handling of errors reading gc.log
      gc: exit with status 128 on failure
      gc: do not return error for prior errors in daemonized mode
      commit-reach: correct accidental #include of C file
      mailmap: consistently normalize brian m. carlson's name
      git doc: direct bug reporters to mailing list archive
      eoie: default to not writing EOIE section
      ieot: default to not writing IEOT section
      index: make index.threads=true enable ieot and eoie

Jonathan Tan (15):
      fetch-object: unify fetch_object[s] functions
      fetch-object: set exact_oid when fetching
      connected: document connectivity in partial clones
      fetch: in partial clone, check presence of targets
      fetch-pack: avoid object flags if no_dependents
      fetch-pack: exclude blobs when lazy-fetching trees
      transport: allow skipping of ref listing
      transport: do not list refs if possible
      transport: list refs before fetch if necessary
      fetch: do not list refs if fetching only hashes
      cache-tree: skip some blob checks in partial clone
      upload-pack: make have_obj not global
      upload-pack: make want_obj not global
      upload-pack: clear flags before each v2 request
      fetch-pack: be more precise in parsing v2 response

Josh Steadmon (4):
      fuzz: add basic fuzz testing target.
      fuzz: add fuzz testing for packfile indices.
      archive: initialize archivers earlier
      Makefile: use FUZZ_CXXFLAGS for linking fuzzers

Joshua Watt (1):
      send-email: explicitly disable authentication

Junio C Hamano (34):
      Revert "doc/Makefile: drop doc-diff worktree and temporary files on "make clean""
      Initial batch post 2.19
      Second batch post 2.19
      Git 2.14.5
      Git 2.15.3
      Git 2.16.5
      Git 2.17.2
      Git 2.18.1
      Git 2.19.1
      t0000: do not get self-test disrupted by environment warnings
      CodingGuidelines: document the API in *.h files
      Declare that the next one will be named 2.20
      Third batch for 2.20
      rebase: fix typoes in error messages
      Fourth batch for 2.20
      Revert "subtree: make install targets depend on build targets"
      Fifth batch for 2.20
      receive: denyCurrentBranch=updateinstead should not blindly update
      cocci: simplify "if (++u > 1)" to "if (u++)"
      fsck: s/++i > 1/i++/
      http: give curl version warnings consistently
      Sixth batch for 2.20
      Seventh batch for 2.20
      fetch: replace string-list used as a look-up table with a hashmap
      rebase: apply cocci patch
      Eighth batch for 2.20
      Ninth batch for 2.20
      Makefile: ease dynamic-gettext-poison transition
      Tenth batch for 2.20
      Git 2.20-rc0
      RelNotes: name the release properly
      Prepare for 2.20-rc1
      Git 2.19.2
      Git 2.20-rc1

Karsten Blees (2):
      mingw: replace MSVCRT's fstat() with a Win32-based implementation
      mingw: implement nanosecond-precision file times

Loo Rong Jie (1):
      win32: replace pthread_cond_*() with much simpler code

Lucas De Marchi (1):
      range-diff: allow to diff files regardless of submodule config

Luke Diamand (3):
      git-p4: do not fail in verbose mode for missing 'fileSize' key
      git-p4: unshelve into refs/remotes/p4-unshelved, not refs/remotes/p4/unshelved
      git-p4: fully support unshelving changelists

Martin Ågren (9):
      Doc: use `--type=bool` instead of `--bool`
      git-config.txt: fix 'see: above' note
      git-commit-graph.txt: fix bullet lists
      git-commit-graph.txt: typeset more in monospace
      git-commit-graph.txt: refer to "*commit*-graph file"
      Doc: refer to the "commit-graph file" with dash
      t1400: drop debug `echo` to actually execute `test`
      builtin/commit-graph.c: UNLEAK variables
      sequencer: break out of loop explicitly

Matthew DeVore (19):
      list-objects: store common func args in struct
      list-objects: refactor to process_tree_contents
      list-objects: always parse trees gently
      t/README: reformat Do, Don't, Keep in mind lists
      Documentation: add shell guidelines
      tests: standardize pipe placement
      t/*: fix ordering of expected/observed arguments
      tests: don't swallow Git errors upstream of pipes
      t9109: don't swallow Git errors upstream of pipes
      tests: order arguments to git-rev-list properly
      rev-list: handle missing tree objects properly
      revision: mark non-user-given objects instead
      list-objects-filter: use BUG rather than die
      list-objects-filter-options: do not over-strbuf_init
      list-objects-filter: implement filter tree:0
      filter-trees: code clean-up of tests
      list-objects: support for skipping tree traversal
      Documentation/git-log.txt: do not show --exclude-promisor-objects
      exclude-promisor-objects: declare when option is allowed

Max Kirillov (1):
      http-backend test: make empty CONTENT_LENGTH test more realistic

Michael Witten (3):
      docs: typo: s/go/to/
      docs: graph: remove unnecessary `graph_update()' call
      docs: typo: s/isimilar/similar/

Michał Górny (6):
      gpg-interface.c: detect and reject multiple signatures on commits
      gpg-interface.c: use flags to determine key/signer info presence
      gpg-interface.c: support getting key fingerprint via %GF format
      gpg-interface.c: obtain primary key fingerprint as well
      t/t7510-signed-commit.sh: Add %GP to custom format checks
      t/t7510-signed-commit.sh: add signing subkey to Eris Discordia key

Mihir Mehta (1):
      doc: fix a typo and clarify a sentence

Nguyễn Thái Ngọc Duy (168):
      clone: report duplicate entries on case-insensitive filesystems
      trace.h: support nested performance tracing
      unpack-trees: add performance tracing
      unpack-trees: optimize walking same trees with cache-tree
      unpack-trees: reduce malloc in cache-tree walk
      unpack-trees: reuse (still valid) cache-tree from src_index
      unpack-trees: add missing cache invalidation
      cache-tree: verify valid cache-tree in the test suite
      Document update for nd/unpack-trees-with-cache-tree
      bisect.c: make show_list() build again
      t/helper: keep test-tool command list sorted
      t/helper: merge test-dump-untracked-cache into test-tool
      t/helper: merge test-pkt-line into test-tool
      t/helper: merge test-parse-options into test-tool
      t/helper: merge test-dump-fsmonitor into test-tool
      Makefile: add a hint about TEST_BUILTINS_OBJS
      config.txt: follow camelCase naming
      config.txt: move fetch part out to a separate file
      config.txt: move format part out to a separate file
      config.txt: move gitcvs part out to a separate file
      config.txt: move gui part out to a separate file
      config.txt: move pull part out to a separate file
      config.txt: move push part out to a separate file
      config.txt: move receive part out to a separate file
      config.txt: move sendemail part out to a separate file
      config.txt: move sequence.editor out of "core" part
      config.txt: move submodule part out to a separate file
      archive.c: remove implicit dependency the_repository
      status: show progress bar if refreshing the index takes too long
      add: do not accept pathspec magic 'attr'
      completion: support "git fetch --multiple"
      read-cache.c: remove 'const' from index_has_changes()
      diff.c: reduce implicit dependency on the_index
      combine-diff.c: remove implicit dependency on the_index
      blame.c: rename "repo" argument to "r"
      diff.c: remove the_index dependency in textconv() functions
      grep.c: remove implicit dependency on the_index
      diff.c: remove implicit dependency on the_index
      read-cache.c: remove implicit dependency on the_index
      diff-lib.c: remove implicit dependency on the_index
      ll-merge.c: remove implicit dependency on the_index
      merge-blobs.c: remove implicit dependency on the_index
      merge.c: remove implicit dependency on the_index
      patch-ids.c: remove implicit dependency on the_index
      sha1-file.c: remove implicit dependency on the_index
      rerere.c: remove implicit dependency on the_index
      userdiff.c: remove implicit dependency on the_index
      line-range.c: remove implicit dependency on the_index
      submodule.c: remove implicit dependency on the_index
      tree-diff.c: remove implicit dependency on the_index
      ws.c: remove implicit dependency on the_index
      revision.c: remove implicit dependency on the_index
      revision.c: reduce implicit dependency the_repository
      read-cache.c: optimize reading index format v4
      config.txt: correct the note about uploadpack.packObjectsHook
      help -a: improve and make --verbose default
      refs.c: indent with tabs, not spaces
      Add a place for (not) sharing stuff between worktrees
      submodule.c: remove some of the_repository references
      completion: fix __gitcomp_builtin no longer consider extra options
      t1300: extract and use test_cmp_config()
      worktree: add per-worktree config files
      refs: new ref types to make per-worktree refs visible to all worktrees
      revision.c: correct a parameter name
      revision.c: better error reporting on ref from different worktrees
      fsck: check HEAD and reflog from other worktrees
      reflog expire: cover reflog from all worktrees
      Update makefile in preparation for Documentation/config/*.txt
      config.txt: move advice.* to a separate file
      config.txt: move core.* to a separate file
      config.txt: move add.* to a separate file
      config.txt: move alias.* to a separate file
      config.txt: move am.* to a separate file
      config.txt: move apply.* to a separate file
      config.txt: move blame.* to a separate file
      config.txt: move branch.* to a separate file
      config.txt: move browser.* to a separate file
      config.txt: move checkout.* to a separate file
      config.txt: move clean.* to a separate file
      config.txt: move color.* to a separate file
      config.txt: move column.* to a separate file
      config.txt: move commit.* to a separate file
      config.txt: move credential.* to a separate file
      config.txt: move completion.* to a separate file
      config.txt: move diff-config.txt to config/
      config.txt: move difftool.* to a separate file
      config.txt: move fastimport.* to a separate file
      config.txt: move fetch-config.txt to config/
      config.txt: move filter.* to a separate file
      config.txt: move format-config.txt to config/
      config.txt: move fmt-merge-msg-config.txt to config/
      config.txt: move fsck.* to a separate file
      config.txt: move gc.* to a separate file
      config.txt: move gitcvs-config.txt to config/
      config.txt: move gitweb.* to a separate file
      config.txt: move grep.* to a separate file
      config.txt: move gpg.* to a separate file
      config.txt: move gui-config.txt to config/
      config.txt: move guitool.* to a separate file
      config.txt: move help.* to a separate file
      config.txt: move ssh.* to a separate file
      config.txt: move http.* to a separate file
      config.txt: move i18n.* to a separate file
      git-imap-send.txt: move imap.* to a separate file
      config.txt: move index.* to a separate file
      config.txt: move init.* to a separate file
      config.txt: move instaweb.* to a separate file
      config.txt: move interactive.* to a separate file
      config.txt: move log.* to a separate file
      config.txt: move mailinfo.* to a separate file
      config.txt: move mailmap.* to a separate file
      config.txt: move man.* to a separate file
      config.txt: move merge-config.txt to config/
      config.txt: move mergetool.* to a separate file
      config.txt: move notes.* to a separate file
      config.txt: move pack.* to a separate file
      config.txt: move pager.* to a separate file
      config.txt: move pretty.* to a separate file
      config.txt: move protocol.* to a separate file
      config.txt: move pull-config.txt to config/
      config.txt: move push-config.txt to config/
      config.txt: move rebase-config.txt to config/
      config.txt: move receive-config.txt to config/
      config.txt: move remote.* to a separate file
      config.txt: move remotes.* to a separate file
      config.txt: move repack.* to a separate file
      config.txt: move rerere.* to a separate file
      config.txt: move reset.* to a separate file
      config.txt: move sendemail-config.txt to config/
      config.txt: move sequencer.* to a separate file
      config.txt: move showBranch.* to a separate file
      config.txt: move splitIndex.* to a separate file
      config.txt: move status.* to a separate file
      config.txt: move stash.* to a separate file
      config.txt: move submodule.* to a separate file
      config.txt: move tag.* to a separate file
      config.txt: move transfer.* to a separate file
      config.txt: move uploadarchive.* to a separate file
      config.txt: move uploadpack.* to a separate file
      config.txt: move url.* to a separate file
      config.txt: move user.* to a separate file
      config.txt: move versionsort.* to a separate file
      config.txt: move web.* to a separate file
      config.txt: move worktree.* to a separate file
      config.txt: remove config/dummy.txt
      thread-utils: macros to unconditionally compile pthreads API
      wildmatch: change behavior of "foo**bar" in WM_PATHNAME mode
      git-worktree.txt: correct linkgit command name
      sequencer.c: remove a stray semicolon
      tree-walk.c: fix overoptimistic inclusion in :(exclude) matching
      run-command.h: include thread-utils.h instead of pthread.h
      send-pack.c: move async's #ifdef NO_PTHREADS back to run-command.c
      index-pack: remove #ifdef NO_PTHREADS
      name-hash.c: remove #ifdef NO_PTHREADS
      attr.c: remove #ifdef NO_PTHREADS
      grep: remove #ifdef NO_PTHREADS
      grep: clean up num_threads handling
      preload-index.c: remove #ifdef NO_PTHREADS
      pack-objects: remove #ifdef NO_PTHREADS
      read-cache.c: remove #ifdef NO_PTHREADS
      read-cache.c: reduce branching based on HAVE_THREADS
      read-cache.c: initialize copy_len to shut up gcc 8
      Clean up pthread_create() error handling
      completion: use __gitcomp_builtin for format-patch
      build: fix broken command-list.h generation with core.autocrlf
      format-patch: respect --stat in cover letter's diffstat
      doc: move extensions.worktreeConfig to the right place
      clone: fix colliding file detection on APFS

Nickolai Belakovski (2):
      worktree: update documentation for lock_reason and lock_reason_valid
      worktree: rename is_worktree_locked to worktree_lock_reason

Noam Postavsky (1):
      log: fix coloring of certain octopus merge shapes

Olga Telezhnaya (3):
      ref-filter: free memory from used_atom
      ls-remote: release memory instead of UNLEAK
      ref-filter: free item->value and item->value->s

Phillip Wood (11):
      diff: fix --color-moved-ws=allow-indentation-change
      diff --color-moved-ws: fix double free crash
      diff --color-moved-ws: fix out of bounds string access
      diff --color-moved-ws: fix a memory leak
      diff --color-moved-ws: fix another memory leak
      diff --color-moved: fix a memory leak
      am: don't die in read_author_script()
      am: improve author-script error reporting
      am: rename read_author_script()
      add read_author_script() to libgit
      sequencer: use read_author_script()

Pratik Karki (46):
      rebase: start implementing it as a builtin
      rebase: refactor common shell functions into their own file
      builtin/rebase: support running "git rebase <upstream>"
      builtin rebase: support --onto
      builtin rebase: support `git rebase --onto A...B`
      builtin rebase: handle the pre-rebase hook and --no-verify
      builtin rebase: support --quiet
      builtin rebase: support the `verbose` and `diffstat` options
      builtin rebase: require a clean worktree
      builtin rebase: try to fast forward when possible
      builtin rebase: support --force-rebase
      builtin rebase: start a new rebase only if none is in progress
      builtin rebase: only store fully-qualified refs in `options.head_name`
      builtin rebase: support `git rebase <upstream> <switch-to>`
      builtin rebase: support --continue
      builtin rebase: support --skip
      builtin rebase: support --abort
      builtin rebase: support --quit
      builtin rebase: support --edit-todo and --show-current-patch
      builtin rebase: actions require a rebase in progress
      builtin rebase: stop if `git am` is in progress
      builtin rebase: allow selecting the rebase "backend"
      builtin rebase: support --signoff
      builtin rebase: support --rerere-autoupdate
      builtin rebase: support --committer-date-is-author-date
      builtin rebase: support `ignore-whitespace` option
      builtin rebase: support `ignore-date` option
      builtin rebase: support `keep-empty` option
      builtin rebase: support `--autosquash`
      builtin rebase: support `--gpg-sign` option
      builtin rebase: support `-C` and `--whitespace=<type>`
      builtin rebase: support `--autostash` option
      builtin rebase: support `--exec`
      builtin rebase: support `--allow-empty-message` option
      builtin rebase: support --rebase-merges[=[no-]rebase-cousins]
      merge-base --fork-point: extract libified function
      builtin rebase: support `fork-point` option
      builtin rebase: add support for custom merge strategies
      builtin rebase: support --root
      builtin rebase: optionally auto-detect the upstream
      builtin rebase: optionally pass custom reflogs to reset_head()
      builtin rebase: fast-forward to onto if it is a proper descendant
      builtin rebase: show progress when connected to a terminal
      builtin rebase: use no-op editor when interactive is "implied"
      builtin rebase: error out on incompatible option/mode combinations
      rebase: default to using the builtin rebase

Rafael Ascensão (2):
      refs: show --exclude failure with --branches/tags/remotes=glob
      refs: fix some exclude patterns being ignored

Ralf Thielow (1):
      git-rebase.sh: fix typos in error messages

Ramsay Jones (12):
      Makefile: add a hdr-check target
      json-writer.h: add missing include (hdr-check)
      ewah/ewok_rlw.h: add missing include (hdr-check)
      refs/ref-cache.h: add missing declarations (hdr-check)
      refs/packed-backend.h: add missing declaration (hdr-check)
      refs/refs-internal.h: add missing declarations (hdr-check)
      midx.h: add missing forward declarations (hdr-check)
      delta-islands.h: add missing forward declarations (hdr-check)
      headers: normalize the spelling of some header guards
      fetch-object.h: add missing declaration (hdr-check)
      ewok_rlw.h: add missing 'inline' to function definition
      commit-reach.h: add missing declarations (hdr-check)

Rasmus Villemoes (6):
      help: redirect to aliased commands for "git cmd --help"
      git.c: handle_alias: prepend alias info when first argument is -h
      git-help.txt: document "git help cmd" vs "git cmd --help" for aliases
      Documentation/git-send-email.txt: style fixes
      send-email: only consider lines containing @ or <> for automatic Cc'ing
      send-email: also pick up cc addresses from -by trailers

René Scharfe (12):
      mailinfo: support format=flowed
      fsck: add a performance test for skipList
      fsck: use strbuf_getline() to read skiplist file
      fsck: use oidset instead of oid_array for skipList
      sequencer: use return value of oidset_insert()
      grep: add -r/--[no-]recursive
      fetch-pack: factor out is_unmatched_ref()
      fetch-pack: load tip_oids eagerly iff needed
      khash: factor out kh_release_*
      oidset: use khash
      oidset: uninline oidset_init()
      commit-reach: fix cast in compare_commits_by_gen()

Roger Strain (1):
      subtree: performance improvement for finding unexpected parent commits

SZEDER Gábor (17):
      t1404: increase core.packedRefsTimeout to avoid occasional test failure
      Documentation/git.txt: clarify that GIT_TRACE=/path appends
      t3701-add-interactive: tighten the check of trace output
      t1700-split-index: drop unnecessary 'grep'
      t0090: disable GIT_TEST_SPLIT_INDEX for the test checking split index
      t1700-split-index: document why FSMONITOR is disabled in this test script
      split-index: add tests to demonstrate the racy split index problem
      t1700-split-index: date back files to avoid racy situations
      split-index: count the number of deleted entries
      split-index: don't compare cached data of entries already marked for split index
      split-index: smudge and add racily clean cache entries to split index
      split-index: BUG() when cache entry refers to non-existing shared entry
      object_id.cocci: match only expressions of type 'struct object_id'
      test-lib: introduce the '-V' short option for '--verbose-log'
      travis-ci: install packages in 'ci/install-dependencies.sh'
      coccicheck: introduce 'pending' semantic patches
      ref-filter: don't look for objects when outside of a repository

Sam McKelvie (1):
      rev-parse: --show-superproject-working-tree should work during a merge

Saulius Gurklys (1):
      doc: fix small typo in git show-branch

Sebastian Staudt (1):
      travis-ci: no longer use containers

Shulhan (1):
      builtin/remote: quote remote name on error to display empty name

Stefan Beller (25):
      git-submodule.sh: align error reporting for update mode to use path
      git-submodule.sh: rename unused variables
      builtin/submodule--helper: factor out submodule updating
      builtin/submodule--helper: store update_clone information in a struct
      builtin/submodule--helper: factor out method to update a single submodule
      submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree
      submodule--helper: introduce new update-module-mode helper
      test_decode_color: understand FAINT and ITALIC
      t3206: add color test for range-diff --dual-color
      diff.c: simplify caller of emit_line_0
      diff.c: reorder arguments for emit_line_ws_markup
      diff.c: add set_sign to emit_line_0
      diff: use emit_line_0 once per line
      diff.c: omit check for line prefix in emit_line_0
      diff.c: rewrite emit_line_0 more understandably
      diff.c: add --output-indicator-{new, old, context}
      range-diff: make use of different output indicators
      range-diff: indent special lines as context
      refs.c: migrate internal ref iteration to pass thru repository argument
      refs.c: upgrade for_each_replace_ref to be a each_repo_ref_fn callback
      string-list: remove unused function print_string_list
      strbuf.h: format according to coding guidelines
      diff.c: pass sign_index to emit_line_ws_markup
      submodule helper: convert relative URL to absolute URL if needed
      builtin/submodule--helper: remove debugging leftover tracing

Stephen P. Smith (10):
      wt-status.c: move has_unmerged earlier in the file
      wt-status: rename commitable to committable
      t7501: add test of "commit --dry-run --short"
      wt-status.c: set the committable flag in the collect phase
      roll wt_status_state into wt_status and populate in the collect phase
      t2000: rename and combine checkout clash tests
      t7509: cleanup description and filename
      t7502: rename commit test script to comply with naming convention
      t7500: rename commit tests script to comply with naming convention
      t7501: rename commit test to comply with naming convention

Steve Hoelzer (1):
      poll: use GetTickCount64() to avoid wrap-around issues

Steven Fernandez (1):
      git-completion.bash: add completion for stash list

Strain, Roger L (4):
      subtree: refactor split of a commit into standalone method
      subtree: make --ignore-joins pay attention to adds
      subtree: use commits before rejoins for splits
      subtree: improve decision on merges kept in split

Tao Qingyun (3):
      refs: docstring typo
      builtin/branch.c: remove useless branch_get
      branch: trivial style fix

Taylor Blau (4):
      transport.c: extract 'fill_alternate_refs_command'
      transport.c: introduce core.alternateRefsCommand
      transport.c: introduce core.alternateRefsPrefixes
      Documentation/config.txt: fix typo in core.alternateRefsCommand

Thomas Gummerer (17):
      rerere: unify error messages when read_cache fails
      rerere: lowercase error messages
      rerere: wrap paths in output in sq
      rerere: mark strings for translation
      rerere: add documentation for conflict normalization
      rerere: fix crash with files rerere can't handle
      rerere: only return whether a path has conflicts or not
      rerere: factor out handle_conflict function
      rerere: return strbuf from handle path
      rerere: teach rerere to handle nested conflicts
      rerere: recalculate conflict ID when unresolved conflict is committed
      rerere: mention caveat about unmatched conflict markers
      rerere: add note about files with existing conflict markers
      .gitattributes: add conflict-marker-size for relevant files
      linear-assignment: fix potential out of bounds memory access
      t5551: move setup code inside test_expect blocks
      t5551: compare sorted cookies files

Tim Schumacher (4):
      Documentation/Makefile: make manpage-base-url.xsl generation quieter
      alias: add support for aliases of an alias
      alias: show the call history when an alias is looping
      t0014: introduce an alias testing suite

Todd Zullinger (1):
      Documentation: build technical/multi-pack-index

Torsten Bögershausen (4):
      Make git_check_attr() a void function
      path.c: char is not (always) signed
      Upcast size_t variables to uintmax_t when printing
      remote-curl.c: xcurl_off_t is not portable (on 32 bit platfoms)

Uwe Kleine-König (1):
      howto/using-merge-subtree: mention --allow-unrelated-histories

brian m. carlson (26):
      t: add test functions to translate hash-related values
      t0000: use hash translation table
      t0000: update tests for SHA-256
      t0002: abstract away SHA-1 specific constants
      t0064: make hash size independent
      t1006: make hash size independent
      t1400: switch hard-coded object ID to variable
      t1405: make hash size independent
      t1406: make hash-size independent
      t1407: make hash size independent
      editorconfig: provide editor settings for Git developers
      editorconfig: indicate settings should be kept in sync
      pack-bitmap-write: use GIT_MAX_RAWSZ for allocation
      builtin/repack: replace hard-coded constants
      builtin/mktree: remove hard-coded constant
      builtin/fetch-pack: remove constants with parse_oid_hex
      pack-revindex: express constants in terms of the_hash_algo
      packfile: express constants in terms of the_hash_algo
      refs/packed-backend: express constants using the_hash_algo
      upload-pack: express constants in terms of the_hash_algo
      transport: use parse_oid_hex instead of a constant
      tag: express constant in terms of the_hash_algo
      apply: replace hard-coded constants
      apply: rename new_sha1_prefix and old_sha1_prefix
      submodule: make zero-oid comparison hash function agnostic
      rerere: convert to use the_hash_algo

Ævar Arnfjörð Bjarmason (33):
      fetch: change "branch" to "reference" in --force -h output
      push tests: make use of unused $1 in test description
      push tests: use spaces in interpolated string
      fetch tests: add a test for clobbering tag behavior
      push doc: remove confusing mention of remote merger
      push doc: move mention of "tag <tag>" later in the prose
      push doc: correct lies about how push refspecs work
      fetch: document local ref updates with/without --force
      fetch: stop clobbering existing tags without --force
      fsck tests: setup of bogus commit object
      fsck tests: add a test for no skipList input
      fsck: document and test sorted skipList input
      fsck: document and test commented & empty line skipList input
      fsck: document that skipList input must be unabbreviated
      fsck: add a performance test
      fsck: support comments & empty lines in skipList
      commit-graph write: add progress output
      commit-graph verify: add progress output
      config doc: add missing list separator for checkout.optimizeNewBranch
      push doc: add spacing between two words
      fetch doc: correct grammar in --force docs
      gc: fix regression in 7b0f229222 impacting --quiet
      gc doc: mention the commit-graph in the intro
      pack-objects test: modernize style
      pack-objects tests: don't leave test .git corrupt at end
      index-pack tests: don't leave test repo dirty at end
      i18n: make GETTEXT_POISON a runtime option
      range-diff doc: add a section about output stability
      range-diff: fix regression in passing along diff options
      range-diff: make diff option behavior (e.g. --stat) consistent
      rebase doc: document rebase.useBuiltin
      tests: add a special setup where rebase.useBuiltin is off
      read-cache: make the split index obey umask settings

Đoàn Trần Công Danh (1):
      git-compat-util: prefer poll.h to sys/poll.h

Re: [ANNOUNCE] Git v2.20.0-rc1

[Ævar Arnfjörð Bjarmason]

On Wed, Nov 21 2018, Junio C Hamano wrote:

>  * The "--no-patch" option, which can be used to get a high-level
>    overview without the actual line-by-line patch difference shown, of
>    the "range-diff" command was earlier broken, which has been
>    corrected.

There's a regression related to this that I wanted to send a headsup
for, but don't have time to fix today. Now range-diff in format-patch
includes --stat output. See e.g. my
https://public-inbox.org/git/20181122132823.9883-1-avarab@gmail.com/

Preliminary patch:

 builtin/log.c         | 3 +++
 t/t3206-range-diff.sh | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/builtin/log.c b/builtin/log.c
index 0fe6f9ba1e..fdaba480d2 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1094,9 +1094,12 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}

 	if (rev->rdiff1) {
+		const int oldfmt = rev->diffopt.output_format;
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
+		rev->diffopt.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
 				rev->creation_factor, 1, &rev->diffopt);
+		rev->diffopt.output_format = oldfmt;
 	}
 }

diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index e497c1358f..2e913542f3 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -248,8 +248,10 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
+		cat actual &&
 		git format-patch --stdout --cover-letter --range-diff=$prev \
 			master..unmodified >actual &&
+		! grep "a => b" actual &&
 		grep "= 1: .* s/5/A" actual &&
 		grep "= 2: .* s/4/A" actual &&
 		grep "= 3: .* s/11/B" actual &&

Re: [ANNOUNCE] Git v2.20.0-rc1

[Eric Sunshine]

On Thu, Nov 22, 2018 at 10:58 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> There's a regression related to this that I wanted to send a headsup
> for, but don't have time to fix today. Now range-diff in format-patch
> includes --stat output. See e.g. my
> https://public-inbox.org/git/20181122132823.9883-1-avarab@gmail.com/

Umf. Unfortunate fallout from [1].

> diff --git a/builtin/log.c b/builtin/log.c
> @@ -1094,9 +1094,12 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
>         if (rev->rdiff1) {
> +               const int oldfmt = rev->diffopt.output_format;
>                 fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
> +               rev->diffopt.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
>                 show_range_diff(rev->rdiff1, rev->rdiff2,
>                                 rev->creation_factor, 1, &rev->diffopt);
> +               rev->diffopt.output_format = oldfmt;
>         }
>  }

A few questions/observations:

Does this same "fix" need to be applied to the --interdiff case just
above this --range-diff block?

Does the same "fix" need to be applied to the --interdiff and
--range-diff cases in log-tree.c:show_log()?

Aside from fixing the broken --no-patches option[2], a goal of the
series was to some day make --stat do something useful. Doesn't this
"fix" go against that goal?

The way this change needs to be spread around at various locations is
making it feel like a BandAid "fix" rather than a proper solution. It
seems like it should be fixed at a different level, though I'm not
sure yet if that level is higher (where the options get set) or lower
(where they actually affect the operation).

Until we figure out the answers to these questions, I wonder if a more
sensible short-term solution would be to back out [1] and just keep
[2], which fixed the --no-patches regression.

> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> @@ -248,8 +248,10 @@ test_expect_success 'dual-coloring' '
>  for prev in topic master..topic
>  do
>         test_expect_success "format-patch --range-diff=$prev" '
> +               cat actual &&

Debugging session gunk?

>                 git format-patch --stdout --cover-letter --range-diff=$prev \
>                         master..unmodified >actual &&
> +               ! grep "a => b" actual &&
>                 grep "= 1: .* s/5/A" actual &&

[1]: https://public-inbox.org/git/20181113185558.23438-4-avarab@gmail.com/
[2]: https://public-inbox.org/git/20181113185558.23438-3-avarab@gmail.com/

[PATCH 0/2] format-patch: pre-2.20 range-diff regression fix

[Ævar Arnfjörð Bjarmason]

[Dropping LKML & git-packagers from CC]

On Thu, Nov 22 2018, Eric Sunshine wrote:

> On Thu, Nov 22, 2018 at 10:58 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> There's a regression related to this that I wanted to send a headsup
>> for, but don't have time to fix today. Now range-diff in format-patch
>> includes --stat output. See e.g. my
>> https://public-inbox.org/git/20181122132823.9883-1-avarab@gmail.com/
>
> Umf. Unfortunate fallout from [1].
>
>> diff --git a/builtin/log.c b/builtin/log.c
>> @@ -1094,9 +1094,12 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
>>         if (rev->rdiff1) {
>> +               const int oldfmt = rev->diffopt.output_format;
>>                 fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
>> +               rev->diffopt.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
>>                 show_range_diff(rev->rdiff1, rev->rdiff2,
>>                                 rev->creation_factor, 1, &rev->diffopt);
>> +               rev->diffopt.output_format = oldfmt;
>>         }
>>  }
>
> A few questions/observations:
>
> Does this same "fix" need to be applied to the --interdiff case just
> above this --range-diff block?
>
> Does the same "fix" need to be applied to the --interdiff and
> --range-diff cases in log-tree.c:show_log()?

No, that seems to do the right thing, but perhaps tests are lacking
there too. I haven't looked.

> Aside from fixing the broken --no-patches option[2], a goal of the
> series was to some day make --stat do something useful. Doesn't this
> "fix" go against that goal?

The goal was to fix the regression introduced in 73a834e9e2
("range-diff: relieve callers of low-level configuration burden",
2018-07-22). One aspect of having fixed that is we might in the future
do stuff with --stat.

> The way this change needs to be spread around at various locations is
> making it feel like a BandAid "fix" rather than a proper solution. It
> seems like it should be fixed at a different level, though I'm not
> sure yet if that level is higher (where the options get set) or lower
> (where they actually affect the operation).
>
> Until we figure out the answers to these questions, I wonder if a more
> sensible short-term solution would be to back out [1] and just keep
> [2], which fixed the --no-patches regression.

I think that patch leaves range-diff itself in a good state without
any bugs, and it would be a mistake to revert it.

But this interaction with format-patch --range-diff is another
matter. As noted in 2/2 I think in practice this series sweeps the
current bugs under the rug.

But as also noted there I think re-using what we get from
setup_revisions() in format-patch for the range-diff was a mistake,
and is always going to lead to some confusion. It should be split up
so we can supply those diff options independently.

> [...]
> [1]: https://public-inbox.org/git/20181113185558.23438-4-avarab@gmail.com/
> [2]: https://public-inbox.org/git/20181113185558.23438-3-avarab@gmail.com/

Ævar Arnfjörð Bjarmason (2):
  format-patch: add a more exhaustive --range-diff test
  format-patch: don't include --stat with --range-diff output

 builtin/log.c         |  7 ++++++-
 t/t3206-range-diff.sh | 15 ++++++++++-----
 2 files changed, 16 insertions(+), 6 deletions(-)

-- 
2.20.0.rc0.387.gc7a69e6b6c

[PATCH 1/2] format-patch: add a more exhaustive --range-diff test

[Ævar Arnfjörð Bjarmason]

Change the narrow test added in 31e2617a5f ("format-patch: add
--range-diff option to embed diff in cover letter", 2018-07-22) to
test the full output. This test would have spotted a regression in the
output if it wasn't beating around the bush and tested the full
output, let's do that.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t3206-range-diff.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)

diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index e497c1358f..0235c038be 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -249,11 +249,28 @@ for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
 		git format-patch --stdout --cover-letter --range-diff=$prev \
-			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+			master..unmodified >actual.raw &&
+		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
+		:1:  4de457d = 1:  35b9b25 s/5/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:2:  fccce22 = 2:  de345ab s/4/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:3:  147e64e = 3:  9af6654 s/11/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:4:  a63e992 = 4:  2901f77 s/12/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:-- :
+		EOF
+		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
+		test_cmp expect actual
 	'
 done
 
-- 
2.20.0.rc0.387.gc7a69e6b6c

[PATCH 2/2] format-patch: don't include --stat with --range-diff output

[Ævar Arnfjörð Bjarmason]

Fix a regression introduced in my a48e12ef7a ("range-diff: make diff
option behavior (e.g. --stat) consistent", 2018-11-13). Since the
format-patch setup code implicitly sets --stat --summary by default,
we started emitting the --stat output in the cover letter's
range-diff.

As noted in df569c3f31 ("range-diff doc: add a section about output
stability", 2018-11-09) the --stat output is currently rather useless,
and just adds noise.

Perhaps we should detect if --stat or --summary were implicitly passed
to format-patch, and then pass them along, but I think fixing it this
way is fine. If our --stat output ever becomes useful in range-diff we
can revisit this.

There's still cases like e.g. --numstat triggering rather useless
range-diff output, but I think it's OK to just handle the default
case. Users are unlikely to produce a formatted patch with the likes
of --numstat, or indeed any other custom diff option except -U<n> or
maybe -W. If they need weirder combinations of options they can always
manually produce the range-diff.

This whole situation comes about because we're assuming that when the
user passes along e.g. -U10 that they want that some 10-line context
for the range-diff as for the patches themselves. As noted in [1] I
think it's worth re-visiting this and making -U10 just apply to the
patches, and e.g. --range-diff-U10 to the range-diff. But that's left
as a topic for another series less close to a rc2.

1. https://public-inbox.org/git/87d0ri7gbs.fsf@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/log.c         |  7 ++++++-
 t/t3206-range-diff.sh | 12 ------------
 2 files changed, 6 insertions(+), 13 deletions(-)

diff --git a/builtin/log.c b/builtin/log.c
index 0fe6f9ba1e..7cd2db0be9 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1094,9 +1094,13 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
+		struct diff_options opts;
+		memcpy(&opts, &rev->diffopt, sizeof(opts));
+		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
+
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &rev->diffopt);
+				rev->creation_factor, 1, &opts);
 	}
 }
 
@@ -1697,6 +1701,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
+		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index 0235c038be..90def330bd 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -252,21 +252,9 @@ do
 			master..unmodified >actual.raw &&
 		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
 		:1:  4de457d = 1:  35b9b25 s/5/A/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:2:  fccce22 = 2:  de345ab s/4/A/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:3:  147e64e = 3:  9af6654 s/11/B/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:4:  a63e992 = 4:  2901f77 s/12/B/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:-- :
 		EOF
 		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
-- 
2.20.0.rc0.387.gc7a69e6b6c

Re: [PATCH 2/2] format-patch: don't include --stat with --range-diff output

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

>  	if (rev->rdiff1) {
> +		struct diff_options opts;
> +		memcpy(&opts, &rev->diffopt, sizeof(opts));
> +		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
> +
>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
>  		show_range_diff(rev->rdiff1, rev->rdiff2,
> -				rev->creation_factor, 1, &rev->diffopt);
> +				rev->creation_factor, 1, &opts);

I am not quite convinced if this shallow copy is a safe thing to do.
Quite honestly at this late in the release cycle, as a band-aid, I'd
rather see a simpler revert than a change like this that we have to
worry about what happens if/when show_range_diff() _thinks_ it is
done with the opts and ends up discarding resources (e.g. "FILE *")
that are shared with rev->diffopt that would still have to be used
later.

Re: [PATCH 1/2] format-patch: add a more exhaustive --range-diff test

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Change the narrow test added in 31e2617a5f ("format-patch: add
> --range-diff option to embed diff in cover letter", 2018-07-22) to
> test the full output. This test would have spotted a regression in the
> output if it wasn't beating around the bush and tested the full
> output, let's do that.

This looks wrong, given that we are declaring that showing the
broken --stat in the output is wrong.  It makes us to expect a
known-wrong output from the command.

If the theme of the topic were "what we are giving by default is a
sensible output when --stat is asked for, but it is rather too noisy
and our default should be quieter---let's tone it down", then this
patch in 1/2 and then a behaviour-change patch with updated
expectation would make sense, but I do not think that is what you
are aiming for with this series.

Perhaps squash this into the real "fix" to show what the desired
output should look like with the same patch?

> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  t/t3206-range-diff.sh | 27 ++++++++++++++++++++++-----
>  1 file changed, 22 insertions(+), 5 deletions(-)
>
> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> index e497c1358f..0235c038be 100755
> --- a/t/t3206-range-diff.sh
> +++ b/t/t3206-range-diff.sh
> @@ -249,11 +249,28 @@ for prev in topic master..topic
>  do
>  	test_expect_success "format-patch --range-diff=$prev" '
>  		git format-patch --stdout --cover-letter --range-diff=$prev \
> -			master..unmodified >actual &&
> -		grep "= 1: .* s/5/A" actual &&
> -		grep "= 2: .* s/4/A" actual &&
> -		grep "= 3: .* s/11/B" actual &&
> -		grep "= 4: .* s/12/B" actual
> +			master..unmodified >actual.raw &&
> +		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
> +		:1:  4de457d = 1:  35b9b25 s/5/A/
> +		:     a => b | 0
> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
> +		:    :
> +		:2:  fccce22 = 2:  de345ab s/4/A/
> +		:     a => b | 0
> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
> +		:    :
> +		:3:  147e64e = 3:  9af6654 s/11/B/
> +		:     a => b | 0
> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
> +		:    :
> +		:4:  a63e992 = 4:  2901f77 s/12/B/
> +		:     a => b | 0
> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
> +		:    :
> +		:-- :
> +		EOF
> +		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
> +		test_cmp expect actual
>  	'
>  done

Re: [PATCH 2/2] format-patch: don't include --stat with --range-diff output

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>>  	if (rev->rdiff1) {
>> +		struct diff_options opts;
>> +		memcpy(&opts, &rev->diffopt, sizeof(opts));
>> +		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
>> +
>>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
>>  		show_range_diff(rev->rdiff1, rev->rdiff2,
>> -				rev->creation_factor, 1, &rev->diffopt);
>> +				rev->creation_factor, 1, &opts);
>
> I am not quite convinced if this shallow copy is a safe thing to do.
> Quite honestly at this late in the release cycle, as a band-aid, I'd
> rather see a simpler revert than a change like this that we have to
> worry about what happens if/when show_range_diff() _thinks_ it is
> done with the opts and ends up discarding resources (e.g. "FILE *")
> that are shared with rev->diffopt that would still have to be used
> later.

Well, let's take it anyway as-is, at least for today, as I notice
show_range_diff() itself does another shallow copy, so we are not
making anything dramatically worse.

Thanks.

Re: [PATCH 1/2] format-patch: add a more exhaustive --range-diff test

[Ævar Arnfjörð Bjarmason]

On Sat, Nov 24 2018, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> Change the narrow test added in 31e2617a5f ("format-patch: add
>> --range-diff option to embed diff in cover letter", 2018-07-22) to
>> test the full output. This test would have spotted a regression in the
>> output if it wasn't beating around the bush and tested the full
>> output, let's do that.
>
> This looks wrong, given that we are declaring that showing the
> broken --stat in the output is wrong.  It makes us to expect a
> known-wrong output from the command.
>
> If the theme of the topic were "what we are giving by default is a
> sensible output when --stat is asked for, but it is rather too noisy
> and our default should be quieter---let's tone it down", then this
> patch in 1/2 and then a behaviour-change patch with updated
> expectation would make sense, but I do not think that is what you
> are aiming for with this series.
>
> Perhaps squash this into the real "fix" to show what the desired
> output should look like with the same patch?

I see you did that in 'pu'. I don't mind, looks good to me.

FWIW I split this up for ease of review, i.e. showing what the output
was before and what effect the code change had, but maybe that was
overdoing it and it's simpler just to have this all in one go.

>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>>  t/t3206-range-diff.sh | 27 ++++++++++++++++++++++-----
>>  1 file changed, 22 insertions(+), 5 deletions(-)
>>
>> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
>> index e497c1358f..0235c038be 100755
>> --- a/t/t3206-range-diff.sh
>> +++ b/t/t3206-range-diff.sh
>> @@ -249,11 +249,28 @@ for prev in topic master..topic
>>  do
>>  	test_expect_success "format-patch --range-diff=$prev" '
>>  		git format-patch --stdout --cover-letter --range-diff=$prev \
>> -			master..unmodified >actual &&
>> -		grep "= 1: .* s/5/A" actual &&
>> -		grep "= 2: .* s/4/A" actual &&
>> -		grep "= 3: .* s/11/B" actual &&
>> -		grep "= 4: .* s/12/B" actual
>> +			master..unmodified >actual.raw &&
>> +		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
>> +		:1:  4de457d = 1:  35b9b25 s/5/A/
>> +		:     a => b | 0
>> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
>> +		:    :
>> +		:2:  fccce22 = 2:  de345ab s/4/A/
>> +		:     a => b | 0
>> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
>> +		:    :
>> +		:3:  147e64e = 3:  9af6654 s/11/B/
>> +		:     a => b | 0
>> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
>> +		:    :
>> +		:4:  a63e992 = 4:  2901f77 s/12/B/
>> +		:     a => b | 0
>> +		:     1 file changed, 0 insertions(+), 0 deletions(-)
>> +		:    :
>> +		:-- :
>> +		EOF
>> +		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
>> +		test_cmp expect actual
>>  	'
>>  done

Re: [ANNOUNCE] Git v2.20.0-rc1

[Ævar Arnfjörð Bjarmason]

On Wed, Nov 21 2018, Junio C Hamano wrote:

>  * "git rebase" and "git rebase -i" have been reimplemented in C.

Here's another regression in the C version (and rc1), note: the
sha1collisiondetection is just a stand in for "some repo":

    (
        rm -rf /tmp/repo &&
        git init /tmp/repo &&
        cd /tmp/repo &&
        for c in 1 2
        do
            touch $c &&
            git add $c &&
            git commit -m"add $c"
        done &&
        git remote add origin https://github.com/cr-marcstevens/sha1collisiondetection.git &&
        git fetch &&
        git branch --set-upstream-to origin/master &&
        git rebase -i
    )

The C version will die with "fatal: unable to read tree
0000000000000000000000000000000000000000". Running this with
rebase.useBuiltin=false does the right thing and rebases as of the merge
base of the two (which here is the root of the history).

I wasn't trying to stress test rebase. I was just wanting to rebase a
history I was about to force-push after cleaning it up, hardly an
obscure use-case. So [repeat last transmission in
https://public-inbox.org/git/87y39w1wc2.fsf@evledraar.gmail.com/ ]

Re: [ANNOUNCE] Git v2.20.0-rc1

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>>  * "git rebase" and "git rebase -i" have been reimplemented in C.
>
> Here's another regression in the C version (and rc1),...
> I wasn't trying to stress test rebase. I was just wanting to rebase a
> history I was about to force-push after cleaning it up, hardly an
> obscure use-case. So [repeat last transmission in
> https://public-inbox.org/git/87y39w1wc2.fsf@evledraar.gmail.com/ ]

which, to those who are reading from sidelines:

    Given that we're still finding regressions bugs in the rebase-in-C
    version should we be considering reverting 5541bd5b8f ("rebase: default
    to using the builtin rebase", 2018-08-08)?

    I love the feature, but fear that the current list of known regressions
    serve as a canary for a larger list which we'd discover if we held off
    for another major release (and would re-enable rebase.useBuiltin=true in
    master right after 2.20 is out the door).

I am fine with the proposed flip, but I'll have to see the extent of
damage this late in the game so that I won't miss anything.  In
addition to the one-liner below, we'd need to update the quoted
release notes entry, and possibly adjust some tests (even though the
"reimplementation" ought to be bug-to-bug compatible, it may not be).

diff --git b/builtin/rebase.c a/builtin/rebase.c
index 9dc8475cd3..60e357c735 100644
--- b/builtin/rebase.c
+++ a/builtin/rebase.c
@@ -54,7 +54,7 @@ static int use_builtin_rebase(void)
 	cp.git_cmd = 1;
 	if (capture_command(&cp, &out, 6)) {
 		strbuf_release(&out);
-		return 1;
+		return 0;
 	}
 
 	strbuf_trim(&out);

[PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>>>  * "git rebase" and "git rebase -i" have been reimplemented in C.
>>
>> Here's another regression in the C version (and rc1),...
>> I wasn't trying to stress test rebase. I was just wanting to rebase a
>> history I was about to force-push after cleaning it up, hardly an
>> obscure use-case. So [repeat last transmission in
>> https://public-inbox.org/git/87y39w1wc2.fsf@evledraar.gmail.com/ ]
>
> which, to those who are reading from sidelines:
>
>     Given that we're still finding regressions bugs in the rebase-in-C
>     version should we be considering reverting 5541bd5b8f ("rebase: default
>     to using the builtin rebase", 2018-08-08)?
>
>     I love the feature, but fear that the current list of known regressions
>     serve as a canary for a larger list which we'd discover if we held off
>     for another major release (and would re-enable rebase.useBuiltin=true in
>     master right after 2.20 is out the door).
>
> I am fine with the proposed flip, but I'll have to see the extent of
> damage this late in the game so that I won't miss anything.  In
> addition to the one-liner below, we'd need to update the quoted
> release notes entry, and possibly adjust some tests (even though the
> "reimplementation" ought to be bug-to-bug compatible, it may not be).

So, in a more concrete form, what you want to see is something like
this in -rc2 and later?

-- >8 --
Subject: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature

It turns out to be a bit too early to unleash the reimplementation
to the general public.  Let's rewrite some documentation and make it
an opt-in feature.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/config/rebase.txt | 16 ++++++----------
 builtin/rebase.c                |  2 +-
 t/README                        |  4 ++--
 3 files changed, 9 insertions(+), 13 deletions(-)

diff --git a/Documentation/config/rebase.txt b/Documentation/config/rebase.txt
index f079bf6b7e..af12623151 100644
--- a/Documentation/config/rebase.txt
+++ b/Documentation/config/rebase.txt
@@ -1,16 +1,12 @@
 rebase.useBuiltin::
-	Set to `false` to use the legacy shellscript implementation of
-	linkgit:git-rebase[1]. Is `true` by default, which means use
-	the built-in rewrite of it in C.
+	Set to `true` to use the experimental reimplementation of
+	linkgit:git-rebase[1] in C.  Defaults to `false`.
 +
 The C rewrite is first included with Git version 2.20. This option
-serves an an escape hatch to re-enable the legacy version in case any
-bugs are found in the rewrite. This option and the shellscript version
-of linkgit:git-rebase[1] will be removed in some future release.
-+
-If you find some reason to set this option to `false` other than
-one-off testing you should report the behavior difference as a bug in
-git.
+allows early adopters to opt into the experimental version to find
+bugs in the rewritten version. This option and the shellscript version
+of linkgit:git-rebase[1] will be removed in some future release once
+the reimplementation becomes more stable.
 
 rebase.stat::
 	Whether to show a diffstat of what changed upstream since the last
diff --git a/builtin/rebase.c b/builtin/rebase.c
index 5b3e5baec8..19ad97b177 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -59,7 +59,7 @@ static int use_builtin_rebase(void)
 	cp.git_cmd = 1;
 	if (capture_command(&cp, &out, 6)) {
 		strbuf_release(&out);
-		return 1;
+		return 0;
 	}
 
 	strbuf_trim(&out);
diff --git a/t/README b/t/README
index 28711cc508..7e925e5fea 100644
--- a/t/README
+++ b/t/README
@@ -345,8 +345,8 @@ for the index version specified.  Can be set to any valid version
 GIT_TEST_PRELOAD_INDEX=<boolean> exercises the preload-index code path
 by overriding the minimum number of cache entries required per thread.
 
-GIT_TEST_REBASE_USE_BUILTIN=<boolean>, when false, disables the
-builtin version of git-rebase. See 'rebase.useBuiltin' in
+GIT_TEST_REBASE_USE_BUILTIN=<boolean>, when true, forces the use of
+builtin version of git-rebase in the test. See 'rebase.useBuiltin' in
 git-config(1).
 
 GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading
-- 
2.20.0-rc1

Re: [ANNOUNCE] Git v2.20.0-rc1

[Junio C Hamano]

Unless I hear otherwise in the next 24 hours, I am planning to
merge the following topics to 'master' before cutting -rc2.  Please
stop me on any of these topics.

 - jc/postpone-rebase-in-c

   This may be the most controversial.  It demotes the C
   reimplementation of "git rebase" to an experimental opt-in
   feature that can only be enabled by setting rebase.useBuiltIn
   configuration that defaults to false.

   cf. <xmqq36roz7ve.fsf_-_@gitster-ct.c.googlers.com>


 - ab/format-patch-rangediff-not-stat

   The "--rangediff" option recently added to "format-patch"
   interspersed a bogus and useless "--stat" information by mistake,
   which is being corrected.

   cf. <xmqqa7lz43d8.fsf@gitster-ct.c.googlers.com>


The following three topics I do not need help in deciding; they are
all good and will be merged before -rc2.

 - jk/t5562-perl-path-fix

   This is to invoke a perl scriptlet with "$PERL_PATH" in one of
   the new tests, instead of relying on (an incorrect) she-bang line
   in the script file.

 - tb/clone-case-smashing-warning-test

   This enables the test to see the behaviour of "git clone" after
   cloning a project that has paths that are different only in case
   on MINGW (earlier it wanted CASE_INSENSITIVE_FS prerequisite and
   in addition not on MINGW).

 - nd/per-worktree-ref-iteration

   This fixes a "return function_that_returns_void(...)" in a
   function that returns void.


Thanks.

Re: [ANNOUNCE] Git v2.20.0-rc1

[Elijah Newren]

On Sun, Nov 25, 2018 at 11:37 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Unless I hear otherwise in the next 24 hours, I am planning to
> merge the following topics to 'master' before cutting -rc2.  Please
> stop me on any of these topics.
>
>  - jc/postpone-rebase-in-c
>
>    This may be the most controversial.  It demotes the C
>    reimplementation of "git rebase" to an experimental opt-in
>    feature that can only be enabled by setting rebase.useBuiltIn
>    configuration that defaults to false.
>
>    cf. <xmqq36roz7ve.fsf_-_@gitster-ct.c.googlers.com>

If we don't set rebase.useBuiltin to false, then there is also a minor
regression in the error message printed by the built-in rebase we may
want to try to address.  I have a patch for it at
<20181122044841.20993-2-newren@gmail.com>, but I don't know if that
patch is acceptable as-is this close to a release since that'd not
give translators much time to update.

Re: [ANNOUNCE] Git v2.20.0-rc1

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1990 bytes --]

Hi Ævar,

On Sat, 24 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> On Wed, Nov 21 2018, Junio C Hamano wrote:
> 
> >  * "git rebase" and "git rebase -i" have been reimplemented in C.
> 
> Here's another regression in the C version (and rc1), note: the
> sha1collisiondetection is just a stand in for "some repo":
> 
>     (
>         rm -rf /tmp/repo &&
>         git init /tmp/repo &&
>         cd /tmp/repo &&
>         for c in 1 2
>         do
>             touch $c &&
>             git add $c &&
>             git commit -m"add $c"
>         done &&
>         git remote add origin https://github.com/cr-marcstevens/sha1collisiondetection.git &&
>         git fetch &&
>         git branch --set-upstream-to origin/master &&
>         git rebase -i
>     )
> 
> The C version will die with "fatal: unable to read tree
> 0000000000000000000000000000000000000000". Running this with
> rebase.useBuiltin=false does the right thing and rebases as of the merge
> base of the two (which here is the root of the history).

Sorry, this bug does not reproduce here:

$ git rebase -i
Successfully rebased and updated refs/heads/master.

> I wasn't trying to stress test rebase. I was just wanting to rebase a
> history I was about to force-push after cleaning it up, hardly an
> obscure use-case. So [repeat last transmission in
> https://public-inbox.org/git/87y39w1wc2.fsf@evledraar.gmail.com/ ]

Maybe you can give me the full details so that I can verify that this is
indeed a bug in the builtin C and not just a regression caused by some
random branches being merged together?

In short: please provide me with the exact URL and branch of your git.git
fork to test. Then please make sure to specify the precise revision of the
sha1collisiondetection/master rev, just in case that it matters.

Ideally, you would reduce the problem to a proper test case, say, for
t3412 (it seems that you try to rebase onto an unrelated history, so it is
*vaguely* related to "rebase-root").

Ciao,
Dscho

Re: [ANNOUNCE] Git v2.20.0-rc1

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 4736 bytes --]

Hi Ævar,

On Mon, 26 Nov 2018, Johannes Schindelin wrote:

> On Sat, 24 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
> 
> > On Wed, Nov 21 2018, Junio C Hamano wrote:
> > 
> > >  * "git rebase" and "git rebase -i" have been reimplemented in C.
> > 
> > Here's another regression in the C version (and rc1), note: the
> > sha1collisiondetection is just a stand in for "some repo":
> > 
> >     (
> >         rm -rf /tmp/repo &&
> >         git init /tmp/repo &&
> >         cd /tmp/repo &&
> >         for c in 1 2
> >         do
> >             touch $c &&
> >             git add $c &&
> >             git commit -m"add $c"
> >         done &&
> >         git remote add origin https://github.com/cr-marcstevens/sha1collisiondetection.git &&
> >         git fetch &&
> >         git branch --set-upstream-to origin/master &&
> >         git rebase -i
> >     )
> > 
> > The C version will die with "fatal: unable to read tree
> > 0000000000000000000000000000000000000000". Running this with
> > rebase.useBuiltin=false does the right thing and rebases as of the merge
> > base of the two (which here is the root of the history).
> 
> Sorry, this bug does not reproduce here:
> 
> $ git rebase -i
> Successfully rebased and updated refs/heads/master.
> 
> > I wasn't trying to stress test rebase. I was just wanting to rebase a
> > history I was about to force-push after cleaning it up, hardly an
> > obscure use-case. So [repeat last transmission in
> > https://public-inbox.org/git/87y39w1wc2.fsf@evledraar.gmail.com/ ]
> 
> Maybe you can give me the full details so that I can verify that this is
> indeed a bug in the builtin C and not just a regression caused by some
> random branches being merged together?
> 
> In short: please provide me with the exact URL and branch of your git.git
> fork to test. Then please make sure to specify the precise revision of the
> sha1collisiondetection/master rev, just in case that it matters.
> 
> Ideally, you would reduce the problem to a proper test case, say, for
> t3412 (it seems that you try to rebase onto an unrelated history, so it is
> *vaguely* related to "rebase-root").

So I was getting spooked enough by your half-complete bug report that I
did more digging (it is really quite a bit frustrating to have so little
hard evidence to go on, a wild goose chase is definitely not what I was
looking forward to after a day of fighting other fires, but you know,
built-in rebase is dear to me).

The error message you copied clearly comes from tree-walk.c, from
`fill_tree_descriptor()` (the other "unable to read tree" messages enclose
the hash in parentheses).

There are exactly 3 calls to said function in the built-in rebase/rebase
-i in the current `master`, a1598010f775 (Merge branch
'nd/per-worktree-ref-iteration', 2018-11-26):

$ git grep fill_tree_descriptor -- builtin/rebase*.c sequencer.[ch] rebase-interactive.[ch]
builtin/rebase.c:       if (!reset_hard && !fill_tree_descriptor(&desc[nr++], &head_oid)) {
builtin/rebase.c:       if (!fill_tree_descriptor(&desc[nr++], oid)) {
sequencer.c:    if (!fill_tree_descriptor(&desc, &oid)) {

The last one of these is in `do_reset()`, i.e. handling a `reset` command
which you did not ask for, as you passed `-i` to `git rebase`, not `-ir`.

The first two *both* are in `reset_head()`. The first of them uses
`head_oid`, which is read directly via `get_oid("HEAD", &head_oid)`, so if
this is all zeroes for you, then it's not rebase's fault.

The second one uses the parameter `oid` passed into `reset_head()`. The
only calls to that function that do not pass `NULL` as `oid` (which would
trigger `oid` to be replaced by `&head_oid`, i.e again not all zeroes
unless your setup is broken) are:

- in the `--abort` code path
- in the `--autostash` code path
- in the fast-forwarding code path
- just after the "First, rewinding head" message in the *non*-interactive
  rebase

None of these apply to your script snippet.

Under the assumption that you might have forgotten to talk about
rebase.autostash=true and some dirty file, I tried to augment the script
snippet accordingly, but the built-in rebase as of current `master` still
works for me, plus: reading the autostash code path, it is hard to imagine
that the `lookup_commit_reference()` would return a pointer to a commit
object whose oid is all zeroes.

In short, even a thorough study of the code (keeping in mind the few
tidbits of information provided by you) leaves me really wondering which
code you run, because it sure does not look like current `master` to me.

And if it is not `master`, then I have to ask why you keep suggesting to
turn off the built-in rebase by default in `master`.

Ciao,
Johannes

P.S.: Maybe you have a hook you forgot to mention?

Re: [ANNOUNCE] Git v2.20.0-rc1

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> If we don't set rebase.useBuiltin to false, then there is also a minor
> regression in the error message printed by the built-in rebase we may
> want to try to address.  I have a patch for it at
> <20181122044841.20993-2-newren@gmail.com>, but I don't know if that
> patch is acceptable as-is this close to a release since that'd not
> give translators much time to update.

For this particular one, I'd rather ship "rebase in C" with known
message glitch, with or without the "mark it experimental and make
it opt-in" last-time change.  Of course, turning it off by default
would let us worry about these message glitches even less, but at
this point, I'd be worried more about bugs that can affect the
actual operation and outcome recorded in the resulting history.

Re: [ANNOUNCE] Git v2.20.0-rc1

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> ...
> In short, even a thorough study of the code (keeping in mind the few
> tidbits of information provided by you) leaves me really wondering which
> code you run, because it sure does not look like current `master` to me.
>
> And if it is not `master`, then I have to ask why you keep suggesting to
> turn off the built-in rebase by default in `master`.
>
> Ciao,
> Johannes
>
> P.S.: Maybe you have a hook you forgot to mention?

Any response?  Or can I retract jc/postpone-rebase-in-c that was
prepared as a reaction to this?

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Jonathan Nieder]

Hi,

Junio C Hamano wrote:
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>>> Given that we're still finding regressions bugs in the rebase-in-C
>>> version should we be considering reverting 5541bd5b8f ("rebase: default
>>> to using the builtin rebase", 2018-08-08)?
>>>
>>> I love the feature, but fear that the current list of known regressions
>>> serve as a canary for a larger list which we'd discover if we held off
>>> for another major release (and would re-enable rebase.useBuiltin=true in
>>> master right after 2.20 is out the door).
[...]
> So, in a more concrete form, what you want to see is something like
> this in -rc2 and later?
>
> -- >8 --
> Subject: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature
>
> It turns out to be a bit too early to unleash the reimplementation
> to the general public.  Let's rewrite some documentation and make it
> an opt-in feature.
>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  Documentation/config/rebase.txt | 16 ++++++----------
>  builtin/rebase.c                |  2 +-
>  t/README                        |  4 ++--
>  3 files changed, 9 insertions(+), 13 deletions(-)

I thought I should weigh in on how this would affect Debian's and
Google's deployments.

First of all, I've looked over the revert patch carefully and it is
well written and does what it says on the tin.

At https://bugs.debian.org/914695 is a report of a test regression in
an outside project that is very likely to have been triggered by the
new faster rebase code.  The issue has not been triaged, so I don't
know yet whether it's a problem in rebase-in-c or a manifestation of a
bug in the test.

That said, Google has been running with the new rebase since ~1 month
ago when it became the default, with no issues reported by users.  As
a result, I am confident that it can cope with what most users of
"next" throw at it, which means that if we are to find more issues to
polish it better, it will need all the exposure it can get.

In the Google deployment, we will keep using rebase-in-c even if it
gets disabled by default, in order to help with that.

From the Debian point of view, it's only a matter of time before
rebase-in-c becomes the default: even if it's not the default in 2.20,
it would presumably be so in 2.21 or 2.22.  That means the community's
attention when resolving security and reliability bugs would be on the
rebase-in-c implementation.  As a result, the Debian package will most
likely enable rebase-in-c by default even if upstream disables it, in
order to increase the package's shelf life (i.e. to ease the
maintenance burden of supporting whichever version of the package ends
up in the next Debian stable).

So with either hat on, it doesn't matter whether you apply this patch
upstream.

Having two pretty different deployments end up with the same
conclusion leads me to suspect that it's best for upstream not to
apply the revert patch, unless either

  (a) we have a concrete regression to address and then try again, or
  (b) we have a test or other plan to follow before trying again.

Thanks and hope that helps,
Jonathan

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3176 bytes --]

Hi Jonathan,

On Tue, 27 Nov 2018, Jonathan Nieder wrote:

> At https://bugs.debian.org/914695 is a report of a test regression in
> an outside project that is very likely to have been triggered by the
> new faster rebase code.

From looking through that log.gz (without having a clue where the test
code lives, so I cannot say what it is supposed to do, and also: this is
the first time I hear about dgit...), it would appear that this must be a
regression in the reflog messages produced by `git rebase`.

> The issue has not been triaged, so I don't know yet whether it's a
> problem in rebase-in-c or a manifestation of a bug in the test.

It ends thusly:

-- snip --
[...]
+ git reflog
+ egrep 'debrebase new-upstream.*checkout'
+ test 1 = 0
+ t-report-failure
+ set +x
TEST FAILED
-- snap --

Which makes me think that the reflog we produce in *some* code path that
originally called `git checkout` differs from the scripted rebase's
generated reflog.

> That said, Google has been running with the new rebase since ~1 month
> ago when it became the default, with no issues reported by users.  As a
> result, I am confident that it can cope with what most users of "next"
> throw at it, which means that if we are to find more issues to polish it
> better, it will need all the exposure it can get.

Right. And there are a few weeks before the holidays, which should give me
time to fix whatever bugs are discovered (I only half mind being the only
one who fixes these bugs).

> In the Google deployment, we will keep using rebase-in-c even if it
> gets disabled by default, in order to help with that.
> 
> From the Debian point of view, it's only a matter of time before
> rebase-in-c becomes the default: even if it's not the default in 2.20,
> it would presumably be so in 2.21 or 2.22.  That means the community's
> attention when resolving security and reliability bugs would be on the
> rebase-in-c implementation.  As a result, the Debian package will most
> likely enable rebase-in-c by default even if upstream disables it, in
> order to increase the package's shelf life (i.e. to ease the
> maintenance burden of supporting whichever version of the package ends
> up in the next Debian stable).
> 
> So with either hat on, it doesn't matter whether you apply this patch
> upstream.
> 
> Having two pretty different deployments end up with the same
> conclusion leads me to suspect that it's best for upstream not to
> apply the revert patch, unless either
> 
>   (a) we have a concrete regression to address and then try again, or
>   (b) we have a test or other plan to follow before trying again.

In this instance, I am more a fan of the "let's move fast and break
things, then move even faster fixing them" approach.

Besides, the bug that Ævar discovered was a bug already in the scripted
rebase, but hidden by yet another bug (the missing error checking).

I get the pretty firm impression that the common code paths are now pretty
robust, and only lesser-exercised features may expose a bug (or
regression, as in the case of the reflogs, where one could argue that the
exact reflog message is not something we promise not to fiddle with).

Ciao,
Dscho

Re: [ANNOUNCE] Git v2.20.0-rc1

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1567 bytes --]

Hi Junio,

On Wed, 28 Nov 2018, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > ...
> > In short, even a thorough study of the code (keeping in mind the few
> > tidbits of information provided by you) leaves me really wondering which
> > code you run, because it sure does not look like current `master` to me.
> >
> > And if it is not `master`, then I have to ask why you keep suggesting to
> > turn off the built-in rebase by default in `master`.
> >
> > Ciao,
> > Johannes
> >
> > P.S.: Maybe you have a hook you forgot to mention?
> 
> Any response?  Or can I retract jc/postpone-rebase-in-c that was
> prepared as a reaction to this?

I worked with Ævar via IRC and we figured out the root cause and I
submitted https://public-inbox.org/git/pull.88.git.gitgitgadget@gmail.com/
to fix it.

Given that this is a really obscure edge case (`git rebase --stat -v -i`
onto an unrelated commit history, if you take away one of these, the bug
does not trigger), and that it was only discovered to be a bug *because*
of the built-in rebase (the scripted version had the same bug, but simply
forgot to do proper error checking), I would not think that the reported
bug is a strong argument in favor of turning off the built-in rebase by
defauly.

In other words, after understanding the bug I am even more confident than
before that the built-in rebase is actually in a pretty good shape.

I do not expect any major regressions, and if any happen: we do have that
escape hatch for corner cases while I fix those bugs.

Ciao,
Dscho

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Ævar Arnfjörð Bjarmason]

On Wed, Nov 28 2018, Johannes Schindelin wrote:

> Hi Jonathan,
>
> On Tue, 27 Nov 2018, Jonathan Nieder wrote:
>
>> At https://bugs.debian.org/914695 is a report of a test regression in
>> an outside project that is very likely to have been triggered by the
>> new faster rebase code.
>
> From looking through that log.gz (without having a clue where the test
> code lives, so I cannot say what it is supposed to do, and also: this is
> the first time I hear about dgit...), it would appear that this must be a
> regression in the reflog messages produced by `git rebase`.
>
>> The issue has not been triaged, so I don't know yet whether it's a
>> problem in rebase-in-c or a manifestation of a bug in the test.
>
> It ends thusly:
>
> -- snip --
> [...]
> + git reflog
> + egrep 'debrebase new-upstream.*checkout'
> + test 1 = 0
> + t-report-failure
> + set +x
> TEST FAILED
> -- snap --
>
> Which makes me think that the reflog we produce in *some* code path that
> originally called `git checkout` differs from the scripted rebase's
> generated reflog.
>
>> That said, Google has been running with the new rebase since ~1 month
>> ago when it became the default, with no issues reported by users.  As a
>> result, I am confident that it can cope with what most users of "next"
>> throw at it, which means that if we are to find more issues to polish it
>> better, it will need all the exposure it can get.
>
> Right. And there are a few weeks before the holidays, which should give me
> time to fix whatever bugs are discovered (I only half mind being the only
> one who fixes these bugs).
>
>> In the Google deployment, we will keep using rebase-in-c even if it
>> gets disabled by default, in order to help with that.
>>
>> From the Debian point of view, it's only a matter of time before
>> rebase-in-c becomes the default: even if it's not the default in 2.20,
>> it would presumably be so in 2.21 or 2.22.  That means the community's
>> attention when resolving security and reliability bugs would be on the
>> rebase-in-c implementation.  As a result, the Debian package will most
>> likely enable rebase-in-c by default even if upstream disables it, in
>> order to increase the package's shelf life (i.e. to ease the
>> maintenance burden of supporting whichever version of the package ends
>> up in the next Debian stable).
>>
>> So with either hat on, it doesn't matter whether you apply this patch
>> upstream.
>>
>> Having two pretty different deployments end up with the same
>> conclusion leads me to suspect that it's best for upstream not to
>> apply the revert patch, unless either
>>
>>   (a) we have a concrete regression to address and then try again, or
>>   (b) we have a test or other plan to follow before trying again.
>
> In this instance, I am more a fan of the "let's move fast and break
> things, then move even faster fixing them" approach.
>
> Besides, the bug that Ævar discovered was a bug already in the scripted
> rebase, but hidden by yet another bug (the missing error checking).
>
> I get the pretty firm impression that the common code paths are now pretty
> robust, and only lesser-exercised features may expose a bug (or
> regression, as in the case of the reflogs, where one could argue that the
> exact reflog message is not something we promise not to fiddle with).

Since I raised this 'should we hold off?' I thought I'd chime in and say
that I'm fine with going along with what you suggest and having the
builtin as the default in the final. IOW not merge
jc/postpone-rebase-in-c down.

[PATCH 0/2] format-patch: fix root cause of recent regression

[Ævar Arnfjörð Bjarmason]

As noted in 2/2 this fixes the root cause of the bug I plastered over
in
https://public-inbox.org/git/20181122211248.24546-3-avarab@gmail.com/
(that patch is sitting in 'next').

1/2 is a test for existing behavior, to make it more easily understood
what's being changed.

Junio: I know it's late, but unless Eric has objections to this UI
change I'd really like to have this in 2.20 since this is a change to
a new command-line UI that's newly added in 2.20.

As noted in 2/2 the current implementation is inherently limited, you
can't tweak diff options for the range-diff in the cover-letter and
the patch independently, now you can, and the implementation is much
less nasty now that we're not having to share diffopts across two
different modes of operation.

Ævar Arnfjörð Bjarmason (2):
  format-patch: add test for --range-diff diff output
  format-patch: allow for independent diff & range-diff options

 Documentation/git-format-patch.txt |  10 ++-
 builtin/log.c                      |  42 +++++++++---
 t/t3206-range-diff.sh              | 101 +++++++++++++++++++++++++++++
 3 files changed, 142 insertions(+), 11 deletions(-)

-- 
2.20.0.rc1.387.gf8505762e3

[PATCH 1/2] format-patch: add test for --range-diff diff output

[Ævar Arnfjörð Bjarmason]

As noted in 43dafc4172 ("format-patch: don't include --stat with
--range-diff output", 2018-11-22) the diff options provided on the
command-line currently affect both the range-diff and the patch
output, but there was no test for checking this with output where we'd
show a patch diff. Let's add one.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t3206-range-diff.sh | 60 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index 90def330bd..bc5facc1cd 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -267,4 +267,64 @@ test_expect_success 'format-patch --range-diff as commentary' '
 	test_i18ngrep "^Range-diff:$" actual
 '
 
+test_expect_success 'format-patch with <common diff option>' '
+	# No diff options
+	git format-patch --cover-letter --stdout --range-diff=topic~..topic \
+		changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -9,7 +9,7 @@ A
+	 9
+	 10
+	 BB
+	-12
+	+B
+	 13
+	 14
+	 15
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0
+	git format-patch --cover-letter --stdout -U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -11 +11 @@
+	    - B
+	    + BB
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff
+'
+
 test_done
-- 
2.20.0.rc1.387.gf8505762e3

[PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Ævar Arnfjörð Bjarmason]

Change the semantics of the "--range-diff" option so that the regular
diff options can be provided separately for the range-diff and the
patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
"format-patch" to provide different context for the range-diff and the
patch. This wasn't possible before.

Ever since the "--range-diff" option was added in
31e2617a5f ("format-patch: add --range-diff option to embed diff in
cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff
machinery has been the one we get from "format-patch"'s own
setup_revisions().

This sort of thing is unique among the log-like commands in
builtin/log.c, no command than format-patch will embed the output of
another log-like command. Since the "rev->diffopt" is reused we need
to munge it before we pass it to show_range_diff(). See
43dafc4172 ("format-patch: don't include --stat with --range-diff
output", 2018-11-22) for a related regression fix which is being
mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown
around die() and will leak memory, but it's too much boilerplate to do
all the frees with/without the die() and not worth it. 2) We call
repo_init_revisions() for "rd_rev" even though we could get away with
a shallow copy like the code we're replacing (and which
show_range_diff() itself does). This is to make this code more easily
understood.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index aba4c5febe..6c048f415f 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
diff --git a/builtin/log.c b/builtin/log.c
index 02d88fa233..7658e56ecc 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
 	fprintf(rev->diffopt.file, "\n");
 }
 
-static void make_cover_letter(struct rev_info *rev, int use_stdout,
+static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
+			      int use_stdout,
 			      struct commit *origin,
 			      int nr, struct commit **list,
 			      const char *branch_name,
@@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
-		struct diff_options opts;
-		memcpy(&opts, &rev->diffopt, sizeof(opts));
-		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
-
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &opts);
+				rev->creation_factor, 1, &rd_rev->diffopt);
 	}
 }
 
@@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	struct commit *commit;
 	struct commit **list = NULL;
 	struct rev_info rev;
+	struct rev_info rd_rev;
 	struct setup_revision_opt s_r_opt;
 	int nr = 0, total, i;
 	int use_stdout = 0;
@@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	init_log_defaults();
 	git_config(git_format_config, NULL);
 	repo_init_revisions(the_repository, &rev, prefix);
+	repo_init_revisions(the_repository, &rd_rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.expand_tabs_in_log_default = 0;
 	rev.verbose_header = 1;
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}
 
 	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
 		die(_("--name-only does not make sense"));
@@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
-		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
@@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (cover_letter) {
 		if (thread)
 			gen_message_id(&rev, "cover");
-		make_cover_letter(&rev, use_stdout,
+		make_cover_letter(&rev, &rd_rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
 		print_bases(&bases, rev.diffopt.file);
 		print_signature(rev.diffopt.file);
diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index bc5facc1cd..6916103888 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
 		--range-diff=topic~..topic changed~..changed >actual.raw &&
 	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
 	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0 & --range-diff-U0
+	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
 	1:  a63e992 ! 1:  d966c5c s/12/B/
 	    @@ -11 +11 @@
 	    - B
@@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
 	test_cmp expect actual.diff
 '
 
+test_expect_success 'format-patch option parsing with --range-diff-*' '
+	test_must_fail git format-patch --stdout --unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr &&
+	test_must_fail git format-patch --stdout --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
+	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr
+'
+
 test_done
-- 
2.20.0.rc1.387.gf8505762e3

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> +		      [--range-diff<common diff option>]]

Let's make sure a random string thrown at this mechanism will
properly get noticed and diagnosed.

> @@ -257,6 +258,13 @@ feeding the result to `git send-email`.
>  	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
>  	for details.
>  
> +--range-diff<common diff option>::
> +	Other options prefixed with `--range-diff` are stripped of
> +	that prefix and passed as-is to the diff machinery used to
> +	generate the range-diff, e.g. `--range-diff-U0` and
> +	`--range-diff--no-color`. This allows for adjusting the format
> +	of the range-diff independently from the patch itself.

Taking anything is of course the most general, but I am afraid if
this backfires if there are some options that do not make sense to
be different between the invocations of range-diff and format-patch.

> @@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  	rev.preserve_subject = keep_subject;
>  
>  	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
> -	if (argc > 1)
> -		die(_("unrecognized argument: %s"), argv[1]);
> +	if (argc > 1) {
> +		struct argv_array args = ARGV_ARRAY_INIT;
> +		const char *prefix = "--range-diff";

Please call that anything but "prefix" that hides the parameter to
the function.  

	const char *range_diff_opt = "--range-diff";

might work OK, or it might not.  Let's read on.

> +		int have_prefix = 0;
> +
> +		for (i = 0; i < argc; i++) {
> +			struct strbuf sb = STRBUF_INIT;
> +			char *str;
> +
> +			strbuf_addstr(&sb, argv[i]);
> +			if (starts_with(argv[i], prefix)) {
> +				have_prefix = 1;
> +				strbuf_remove(&sb, 0, strlen(prefix));
> +			}
> +			str = strbuf_detach(&sb, NULL);
> +			strbuf_release(&sb);
> +
> +			argv_array_push(&args, str);
> +		}
> +

Is the body of the loop essentially this?

			char *passopt = argv[i];
			if (!skip_prefix(passopt, range_diff_opt, &passopt))
				saw_range_diff_opt = 1;
			argv_array_push(&args, xstrdup(passopt));

We only use that "prefix" thing once, so we may not even need the
variable.

> +		if (!have_prefix)
> +			die(_("unrecognized argument: %s"), argv[1]);

So we take normal options and check the leftover args; if there is
no --range-diff<whatever> among the leftover bits, we pretend that
we stumbled while reading the first such leftover arg.

> +		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
> +		if (argc > 1)
> +			die(_("unrecognized argument: %s"), argv[1]);
> +	}

Otherwise, we pass all the leftover bits, which is a random mixture
but guaranteed to have at least one meant for range-diff, to another
setup_revisions().  If it leaves a leftover arg, then that is
diagnosed here, so we'd be OK (iow, this is not a new "attack
vector" to inject random string to command line parser).

One minor glitch I can see is "format-patch --range-diffSilly" would
report "unrecognised arg: Silly".  As we are pretending to be and
reporting errors as format-patch, it would be better if we report
that --range-diffSilly was what we did not understand.

> Junio: I know it's late, but unless Eric has objections to this UI
> change I'd really like to have this in 2.20 since this is a change to
> a new command-line UI that's newly added in 2.20.

Quite honestly, I'd rather document "driving range-diff from
format-patch is experimental and does silly things when given
non-standard options in this release" and not touch the code at this
late stage in the game.  Would it be less intrusive a change to
*not* support the --range-diff<whatever> option, still use rd_rev
that is separate from the main rev, and use a reasonable hardcoded
default settings when preparing rd_rev?

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> Since I raised this 'should we hold off?' I thought I'd chime in and say
> that I'm fine with going along with what you suggest and having the
> builtin as the default in the final. IOW not merge
> jc/postpone-rebase-in-c down.

OK.

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 9368 bytes --]

Hi Ævar,

On Wed, 28 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> Change the semantics of the "--range-diff" option so that the regular
> diff options can be provided separately for the range-diff and the
> patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
> "format-patch" to provide different context for the range-diff and the
> patch. This wasn't possible before.

I really, really dislike the `--range-diff-<random-thing>`. We have
precedent for passing optional arguments that are passed to some other
command, so a much more logical and consistent convention would be to use
`--range-diff[=<diff-option>..]`, allowing all of the diff options that
you might want to pass to the outer diff in one go rather than having a
lengthy string of `--range-diff-this` and `--range-diff-that` options.

I only had time to skim the patch, and I have to wonder why you pass
around full-blown `rev_info` structs for range diff (and with that really
awful name `rd_rev`) rather than just the `diff_options` that you
*actually* care about?

Ciao,
Dscho

> 
> Ever since the "--range-diff" option was added in
> 31e2617a5f ("format-patch: add --range-diff option to embed diff in
> cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff
> machinery has been the one we get from "format-patch"'s own
> setup_revisions().
> 
> This sort of thing is unique among the log-like commands in
> builtin/log.c, no command than format-patch will embed the output of
> another log-like command. Since the "rev->diffopt" is reused we need
> to munge it before we pass it to show_range_diff(). See
> 43dafc4172 ("format-patch: don't include --stat with --range-diff
> output", 2018-11-22) for a related regression fix which is being
> mostly reverted here.
> 
> Implementation notes: 1) We're not bothering with the full teardown
> around die() and will leak memory, but it's too much boilerplate to do
> all the frees with/without the die() and not worth it. 2) We call
> repo_init_revisions() for "rd_rev" even though we could get away with
> a shallow copy like the code we're replacing (and which
> show_range_diff() itself does). This is to make this code more easily
> understood.
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  Documentation/git-format-patch.txt | 10 ++++++-
>  builtin/log.c                      | 42 +++++++++++++++++++++++-------
>  t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
>  3 files changed, 82 insertions(+), 11 deletions(-)
> 
> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
> index aba4c5febe..6c048f415f 100644
> --- a/Documentation/git-format-patch.txt
> +++ b/Documentation/git-format-patch.txt
> @@ -24,7 +24,8 @@ SYNOPSIS
>  		   [--to=<email>] [--cc=<email>]
>  		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
>  		   [--interdiff=<previous>]
> -		   [--range-diff=<previous> [--creation-factor=<percent>]]
> +		   [--range-diff=<previous> [--creation-factor=<percent>]
> +		      [--range-diff<common diff option>]]
>  		   [--progress]
>  		   [<common diff options>]
>  		   [ <since> | <revision range> ]
> @@ -257,6 +258,13 @@ feeding the result to `git send-email`.
>  	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
>  	for details.
>  
> +--range-diff<common diff option>::
> +	Other options prefixed with `--range-diff` are stripped of
> +	that prefix and passed as-is to the diff machinery used to
> +	generate the range-diff, e.g. `--range-diff-U0` and
> +	`--range-diff--no-color`. This allows for adjusting the format
> +	of the range-diff independently from the patch itself.
> +
>  --notes[=<ref>]::
>  	Append the notes (see linkgit:git-notes[1]) for the commit
>  	after the three-dash line.
> diff --git a/builtin/log.c b/builtin/log.c
> index 02d88fa233..7658e56ecc 100644
> --- a/builtin/log.c
> +++ b/builtin/log.c
> @@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
>  	fprintf(rev->diffopt.file, "\n");
>  }
>  
> -static void make_cover_letter(struct rev_info *rev, int use_stdout,
> +static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
> +			      int use_stdout,
>  			      struct commit *origin,
>  			      int nr, struct commit **list,
>  			      const char *branch_name,
> @@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
>  	}
>  
>  	if (rev->rdiff1) {
> -		struct diff_options opts;
> -		memcpy(&opts, &rev->diffopt, sizeof(opts));
> -		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
> -
>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
>  		show_range_diff(rev->rdiff1, rev->rdiff2,
> -				rev->creation_factor, 1, &opts);
> +				rev->creation_factor, 1, &rd_rev->diffopt);
>  	}
>  }
>  
> @@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  	struct commit *commit;
>  	struct commit **list = NULL;
>  	struct rev_info rev;
> +	struct rev_info rd_rev;
>  	struct setup_revision_opt s_r_opt;
>  	int nr = 0, total, i;
>  	int use_stdout = 0;
> @@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  	init_log_defaults();
>  	git_config(git_format_config, NULL);
>  	repo_init_revisions(the_repository, &rev, prefix);
> +	repo_init_revisions(the_repository, &rd_rev, prefix);
>  	rev.commit_format = CMIT_FMT_EMAIL;
>  	rev.expand_tabs_in_log_default = 0;
>  	rev.verbose_header = 1;
> @@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  	rev.preserve_subject = keep_subject;
>  
>  	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
> -	if (argc > 1)
> -		die(_("unrecognized argument: %s"), argv[1]);
> +	if (argc > 1) {
> +		struct argv_array args = ARGV_ARRAY_INIT;
> +		const char *prefix = "--range-diff";
> +		int have_prefix = 0;
> +
> +		for (i = 0; i < argc; i++) {
> +			struct strbuf sb = STRBUF_INIT;
> +			char *str;
> +
> +			strbuf_addstr(&sb, argv[i]);
> +			if (starts_with(argv[i], prefix)) {
> +				have_prefix = 1;
> +				strbuf_remove(&sb, 0, strlen(prefix));
> +			}
> +			str = strbuf_detach(&sb, NULL);
> +			strbuf_release(&sb);
> +
> +			argv_array_push(&args, str);
> +		}
> +
> +		if (!have_prefix)
> +			die(_("unrecognized argument: %s"), argv[1]);
> +		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
> +		if (argc > 1)
> +			die(_("unrecognized argument: %s"), argv[1]);
> +	}
>  
>  	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
>  		die(_("--name-only does not make sense"));
> @@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  	if (!use_patch_format &&
>  		(!rev.diffopt.output_format ||
>  		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
> -		/* Needs to be mirrored in show_range_diff() invocation */
>  		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
>  	if (!rev.diffopt.stat_width)
>  		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
> @@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>  	if (cover_letter) {
>  		if (thread)
>  			gen_message_id(&rev, "cover");
> -		make_cover_letter(&rev, use_stdout,
> +		make_cover_letter(&rev, &rd_rev, use_stdout,
>  				  origin, nr, list, branch_name, quiet);
>  		print_bases(&bases, rev.diffopt.file);
>  		print_signature(rev.diffopt.file);
> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> index bc5facc1cd..6916103888 100755
> --- a/t/t3206-range-diff.sh
> +++ b/t/t3206-range-diff.sh
> @@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
>  		--range-diff=topic~..topic changed~..changed >actual.raw &&
>  	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
>  	sed -e "s|:$||" >expect <<-\EOF &&
> +	1:  a63e992 ! 1:  d966c5c s/12/B/
> +	    @@ -8,7 +8,7 @@
> +	     @@
> +	      9
> +	      10
> +	    - B
> +	    + BB
> +	     -12
> +	     +B
> +	      13
> +	-- :
> +	EOF
> +	test_cmp expect actual.range-diff &&
> +	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
> +	sed -e "s|:$||" >expect <<-\EOF &&
> +	--- a/file
> +	+++ b/file
> +	@@ -12 +12 @@ BB
> +	-12
> +	+B
> +	-- :
> +	EOF
> +	test_cmp expect actual.diff &&
> +
> +	# -U0 & --range-diff-U0
> +	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
> +		--range-diff=topic~..topic changed~..changed >actual.raw &&
> +	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
> +	sed -e "s|:$||" >expect <<-\EOF &&
>  	1:  a63e992 ! 1:  d966c5c s/12/B/
>  	    @@ -11 +11 @@
>  	    - B
> @@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
>  	test_cmp expect actual.diff
>  '
>  
> +test_expect_success 'format-patch option parsing with --range-diff-*' '
> +	test_must_fail git format-patch --stdout --unknown \
> +		master..unmodified 2>stderr &&
> +	test_i18ngrep "unrecognized argument: --unknown" stderr &&
> +	test_must_fail git format-patch --stdout --range-diff-unknown \
> +		master..unmodified 2>stderr &&
> +	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
> +	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
> +		master..unmodified 2>stderr &&
> +	test_i18ngrep "unrecognized argument: --unknown" stderr
> +'
> +
>  test_done
> -- 
> 2.20.0.rc1.387.gf8505762e3
> 
> 

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 29 2018, Johannes Schindelin wrote:

> Hi Ævar,
>
> On Wed, 28 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
>
>> Change the semantics of the "--range-diff" option so that the regular
>> diff options can be provided separately for the range-diff and the
>> patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
>> "format-patch" to provide different context for the range-diff and the
>> patch. This wasn't possible before.
>
> I really, really dislike the `--range-diff-<random-thing>`. We have
> precedent for passing optional arguments that are passed to some other
> command, so a much more logical and consistent convention would be to use
> `--range-diff[=<diff-option>..]`, allowing all of the diff options that
> you might want to pass to the outer diff in one go rather than having a
> lengthy string of `--range-diff-this` and `--range-diff-that` options.

Where do we pass those sorts of arguments?

Reasons I did it this way:

 a) Passing it as one option will require the user to double-quote those
    options that take quoted arguments (e.g. --word-diff-regex), which I
    thought sucked more than the prefix. On the implementation side we
    couldn't leave the parsing of the command-line to the shell anymore.

 b) I think people will want to tweak this very rarely, much more rarely
    than e.g. -U10 in format-patch itself, so having something long-ish
    doesn't sound bad.

> I only had time to skim the patch, and I have to wonder why you pass
> around full-blown `rev_info` structs for range diff (and with that really
> awful name `rd_rev`) rather than just the `diff_options` that you
> *actually* care about?

Because setup_revisions() which does all the command-line parsing needs
a rev_info, so even if we only need the diffopt in the end we need to
initiate the whole thing.

Suggestions for a better varibale name most welcome.

> Ciao,
> Dscho
>
>>
>> Ever since the "--range-diff" option was added in
>> 31e2617a5f ("format-patch: add --range-diff option to embed diff in
>> cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff
>> machinery has been the one we get from "format-patch"'s own
>> setup_revisions().
>>
>> This sort of thing is unique among the log-like commands in
>> builtin/log.c, no command than format-patch will embed the output of
>> another log-like command. Since the "rev->diffopt" is reused we need
>> to munge it before we pass it to show_range_diff(). See
>> 43dafc4172 ("format-patch: don't include --stat with --range-diff
>> output", 2018-11-22) for a related regression fix which is being
>> mostly reverted here.
>>
>> Implementation notes: 1) We're not bothering with the full teardown
>> around die() and will leak memory, but it's too much boilerplate to do
>> all the frees with/without the die() and not worth it. 2) We call
>> repo_init_revisions() for "rd_rev" even though we could get away with
>> a shallow copy like the code we're replacing (and which
>> show_range_diff() itself does). This is to make this code more easily
>> understood.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>>  Documentation/git-format-patch.txt | 10 ++++++-
>>  builtin/log.c                      | 42 +++++++++++++++++++++++-------
>>  t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
>>  3 files changed, 82 insertions(+), 11 deletions(-)
>>
>> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
>> index aba4c5febe..6c048f415f 100644
>> --- a/Documentation/git-format-patch.txt
>> +++ b/Documentation/git-format-patch.txt
>> @@ -24,7 +24,8 @@ SYNOPSIS
>>  		   [--to=<email>] [--cc=<email>]
>>  		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
>>  		   [--interdiff=<previous>]
>> -		   [--range-diff=<previous> [--creation-factor=<percent>]]
>> +		   [--range-diff=<previous> [--creation-factor=<percent>]
>> +		      [--range-diff<common diff option>]]
>>  		   [--progress]
>>  		   [<common diff options>]
>>  		   [ <since> | <revision range> ]
>> @@ -257,6 +258,13 @@ feeding the result to `git send-email`.
>>  	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
>>  	for details.
>>
>> +--range-diff<common diff option>::
>> +	Other options prefixed with `--range-diff` are stripped of
>> +	that prefix and passed as-is to the diff machinery used to
>> +	generate the range-diff, e.g. `--range-diff-U0` and
>> +	`--range-diff--no-color`. This allows for adjusting the format
>> +	of the range-diff independently from the patch itself.
>> +
>>  --notes[=<ref>]::
>>  	Append the notes (see linkgit:git-notes[1]) for the commit
>>  	after the three-dash line.
>> diff --git a/builtin/log.c b/builtin/log.c
>> index 02d88fa233..7658e56ecc 100644
>> --- a/builtin/log.c
>> +++ b/builtin/log.c
>> @@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
>>  	fprintf(rev->diffopt.file, "\n");
>>  }
>>
>> -static void make_cover_letter(struct rev_info *rev, int use_stdout,
>> +static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
>> +			      int use_stdout,
>>  			      struct commit *origin,
>>  			      int nr, struct commit **list,
>>  			      const char *branch_name,
>> @@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
>>  	}
>>
>>  	if (rev->rdiff1) {
>> -		struct diff_options opts;
>> -		memcpy(&opts, &rev->diffopt, sizeof(opts));
>> -		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
>> -
>>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
>>  		show_range_diff(rev->rdiff1, rev->rdiff2,
>> -				rev->creation_factor, 1, &opts);
>> +				rev->creation_factor, 1, &rd_rev->diffopt);
>>  	}
>>  }
>>
>> @@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  	struct commit *commit;
>>  	struct commit **list = NULL;
>>  	struct rev_info rev;
>> +	struct rev_info rd_rev;
>>  	struct setup_revision_opt s_r_opt;
>>  	int nr = 0, total, i;
>>  	int use_stdout = 0;
>> @@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  	init_log_defaults();
>>  	git_config(git_format_config, NULL);
>>  	repo_init_revisions(the_repository, &rev, prefix);
>> +	repo_init_revisions(the_repository, &rd_rev, prefix);
>>  	rev.commit_format = CMIT_FMT_EMAIL;
>>  	rev.expand_tabs_in_log_default = 0;
>>  	rev.verbose_header = 1;
>> @@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  	rev.preserve_subject = keep_subject;
>>
>>  	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
>> -	if (argc > 1)
>> -		die(_("unrecognized argument: %s"), argv[1]);
>> +	if (argc > 1) {
>> +		struct argv_array args = ARGV_ARRAY_INIT;
>> +		const char *prefix = "--range-diff";
>> +		int have_prefix = 0;
>> +
>> +		for (i = 0; i < argc; i++) {
>> +			struct strbuf sb = STRBUF_INIT;
>> +			char *str;
>> +
>> +			strbuf_addstr(&sb, argv[i]);
>> +			if (starts_with(argv[i], prefix)) {
>> +				have_prefix = 1;
>> +				strbuf_remove(&sb, 0, strlen(prefix));
>> +			}
>> +			str = strbuf_detach(&sb, NULL);
>> +			strbuf_release(&sb);
>> +
>> +			argv_array_push(&args, str);
>> +		}
>> +
>> +		if (!have_prefix)
>> +			die(_("unrecognized argument: %s"), argv[1]);
>> +		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
>> +		if (argc > 1)
>> +			die(_("unrecognized argument: %s"), argv[1]);
>> +	}
>>
>>  	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
>>  		die(_("--name-only does not make sense"));
>> @@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  	if (!use_patch_format &&
>>  		(!rev.diffopt.output_format ||
>>  		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
>> -		/* Needs to be mirrored in show_range_diff() invocation */
>>  		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
>>  	if (!rev.diffopt.stat_width)
>>  		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
>> @@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>>  	if (cover_letter) {
>>  		if (thread)
>>  			gen_message_id(&rev, "cover");
>> -		make_cover_letter(&rev, use_stdout,
>> +		make_cover_letter(&rev, &rd_rev, use_stdout,
>>  				  origin, nr, list, branch_name, quiet);
>>  		print_bases(&bases, rev.diffopt.file);
>>  		print_signature(rev.diffopt.file);
>> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
>> index bc5facc1cd..6916103888 100755
>> --- a/t/t3206-range-diff.sh
>> +++ b/t/t3206-range-diff.sh
>> @@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
>>  		--range-diff=topic~..topic changed~..changed >actual.raw &&
>>  	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
>>  	sed -e "s|:$||" >expect <<-\EOF &&
>> +	1:  a63e992 ! 1:  d966c5c s/12/B/
>> +	    @@ -8,7 +8,7 @@
>> +	     @@
>> +	      9
>> +	      10
>> +	    - B
>> +	    + BB
>> +	     -12
>> +	     +B
>> +	      13
>> +	-- :
>> +	EOF
>> +	test_cmp expect actual.range-diff &&
>> +	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
>> +	sed -e "s|:$||" >expect <<-\EOF &&
>> +	--- a/file
>> +	+++ b/file
>> +	@@ -12 +12 @@ BB
>> +	-12
>> +	+B
>> +	-- :
>> +	EOF
>> +	test_cmp expect actual.diff &&
>> +
>> +	# -U0 & --range-diff-U0
>> +	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
>> +		--range-diff=topic~..topic changed~..changed >actual.raw &&
>> +	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
>> +	sed -e "s|:$||" >expect <<-\EOF &&
>>  	1:  a63e992 ! 1:  d966c5c s/12/B/
>>  	    @@ -11 +11 @@
>>  	    - B
>> @@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
>>  	test_cmp expect actual.diff
>>  '
>>
>> +test_expect_success 'format-patch option parsing with --range-diff-*' '
>> +	test_must_fail git format-patch --stdout --unknown \
>> +		master..unmodified 2>stderr &&
>> +	test_i18ngrep "unrecognized argument: --unknown" stderr &&
>> +	test_must_fail git format-patch --stdout --range-diff-unknown \
>> +		master..unmodified 2>stderr &&
>> +	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
>> +	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
>> +		master..unmodified 2>stderr &&
>> +	test_i18ngrep "unrecognized argument: --unknown" stderr
>> +'
>> +
>>  test_done
>> --
>> 2.20.0.rc1.387.gf8505762e3
>>
>>

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 11467 bytes --]

Hi Ævar,

On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> On Thu, Nov 29 2018, Johannes Schindelin wrote:
> 
> > On Wed, 28 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
> >
> >> Change the semantics of the "--range-diff" option so that the regular
> >> diff options can be provided separately for the range-diff and the
> >> patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
> >> "format-patch" to provide different context for the range-diff and the
> >> patch. This wasn't possible before.
> >
> > I really, really dislike the `--range-diff-<random-thing>`. We have
> > precedent for passing optional arguments that are passed to some other
> > command, so a much more logical and consistent convention would be to use
> > `--range-diff[=<diff-option>..]`, allowing all of the diff options that
> > you might want to pass to the outer diff in one go rather than having a
> > lengthy string of `--range-diff-this` and `--range-diff-that` options.
> 
> Where do we pass those sorts of arguments?
> 
> Reasons I did it this way:
> 
>  a) Passing it as one option will require the user to double-quote those
>     options that take quoted arguments (e.g. --word-diff-regex), which I
>     thought sucked more than the prefix. On the implementation side we
>     couldn't leave the parsing of the command-line to the shell anymore.
> 
>  b) I think people will want to tweak this very rarely, much more rarely
>     than e.g. -U10 in format-patch itself, so having something long-ish
>     doesn't sound bad.

Hmm. I still don't like it. It sets a precedent, and we simply do not do
it that way in other circumstances (most obvious would be the -X merge
options). The more divergent user interfaces for the same sort of thing
are, the more brain cycles you force users to spend on navigating said
interfaces.

> > I only had time to skim the patch, and I have to wonder why you pass
> > around full-blown `rev_info` structs for range diff (and with that really
> > awful name `rd_rev`) rather than just the `diff_options` that you
> > *actually* care about?
> 
> Because setup_revisions() which does all the command-line parsing needs
> a rev_info, so even if we only need the diffopt in the end we need to
> initiate the whole thing.
> 
> Suggestions for a better varibale name most welcome.

`range_diff_revs`

And you do not need to pass around the whole thing. You can easily pass
`&range_diff_revs.diffopt`.

Don't pass around what you do not need to pass around.

Ciao,
Dscho

> 
> > Ciao,
> > Dscho
> >
> >>
> >> Ever since the "--range-diff" option was added in
> >> 31e2617a5f ("format-patch: add --range-diff option to embed diff in
> >> cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff
> >> machinery has been the one we get from "format-patch"'s own
> >> setup_revisions().
> >>
> >> This sort of thing is unique among the log-like commands in
> >> builtin/log.c, no command than format-patch will embed the output of
> >> another log-like command. Since the "rev->diffopt" is reused we need
> >> to munge it before we pass it to show_range_diff(). See
> >> 43dafc4172 ("format-patch: don't include --stat with --range-diff
> >> output", 2018-11-22) for a related regression fix which is being
> >> mostly reverted here.
> >>
> >> Implementation notes: 1) We're not bothering with the full teardown
> >> around die() and will leak memory, but it's too much boilerplate to do
> >> all the frees with/without the die() and not worth it. 2) We call
> >> repo_init_revisions() for "rd_rev" even though we could get away with
> >> a shallow copy like the code we're replacing (and which
> >> show_range_diff() itself does). This is to make this code more easily
> >> understood.
> >>
> >> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> >> ---
> >>  Documentation/git-format-patch.txt | 10 ++++++-
> >>  builtin/log.c                      | 42 +++++++++++++++++++++++-------
> >>  t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
> >>  3 files changed, 82 insertions(+), 11 deletions(-)
> >>
> >> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
> >> index aba4c5febe..6c048f415f 100644
> >> --- a/Documentation/git-format-patch.txt
> >> +++ b/Documentation/git-format-patch.txt
> >> @@ -24,7 +24,8 @@ SYNOPSIS
> >>  		   [--to=<email>] [--cc=<email>]
> >>  		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
> >>  		   [--interdiff=<previous>]
> >> -		   [--range-diff=<previous> [--creation-factor=<percent>]]
> >> +		   [--range-diff=<previous> [--creation-factor=<percent>]
> >> +		      [--range-diff<common diff option>]]
> >>  		   [--progress]
> >>  		   [<common diff options>]
> >>  		   [ <since> | <revision range> ]
> >> @@ -257,6 +258,13 @@ feeding the result to `git send-email`.
> >>  	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
> >>  	for details.
> >>
> >> +--range-diff<common diff option>::
> >> +	Other options prefixed with `--range-diff` are stripped of
> >> +	that prefix and passed as-is to the diff machinery used to
> >> +	generate the range-diff, e.g. `--range-diff-U0` and
> >> +	`--range-diff--no-color`. This allows for adjusting the format
> >> +	of the range-diff independently from the patch itself.
> >> +
> >>  --notes[=<ref>]::
> >>  	Append the notes (see linkgit:git-notes[1]) for the commit
> >>  	after the three-dash line.
> >> diff --git a/builtin/log.c b/builtin/log.c
> >> index 02d88fa233..7658e56ecc 100644
> >> --- a/builtin/log.c
> >> +++ b/builtin/log.c
> >> @@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
> >>  	fprintf(rev->diffopt.file, "\n");
> >>  }
> >>
> >> -static void make_cover_letter(struct rev_info *rev, int use_stdout,
> >> +static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
> >> +			      int use_stdout,
> >>  			      struct commit *origin,
> >>  			      int nr, struct commit **list,
> >>  			      const char *branch_name,
> >> @@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
> >>  	}
> >>
> >>  	if (rev->rdiff1) {
> >> -		struct diff_options opts;
> >> -		memcpy(&opts, &rev->diffopt, sizeof(opts));
> >> -		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
> >> -
> >>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
> >>  		show_range_diff(rev->rdiff1, rev->rdiff2,
> >> -				rev->creation_factor, 1, &opts);
> >> +				rev->creation_factor, 1, &rd_rev->diffopt);
> >>  	}
> >>  }
> >>
> >> @@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >>  	struct commit *commit;
> >>  	struct commit **list = NULL;
> >>  	struct rev_info rev;
> >> +	struct rev_info rd_rev;
> >>  	struct setup_revision_opt s_r_opt;
> >>  	int nr = 0, total, i;
> >>  	int use_stdout = 0;
> >> @@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >>  	init_log_defaults();
> >>  	git_config(git_format_config, NULL);
> >>  	repo_init_revisions(the_repository, &rev, prefix);
> >> +	repo_init_revisions(the_repository, &rd_rev, prefix);
> >>  	rev.commit_format = CMIT_FMT_EMAIL;
> >>  	rev.expand_tabs_in_log_default = 0;
> >>  	rev.verbose_header = 1;
> >> @@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >>  	rev.preserve_subject = keep_subject;
> >>
> >>  	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
> >> -	if (argc > 1)
> >> -		die(_("unrecognized argument: %s"), argv[1]);
> >> +	if (argc > 1) {
> >> +		struct argv_array args = ARGV_ARRAY_INIT;
> >> +		const char *prefix = "--range-diff";
> >> +		int have_prefix = 0;
> >> +
> >> +		for (i = 0; i < argc; i++) {
> >> +			struct strbuf sb = STRBUF_INIT;
> >> +			char *str;
> >> +
> >> +			strbuf_addstr(&sb, argv[i]);
> >> +			if (starts_with(argv[i], prefix)) {
> >> +				have_prefix = 1;
> >> +				strbuf_remove(&sb, 0, strlen(prefix));
> >> +			}
> >> +			str = strbuf_detach(&sb, NULL);
> >> +			strbuf_release(&sb);
> >> +
> >> +			argv_array_push(&args, str);
> >> +		}
> >> +
> >> +		if (!have_prefix)
> >> +			die(_("unrecognized argument: %s"), argv[1]);
> >> +		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
> >> +		if (argc > 1)
> >> +			die(_("unrecognized argument: %s"), argv[1]);
> >> +	}
> >>
> >>  	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
> >>  		die(_("--name-only does not make sense"));
> >> @@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >>  	if (!use_patch_format &&
> >>  		(!rev.diffopt.output_format ||
> >>  		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
> >> -		/* Needs to be mirrored in show_range_diff() invocation */
> >>  		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
> >>  	if (!rev.diffopt.stat_width)
> >>  		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
> >> @@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >>  	if (cover_letter) {
> >>  		if (thread)
> >>  			gen_message_id(&rev, "cover");
> >> -		make_cover_letter(&rev, use_stdout,
> >> +		make_cover_letter(&rev, &rd_rev, use_stdout,
> >>  				  origin, nr, list, branch_name, quiet);
> >>  		print_bases(&bases, rev.diffopt.file);
> >>  		print_signature(rev.diffopt.file);
> >> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> >> index bc5facc1cd..6916103888 100755
> >> --- a/t/t3206-range-diff.sh
> >> +++ b/t/t3206-range-diff.sh
> >> @@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
> >>  		--range-diff=topic~..topic changed~..changed >actual.raw &&
> >>  	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
> >>  	sed -e "s|:$||" >expect <<-\EOF &&
> >> +	1:  a63e992 ! 1:  d966c5c s/12/B/
> >> +	    @@ -8,7 +8,7 @@
> >> +	     @@
> >> +	      9
> >> +	      10
> >> +	    - B
> >> +	    + BB
> >> +	     -12
> >> +	     +B
> >> +	      13
> >> +	-- :
> >> +	EOF
> >> +	test_cmp expect actual.range-diff &&
> >> +	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
> >> +	sed -e "s|:$||" >expect <<-\EOF &&
> >> +	--- a/file
> >> +	+++ b/file
> >> +	@@ -12 +12 @@ BB
> >> +	-12
> >> +	+B
> >> +	-- :
> >> +	EOF
> >> +	test_cmp expect actual.diff &&
> >> +
> >> +	# -U0 & --range-diff-U0
> >> +	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
> >> +		--range-diff=topic~..topic changed~..changed >actual.raw &&
> >> +	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
> >> +	sed -e "s|:$||" >expect <<-\EOF &&
> >>  	1:  a63e992 ! 1:  d966c5c s/12/B/
> >>  	    @@ -11 +11 @@
> >>  	    - B
> >> @@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
> >>  	test_cmp expect actual.diff
> >>  '
> >>
> >> +test_expect_success 'format-patch option parsing with --range-diff-*' '
> >> +	test_must_fail git format-patch --stdout --unknown \
> >> +		master..unmodified 2>stderr &&
> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr &&
> >> +	test_must_fail git format-patch --stdout --range-diff-unknown \
> >> +		master..unmodified 2>stderr &&
> >> +	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
> >> +	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
> >> +		master..unmodified 2>stderr &&
> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr
> >> +'
> >> +
> >>  test_done
> >> --
> >> 2.20.0.rc1.387.gf8505762e3
> >>
> >>
> 

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3675 bytes --]

Hi Jonathan,

if you could pry more information (or better information) out of that bug
reporter, that would be nice. Apparently my email address is blacklisted
by his mail provider, so he is unlikely to have received my previous mail
(nor will he receive this one, I am sure).

Thanks,
Dscho

On Wed, 28 Nov 2018, Johannes Schindelin wrote:

> Hi Jonathan,
> 
> On Tue, 27 Nov 2018, Jonathan Nieder wrote:
> 
> > At https://bugs.debian.org/914695 is a report of a test regression in
> > an outside project that is very likely to have been triggered by the
> > new faster rebase code.
> 
> From looking through that log.gz (without having a clue where the test
> code lives, so I cannot say what it is supposed to do, and also: this is
> the first time I hear about dgit...), it would appear that this must be a
> regression in the reflog messages produced by `git rebase`.
> 
> > The issue has not been triaged, so I don't know yet whether it's a
> > problem in rebase-in-c or a manifestation of a bug in the test.
> 
> It ends thusly:
> 
> -- snip --
> [...]
> + git reflog
> + egrep 'debrebase new-upstream.*checkout'
> + test 1 = 0
> + t-report-failure
> + set +x
> TEST FAILED
> -- snap --
> 
> Which makes me think that the reflog we produce in *some* code path that
> originally called `git checkout` differs from the scripted rebase's
> generated reflog.
> 
> > That said, Google has been running with the new rebase since ~1 month
> > ago when it became the default, with no issues reported by users.  As a
> > result, I am confident that it can cope with what most users of "next"
> > throw at it, which means that if we are to find more issues to polish it
> > better, it will need all the exposure it can get.
> 
> Right. And there are a few weeks before the holidays, which should give me
> time to fix whatever bugs are discovered (I only half mind being the only
> one who fixes these bugs).
> 
> > In the Google deployment, we will keep using rebase-in-c even if it
> > gets disabled by default, in order to help with that.
> > 
> > From the Debian point of view, it's only a matter of time before
> > rebase-in-c becomes the default: even if it's not the default in 2.20,
> > it would presumably be so in 2.21 or 2.22.  That means the community's
> > attention when resolving security and reliability bugs would be on the
> > rebase-in-c implementation.  As a result, the Debian package will most
> > likely enable rebase-in-c by default even if upstream disables it, in
> > order to increase the package's shelf life (i.e. to ease the
> > maintenance burden of supporting whichever version of the package ends
> > up in the next Debian stable).
> > 
> > So with either hat on, it doesn't matter whether you apply this patch
> > upstream.
> > 
> > Having two pretty different deployments end up with the same
> > conclusion leads me to suspect that it's best for upstream not to
> > apply the revert patch, unless either
> > 
> >   (a) we have a concrete regression to address and then try again, or
> >   (b) we have a test or other plan to follow before trying again.
> 
> In this instance, I am more a fan of the "let's move fast and break
> things, then move even faster fixing them" approach.
> 
> Besides, the bug that Ævar discovered was a bug already in the scripted
> rebase, but hidden by yet another bug (the missing error checking).
> 
> I get the pretty firm impression that the common code paths are now pretty
> robust, and only lesser-exercised features may expose a bug (or
> regression, as in the case of the reflogs, where one could argue that the
> exact reflog message is not something we promise not to fiddle with).
> 
> Ciao,
> Dscho

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Ian Jackson]

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):
> if you could pry more information (or better information) out of that bug
> reporter, that would be nice. Apparently my email address is blacklisted
> by his mail provider, so he is unlikely to have received my previous mail
> (nor will he receive this one, I am sure).

(I did receive this mail.  Sorry for the inconvenience, which sadly is
inevitable occasionally in the modern email world.  FTR in future feel
free to send the bounce to postmaster@chiark and I will make a
you-shaped hole in my spamfilter.  Also with Debian bugs you can
launder your messages by, eg, emailing 914695-submitter@bugs.)

> > > At https://bugs.debian.org/914695 is a report of a test regression in
> > > an outside project that is very likely to have been triggered by the
> > > new faster rebase code.

As I wrote in the bug report last night:

 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914695#15

 I have investigated and the bug seems to be that git-rebase --onto now
 fails to honour GIT_REFLOG_ACTION for the initial checkout.

 In a successful run with older git I get a reflog like this:

   4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
   4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file
   cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
   0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
   29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3
   85e0c46 HEAD@{5}: debrebase: launder for new upstream

 With a newer git I get this:

   6d3fb91 HEAD@{0}: rebase finished: returning to refs/heads/master
   6d3fb91 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file
   86c0721 HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
   50ba56c HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
   8272825 HEAD@{4}: rebase: checkout 8272825bb4ff6eba89afa936e32b6460f963a36a
   c78db55 HEAD@{5}: debrebase: launder for new upstream

 This breaks the test because my test suite is checking that I set
 GIT_REFLOG_ACTION appropriately.

 If you want I can provide a minimal test case but this should suffice
 to see the bug I hope...

Regards
Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 29 2018, Johannes Schindelin wrote:

> Hi Ævar,
>
> On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
>
>> On Thu, Nov 29 2018, Johannes Schindelin wrote:
>>
>> > On Wed, 28 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
>> >
>> >> Change the semantics of the "--range-diff" option so that the regular
>> >> diff options can be provided separately for the range-diff and the
>> >> patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
>> >> "format-patch" to provide different context for the range-diff and the
>> >> patch. This wasn't possible before.
>> >
>> > I really, really dislike the `--range-diff-<random-thing>`. We have
>> > precedent for passing optional arguments that are passed to some other
>> > command, so a much more logical and consistent convention would be to use
>> > `--range-diff[=<diff-option>..]`, allowing all of the diff options that
>> > you might want to pass to the outer diff in one go rather than having a
>> > lengthy string of `--range-diff-this` and `--range-diff-that` options.
>>
>> Where do we pass those sorts of arguments?
>>
>> Reasons I did it this way:
>>
>>  a) Passing it as one option will require the user to double-quote those
>>     options that take quoted arguments (e.g. --word-diff-regex), which I
>>     thought sucked more than the prefix. On the implementation side we
>>     couldn't leave the parsing of the command-line to the shell anymore.
>>
>>  b) I think people will want to tweak this very rarely, much more rarely
>>     than e.g. -U10 in format-patch itself, so having something long-ish
>>     doesn't sound bad.
>
> Hmm. I still don't like it. It sets a precedent, and we simply do not do
> it that way in other circumstances (most obvious would be the -X merge
> options). The more divergent user interfaces for the same sort of thing
> are, the more brain cycles you force users to spend on navigating said
> interfaces.

Yeah it sucks, I just think it sucks less than the alternative :)
I.e. I'm not picky about --range-diff-* prefix the name, but I think
doing our own shell parsing would be nasty.

>> > I only had time to skim the patch, and I have to wonder why you pass
>> > around full-blown `rev_info` structs for range diff (and with that really
>> > awful name `rd_rev`) rather than just the `diff_options` that you
>> > *actually* care about?
>>
>> Because setup_revisions() which does all the command-line parsing needs
>> a rev_info, so even if we only need the diffopt in the end we need to
>> initiate the whole thing.
>>
>> Suggestions for a better varibale name most welcome.
>
> `range_diff_revs`
>
> And you do not need to pass around the whole thing. You can easily pass
> `&range_diff_revs.diffopt`.
>
> Don't pass around what you do not need to pass around.

Ah, you mean internally in log.c, yes that makes sense. I thought you
meant just pass diffopt to setup_revisions() (which needs the containing
struct). Willdo.

> Ciao,
> Dscho
>
>>
>> > Ciao,
>> > Dscho
>> >
>> >>
>> >> Ever since the "--range-diff" option was added in
>> >> 31e2617a5f ("format-patch: add --range-diff option to embed diff in
>> >> cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff
>> >> machinery has been the one we get from "format-patch"'s own
>> >> setup_revisions().
>> >>
>> >> This sort of thing is unique among the log-like commands in
>> >> builtin/log.c, no command than format-patch will embed the output of
>> >> another log-like command. Since the "rev->diffopt" is reused we need
>> >> to munge it before we pass it to show_range_diff(). See
>> >> 43dafc4172 ("format-patch: don't include --stat with --range-diff
>> >> output", 2018-11-22) for a related regression fix which is being
>> >> mostly reverted here.
>> >>
>> >> Implementation notes: 1) We're not bothering with the full teardown
>> >> around die() and will leak memory, but it's too much boilerplate to do
>> >> all the frees with/without the die() and not worth it. 2) We call
>> >> repo_init_revisions() for "rd_rev" even though we could get away with
>> >> a shallow copy like the code we're replacing (and which
>> >> show_range_diff() itself does). This is to make this code more easily
>> >> understood.
>> >>
>> >> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> >> ---
>> >>  Documentation/git-format-patch.txt | 10 ++++++-
>> >>  builtin/log.c                      | 42 +++++++++++++++++++++++-------
>> >>  t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
>> >>  3 files changed, 82 insertions(+), 11 deletions(-)
>> >>
>> >> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
>> >> index aba4c5febe..6c048f415f 100644
>> >> --- a/Documentation/git-format-patch.txt
>> >> +++ b/Documentation/git-format-patch.txt
>> >> @@ -24,7 +24,8 @@ SYNOPSIS
>> >>  		   [--to=<email>] [--cc=<email>]
>> >>  		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
>> >>  		   [--interdiff=<previous>]
>> >> -		   [--range-diff=<previous> [--creation-factor=<percent>]]
>> >> +		   [--range-diff=<previous> [--creation-factor=<percent>]
>> >> +		      [--range-diff<common diff option>]]
>> >>  		   [--progress]
>> >>  		   [<common diff options>]
>> >>  		   [ <since> | <revision range> ]
>> >> @@ -257,6 +258,13 @@ feeding the result to `git send-email`.
>> >>  	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
>> >>  	for details.
>> >>
>> >> +--range-diff<common diff option>::
>> >> +	Other options prefixed with `--range-diff` are stripped of
>> >> +	that prefix and passed as-is to the diff machinery used to
>> >> +	generate the range-diff, e.g. `--range-diff-U0` and
>> >> +	`--range-diff--no-color`. This allows for adjusting the format
>> >> +	of the range-diff independently from the patch itself.
>> >> +
>> >>  --notes[=<ref>]::
>> >>  	Append the notes (see linkgit:git-notes[1]) for the commit
>> >>  	after the three-dash line.
>> >> diff --git a/builtin/log.c b/builtin/log.c
>> >> index 02d88fa233..7658e56ecc 100644
>> >> --- a/builtin/log.c
>> >> +++ b/builtin/log.c
>> >> @@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
>> >>  	fprintf(rev->diffopt.file, "\n");
>> >>  }
>> >>
>> >> -static void make_cover_letter(struct rev_info *rev, int use_stdout,
>> >> +static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
>> >> +			      int use_stdout,
>> >>  			      struct commit *origin,
>> >>  			      int nr, struct commit **list,
>> >>  			      const char *branch_name,
>> >> @@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
>> >>  	}
>> >>
>> >>  	if (rev->rdiff1) {
>> >> -		struct diff_options opts;
>> >> -		memcpy(&opts, &rev->diffopt, sizeof(opts));
>> >> -		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
>> >> -
>> >>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
>> >>  		show_range_diff(rev->rdiff1, rev->rdiff2,
>> >> -				rev->creation_factor, 1, &opts);
>> >> +				rev->creation_factor, 1, &rd_rev->diffopt);
>> >>  	}
>> >>  }
>> >>
>> >> @@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >>  	struct commit *commit;
>> >>  	struct commit **list = NULL;
>> >>  	struct rev_info rev;
>> >> +	struct rev_info rd_rev;
>> >>  	struct setup_revision_opt s_r_opt;
>> >>  	int nr = 0, total, i;
>> >>  	int use_stdout = 0;
>> >> @@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >>  	init_log_defaults();
>> >>  	git_config(git_format_config, NULL);
>> >>  	repo_init_revisions(the_repository, &rev, prefix);
>> >> +	repo_init_revisions(the_repository, &rd_rev, prefix);
>> >>  	rev.commit_format = CMIT_FMT_EMAIL;
>> >>  	rev.expand_tabs_in_log_default = 0;
>> >>  	rev.verbose_header = 1;
>> >> @@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >>  	rev.preserve_subject = keep_subject;
>> >>
>> >>  	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
>> >> -	if (argc > 1)
>> >> -		die(_("unrecognized argument: %s"), argv[1]);
>> >> +	if (argc > 1) {
>> >> +		struct argv_array args = ARGV_ARRAY_INIT;
>> >> +		const char *prefix = "--range-diff";
>> >> +		int have_prefix = 0;
>> >> +
>> >> +		for (i = 0; i < argc; i++) {
>> >> +			struct strbuf sb = STRBUF_INIT;
>> >> +			char *str;
>> >> +
>> >> +			strbuf_addstr(&sb, argv[i]);
>> >> +			if (starts_with(argv[i], prefix)) {
>> >> +				have_prefix = 1;
>> >> +				strbuf_remove(&sb, 0, strlen(prefix));
>> >> +			}
>> >> +			str = strbuf_detach(&sb, NULL);
>> >> +			strbuf_release(&sb);
>> >> +
>> >> +			argv_array_push(&args, str);
>> >> +		}
>> >> +
>> >> +		if (!have_prefix)
>> >> +			die(_("unrecognized argument: %s"), argv[1]);
>> >> +		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
>> >> +		if (argc > 1)
>> >> +			die(_("unrecognized argument: %s"), argv[1]);
>> >> +	}
>> >>
>> >>  	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
>> >>  		die(_("--name-only does not make sense"));
>> >> @@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >>  	if (!use_patch_format &&
>> >>  		(!rev.diffopt.output_format ||
>> >>  		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
>> >> -		/* Needs to be mirrored in show_range_diff() invocation */
>> >>  		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
>> >>  	if (!rev.diffopt.stat_width)
>> >>  		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
>> >> @@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >>  	if (cover_letter) {
>> >>  		if (thread)
>> >>  			gen_message_id(&rev, "cover");
>> >> -		make_cover_letter(&rev, use_stdout,
>> >> +		make_cover_letter(&rev, &rd_rev, use_stdout,
>> >>  				  origin, nr, list, branch_name, quiet);
>> >>  		print_bases(&bases, rev.diffopt.file);
>> >>  		print_signature(rev.diffopt.file);
>> >> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
>> >> index bc5facc1cd..6916103888 100755
>> >> --- a/t/t3206-range-diff.sh
>> >> +++ b/t/t3206-range-diff.sh
>> >> @@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
>> >>  		--range-diff=topic~..topic changed~..changed >actual.raw &&
>> >>  	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
>> >>  	sed -e "s|:$||" >expect <<-\EOF &&
>> >> +	1:  a63e992 ! 1:  d966c5c s/12/B/
>> >> +	    @@ -8,7 +8,7 @@
>> >> +	     @@
>> >> +	      9
>> >> +	      10
>> >> +	    - B
>> >> +	    + BB
>> >> +	     -12
>> >> +	     +B
>> >> +	      13
>> >> +	-- :
>> >> +	EOF
>> >> +	test_cmp expect actual.range-diff &&
>> >> +	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
>> >> +	sed -e "s|:$||" >expect <<-\EOF &&
>> >> +	--- a/file
>> >> +	+++ b/file
>> >> +	@@ -12 +12 @@ BB
>> >> +	-12
>> >> +	+B
>> >> +	-- :
>> >> +	EOF
>> >> +	test_cmp expect actual.diff &&
>> >> +
>> >> +	# -U0 & --range-diff-U0
>> >> +	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
>> >> +		--range-diff=topic~..topic changed~..changed >actual.raw &&
>> >> +	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
>> >> +	sed -e "s|:$||" >expect <<-\EOF &&
>> >>  	1:  a63e992 ! 1:  d966c5c s/12/B/
>> >>  	    @@ -11 +11 @@
>> >>  	    - B
>> >> @@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
>> >>  	test_cmp expect actual.diff
>> >>  '
>> >>
>> >> +test_expect_success 'format-patch option parsing with --range-diff-*' '
>> >> +	test_must_fail git format-patch --stdout --unknown \
>> >> +		master..unmodified 2>stderr &&
>> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr &&
>> >> +	test_must_fail git format-patch --stdout --range-diff-unknown \
>> >> +		master..unmodified 2>stderr &&
>> >> +	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
>> >> +	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
>> >> +		master..unmodified 2>stderr &&
>> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr
>> >> +'
>> >> +
>> >>  test_done
>> >> --
>> >> 2.20.0.rc1.387.gf8505762e3
>> >>
>> >>
>>

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Johannes Schindelin]

Hi Ian,

On Thu, 29 Nov 2018, Ian Jackson wrote:

> Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):
> > if you could pry more information (or better information) out of that bug
> > reporter, that would be nice. Apparently my email address is blacklisted
> > by his mail provider, so he is unlikely to have received my previous mail
> > (nor will he receive this one, I am sure).
> 
> (I did receive this mail.  Sorry for the inconvenience, which sadly is
> inevitable occasionally in the modern email world.  FTR in future feel
> free to send the bounce to postmaster@chiark and I will make a
> you-shaped hole in my spamfilter.  Also with Debian bugs you can
> launder your messages by, eg, emailing 914695-submitter@bugs.)

Right. I myself have plenty of email-related problems that seem to crop up
this year in particular.

> > > > At https://bugs.debian.org/914695 is a report of a test regression in
> > > > an outside project that is very likely to have been triggered by the
> > > > new faster rebase code.
> 
> As I wrote in the bug report last night:
> 
>  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914695#15
> 
>  I have investigated and the bug seems to be that git-rebase --onto now
>  fails to honour GIT_REFLOG_ACTION for the initial checkout.
> 
>  In a successful run with older git I get a reflog like this:
> 
>    4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
>    4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file
>    cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
>    0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
>    29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3
>    85e0c46 HEAD@{5}: debrebase: launder for new upstream
> 
>  With a newer git I get this:
> 
>    6d3fb91 HEAD@{0}: rebase finished: returning to refs/heads/master
>    6d3fb91 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file
>    86c0721 HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
>    50ba56c HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
>    8272825 HEAD@{4}: rebase: checkout 8272825bb4ff6eba89afa936e32b6460f963a36a
>    c78db55 HEAD@{5}: debrebase: launder for new upstream
> 
>  This breaks the test because my test suite is checking that I set
>  GIT_REFLOG_ACTION appropriately.
> 
>  If you want I can provide a minimal test case but this should suffice
>  to see the bug I hope...

This should be plenty for me to get going. Thank you!

Ciao,
Johannes

> 
> Regards
> Ian.
> 
> -- 
> Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.
> 
> If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
> a private address which bypasses my fierce spamfilter.
> 

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 12845 bytes --]

Hi Ævar,

On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> On Thu, Nov 29 2018, Johannes Schindelin wrote:
> 
> > On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
> >
> >> On Thu, Nov 29 2018, Johannes Schindelin wrote:
> >>
> >> > On Wed, 28 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
> >> >
> >> >> Change the semantics of the "--range-diff" option so that the regular
> >> >> diff options can be provided separately for the range-diff and the
> >> >> patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
> >> >> "format-patch" to provide different context for the range-diff and the
> >> >> patch. This wasn't possible before.
> >> >
> >> > I really, really dislike the `--range-diff-<random-thing>`. We have
> >> > precedent for passing optional arguments that are passed to some other
> >> > command, so a much more logical and consistent convention would be to use
> >> > `--range-diff[=<diff-option>..]`, allowing all of the diff options that
> >> > you might want to pass to the outer diff in one go rather than having a
> >> > lengthy string of `--range-diff-this` and `--range-diff-that` options.
> >>
> >> Where do we pass those sorts of arguments?
> >>
> >> Reasons I did it this way:
> >>
> >>  a) Passing it as one option will require the user to double-quote those
> >>     options that take quoted arguments (e.g. --word-diff-regex), which I
> >>     thought sucked more than the prefix. On the implementation side we
> >>     couldn't leave the parsing of the command-line to the shell anymore.
> >>
> >>  b) I think people will want to tweak this very rarely, much more rarely
> >>     than e.g. -U10 in format-patch itself, so having something long-ish
> >>     doesn't sound bad.
> >
> > Hmm. I still don't like it. It sets a precedent, and we simply do not do
> > it that way in other circumstances (most obvious would be the -X merge
> > options). The more divergent user interfaces for the same sort of thing
> > are, the more brain cycles you force users to spend on navigating said
> > interfaces.
> 
> Yeah it sucks, I just think it sucks less than the alternative :)
> I.e. I'm not picky about --range-diff-* prefix the name, but I think
> doing our own shell parsing would be nasty.

What prevents you from using `sq_dequote_to_argv()`?

> >> > I only had time to skim the patch, and I have to wonder why you pass
> >> > around full-blown `rev_info` structs for range diff (and with that really
> >> > awful name `rd_rev`) rather than just the `diff_options` that you
> >> > *actually* care about?
> >>
> >> Because setup_revisions() which does all the command-line parsing needs
> >> a rev_info, so even if we only need the diffopt in the end we need to
> >> initiate the whole thing.
> >>
> >> Suggestions for a better varibale name most welcome.
> >
> > `range_diff_revs`
> >
> > And you do not need to pass around the whole thing. You can easily pass
> > `&range_diff_revs.diffopt`.
> >
> > Don't pass around what you do not need to pass around.
> 
> Ah, you mean internally in log.c, yes that makes sense. I thought you
> meant just pass diffopt to setup_revisions() (which needs the containing
> struct). Willdo.

Thanks,
Dscho

> 
> > Ciao,
> > Dscho
> >
> >>
> >> > Ciao,
> >> > Dscho
> >> >
> >> >>
> >> >> Ever since the "--range-diff" option was added in
> >> >> 31e2617a5f ("format-patch: add --range-diff option to embed diff in
> >> >> cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff
> >> >> machinery has been the one we get from "format-patch"'s own
> >> >> setup_revisions().
> >> >>
> >> >> This sort of thing is unique among the log-like commands in
> >> >> builtin/log.c, no command than format-patch will embed the output of
> >> >> another log-like command. Since the "rev->diffopt" is reused we need
> >> >> to munge it before we pass it to show_range_diff(). See
> >> >> 43dafc4172 ("format-patch: don't include --stat with --range-diff
> >> >> output", 2018-11-22) for a related regression fix which is being
> >> >> mostly reverted here.
> >> >>
> >> >> Implementation notes: 1) We're not bothering with the full teardown
> >> >> around die() and will leak memory, but it's too much boilerplate to do
> >> >> all the frees with/without the die() and not worth it. 2) We call
> >> >> repo_init_revisions() for "rd_rev" even though we could get away with
> >> >> a shallow copy like the code we're replacing (and which
> >> >> show_range_diff() itself does). This is to make this code more easily
> >> >> understood.
> >> >>
> >> >> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> >> >> ---
> >> >>  Documentation/git-format-patch.txt | 10 ++++++-
> >> >>  builtin/log.c                      | 42 +++++++++++++++++++++++-------
> >> >>  t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
> >> >>  3 files changed, 82 insertions(+), 11 deletions(-)
> >> >>
> >> >> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
> >> >> index aba4c5febe..6c048f415f 100644
> >> >> --- a/Documentation/git-format-patch.txt
> >> >> +++ b/Documentation/git-format-patch.txt
> >> >> @@ -24,7 +24,8 @@ SYNOPSIS
> >> >>  		   [--to=<email>] [--cc=<email>]
> >> >>  		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
> >> >>  		   [--interdiff=<previous>]
> >> >> -		   [--range-diff=<previous> [--creation-factor=<percent>]]
> >> >> +		   [--range-diff=<previous> [--creation-factor=<percent>]
> >> >> +		      [--range-diff<common diff option>]]
> >> >>  		   [--progress]
> >> >>  		   [<common diff options>]
> >> >>  		   [ <since> | <revision range> ]
> >> >> @@ -257,6 +258,13 @@ feeding the result to `git send-email`.
> >> >>  	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
> >> >>  	for details.
> >> >>
> >> >> +--range-diff<common diff option>::
> >> >> +	Other options prefixed with `--range-diff` are stripped of
> >> >> +	that prefix and passed as-is to the diff machinery used to
> >> >> +	generate the range-diff, e.g. `--range-diff-U0` and
> >> >> +	`--range-diff--no-color`. This allows for adjusting the format
> >> >> +	of the range-diff independently from the patch itself.
> >> >> +
> >> >>  --notes[=<ref>]::
> >> >>  	Append the notes (see linkgit:git-notes[1]) for the commit
> >> >>  	after the three-dash line.
> >> >> diff --git a/builtin/log.c b/builtin/log.c
> >> >> index 02d88fa233..7658e56ecc 100644
> >> >> --- a/builtin/log.c
> >> >> +++ b/builtin/log.c
> >> >> @@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
> >> >>  	fprintf(rev->diffopt.file, "\n");
> >> >>  }
> >> >>
> >> >> -static void make_cover_letter(struct rev_info *rev, int use_stdout,
> >> >> +static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
> >> >> +			      int use_stdout,
> >> >>  			      struct commit *origin,
> >> >>  			      int nr, struct commit **list,
> >> >>  			      const char *branch_name,
> >> >> @@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
> >> >>  	}
> >> >>
> >> >>  	if (rev->rdiff1) {
> >> >> -		struct diff_options opts;
> >> >> -		memcpy(&opts, &rev->diffopt, sizeof(opts));
> >> >> -		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
> >> >> -
> >> >>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
> >> >>  		show_range_diff(rev->rdiff1, rev->rdiff2,
> >> >> -				rev->creation_factor, 1, &opts);
> >> >> +				rev->creation_factor, 1, &rd_rev->diffopt);
> >> >>  	}
> >> >>  }
> >> >>
> >> >> @@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >> >>  	struct commit *commit;
> >> >>  	struct commit **list = NULL;
> >> >>  	struct rev_info rev;
> >> >> +	struct rev_info rd_rev;
> >> >>  	struct setup_revision_opt s_r_opt;
> >> >>  	int nr = 0, total, i;
> >> >>  	int use_stdout = 0;
> >> >> @@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >> >>  	init_log_defaults();
> >> >>  	git_config(git_format_config, NULL);
> >> >>  	repo_init_revisions(the_repository, &rev, prefix);
> >> >> +	repo_init_revisions(the_repository, &rd_rev, prefix);
> >> >>  	rev.commit_format = CMIT_FMT_EMAIL;
> >> >>  	rev.expand_tabs_in_log_default = 0;
> >> >>  	rev.verbose_header = 1;
> >> >> @@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >> >>  	rev.preserve_subject = keep_subject;
> >> >>
> >> >>  	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
> >> >> -	if (argc > 1)
> >> >> -		die(_("unrecognized argument: %s"), argv[1]);
> >> >> +	if (argc > 1) {
> >> >> +		struct argv_array args = ARGV_ARRAY_INIT;
> >> >> +		const char *prefix = "--range-diff";
> >> >> +		int have_prefix = 0;
> >> >> +
> >> >> +		for (i = 0; i < argc; i++) {
> >> >> +			struct strbuf sb = STRBUF_INIT;
> >> >> +			char *str;
> >> >> +
> >> >> +			strbuf_addstr(&sb, argv[i]);
> >> >> +			if (starts_with(argv[i], prefix)) {
> >> >> +				have_prefix = 1;
> >> >> +				strbuf_remove(&sb, 0, strlen(prefix));
> >> >> +			}
> >> >> +			str = strbuf_detach(&sb, NULL);
> >> >> +			strbuf_release(&sb);
> >> >> +
> >> >> +			argv_array_push(&args, str);
> >> >> +		}
> >> >> +
> >> >> +		if (!have_prefix)
> >> >> +			die(_("unrecognized argument: %s"), argv[1]);
> >> >> +		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
> >> >> +		if (argc > 1)
> >> >> +			die(_("unrecognized argument: %s"), argv[1]);
> >> >> +	}
> >> >>
> >> >>  	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
> >> >>  		die(_("--name-only does not make sense"));
> >> >> @@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >> >>  	if (!use_patch_format &&
> >> >>  		(!rev.diffopt.output_format ||
> >> >>  		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
> >> >> -		/* Needs to be mirrored in show_range_diff() invocation */
> >> >>  		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
> >> >>  	if (!rev.diffopt.stat_width)
> >> >>  		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
> >> >> @@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
> >> >>  	if (cover_letter) {
> >> >>  		if (thread)
> >> >>  			gen_message_id(&rev, "cover");
> >> >> -		make_cover_letter(&rev, use_stdout,
> >> >> +		make_cover_letter(&rev, &rd_rev, use_stdout,
> >> >>  				  origin, nr, list, branch_name, quiet);
> >> >>  		print_bases(&bases, rev.diffopt.file);
> >> >>  		print_signature(rev.diffopt.file);
> >> >> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> >> >> index bc5facc1cd..6916103888 100755
> >> >> --- a/t/t3206-range-diff.sh
> >> >> +++ b/t/t3206-range-diff.sh
> >> >> @@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
> >> >>  		--range-diff=topic~..topic changed~..changed >actual.raw &&
> >> >>  	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
> >> >>  	sed -e "s|:$||" >expect <<-\EOF &&
> >> >> +	1:  a63e992 ! 1:  d966c5c s/12/B/
> >> >> +	    @@ -8,7 +8,7 @@
> >> >> +	     @@
> >> >> +	      9
> >> >> +	      10
> >> >> +	    - B
> >> >> +	    + BB
> >> >> +	     -12
> >> >> +	     +B
> >> >> +	      13
> >> >> +	-- :
> >> >> +	EOF
> >> >> +	test_cmp expect actual.range-diff &&
> >> >> +	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
> >> >> +	sed -e "s|:$||" >expect <<-\EOF &&
> >> >> +	--- a/file
> >> >> +	+++ b/file
> >> >> +	@@ -12 +12 @@ BB
> >> >> +	-12
> >> >> +	+B
> >> >> +	-- :
> >> >> +	EOF
> >> >> +	test_cmp expect actual.diff &&
> >> >> +
> >> >> +	# -U0 & --range-diff-U0
> >> >> +	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
> >> >> +		--range-diff=topic~..topic changed~..changed >actual.raw &&
> >> >> +	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
> >> >> +	sed -e "s|:$||" >expect <<-\EOF &&
> >> >>  	1:  a63e992 ! 1:  d966c5c s/12/B/
> >> >>  	    @@ -11 +11 @@
> >> >>  	    - B
> >> >> @@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
> >> >>  	test_cmp expect actual.diff
> >> >>  '
> >> >>
> >> >> +test_expect_success 'format-patch option parsing with --range-diff-*' '
> >> >> +	test_must_fail git format-patch --stdout --unknown \
> >> >> +		master..unmodified 2>stderr &&
> >> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr &&
> >> >> +	test_must_fail git format-patch --stdout --range-diff-unknown \
> >> >> +		master..unmodified 2>stderr &&
> >> >> +	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
> >> >> +	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
> >> >> +		master..unmodified 2>stderr &&
> >> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr
> >> >> +'
> >> >> +
> >> >>  test_done
> >> >> --
> >> >> 2.20.0.rc1.387.gf8505762e3
> >> >>
> >> >>
> >>
> 

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Ian Jackson]

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):
> >  In a successful run with older git I get a reflog like this:
> > 
> >    4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
> >    4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file
> >    cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
> >    0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
> >    29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3
> >    85e0c46 HEAD@{5}: debrebase: launder for new upstream
...
> >  This breaks the test because my test suite is checking that I set
> >  GIT_REFLOG_ACTION appropriately.
> > 
> >  If you want I can provide a minimal test case but this should suffice
> >  to see the bug I hope...
> 
> This should be plenty for me to get going. Thank you!

Happy hunting.

While you're looking at this, I observe that the fact that the `rebase
finished' message also does not honour GIT_REFLOG_ACTION appears to be
a pre-existing bug.

(In general one often can't rely on GIT_REFLOG_ACTION still being set
because the rebase might have been interrupted and restarted, which I
think is why my test case looks for it in the initial `checkout'
message.)

Regards,
Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Ævar Arnfjörð Bjarmason]

On Thu, Nov 29 2018, Johannes Schindelin wrote:

> Hi Ævar,
>
> On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
>
>> On Thu, Nov 29 2018, Johannes Schindelin wrote:
>>
>> > On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
>> >
>> >> On Thu, Nov 29 2018, Johannes Schindelin wrote:
>> >>
>> >> > On Wed, 28 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
>> >> >
>> >> >> Change the semantics of the "--range-diff" option so that the regular
>> >> >> diff options can be provided separately for the range-diff and the
>> >> >> patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
>> >> >> "format-patch" to provide different context for the range-diff and the
>> >> >> patch. This wasn't possible before.
>> >> >
>> >> > I really, really dislike the `--range-diff-<random-thing>`. We have
>> >> > precedent for passing optional arguments that are passed to some other
>> >> > command, so a much more logical and consistent convention would be to use
>> >> > `--range-diff[=<diff-option>..]`, allowing all of the diff options that
>> >> > you might want to pass to the outer diff in one go rather than having a
>> >> > lengthy string of `--range-diff-this` and `--range-diff-that` options.
>> >>
>> >> Where do we pass those sorts of arguments?
>> >>
>> >> Reasons I did it this way:
>> >>
>> >>  a) Passing it as one option will require the user to double-quote those
>> >>     options that take quoted arguments (e.g. --word-diff-regex), which I
>> >>     thought sucked more than the prefix. On the implementation side we
>> >>     couldn't leave the parsing of the command-line to the shell anymore.
>> >>
>> >>  b) I think people will want to tweak this very rarely, much more rarely
>> >>     than e.g. -U10 in format-patch itself, so having something long-ish
>> >>     doesn't sound bad.
>> >
>> > Hmm. I still don't like it. It sets a precedent, and we simply do not do
>> > it that way in other circumstances (most obvious would be the -X merge
>> > options). The more divergent user interfaces for the same sort of thing
>> > are, the more brain cycles you force users to spend on navigating said
>> > interfaces.
>>
>> Yeah it sucks, I just think it sucks less than the alternative :)
>> I.e. I'm not picky about --range-diff-* prefix the name, but I think
>> doing our own shell parsing would be nasty.
>
> What prevents you from using `sq_dequote_to_argv()`?

I mean not just nasty in terms of implementation, yeah we could do it,
but also a nasty UX for things like --word-diff-regex. I.e. instead of:

    --range-diff-word-diff-regex='[0-9"]'

You need:

    --range-diff-opts="--word-diff-regex='[0-9\"]'"

Now admittedly that in itself isn't very painful *in this case*, but in
terms of precedent I really dislike that option, i.e. git having some
mode where I need to work to escape input to pass to another command.

Not saying that this --range-diff-* thing is what we should go for, but
surely we can find some way to do deal with this that doesn't involve
the user needing to escape stuff like this.

It also has other downstream effects in the UI, e.g. it's presumably
easy to teach the bash completion that a --foo=XYZ option is also called
--some-prefix--foo=XYZ and to enable completion for that, less so for
making it smart enough to complete "--some-prefix-opts="--foo=<TAB>".

>> >> > I only had time to skim the patch, and I have to wonder why you pass
>> >> > around full-blown `rev_info` structs for range diff (and with that really
>> >> > awful name `rd_rev`) rather than just the `diff_options` that you
>> >> > *actually* care about?
>> >>
>> >> Because setup_revisions() which does all the command-line parsing needs
>> >> a rev_info, so even if we only need the diffopt in the end we need to
>> >> initiate the whole thing.
>> >>
>> >> Suggestions for a better varibale name most welcome.
>> >
>> > `range_diff_revs`
>> >
>> > And you do not need to pass around the whole thing. You can easily pass
>> > `&range_diff_revs.diffopt`.
>> >
>> > Don't pass around what you do not need to pass around.
>>
>> Ah, you mean internally in log.c, yes that makes sense. I thought you
>> meant just pass diffopt to setup_revisions() (which needs the containing
>> struct). Willdo.
>
> Thanks,
> Dscho
>
>>
>> > Ciao,
>> > Dscho
>> >
>> >>
>> >> > Ciao,
>> >> > Dscho
>> >> >
>> >> >>
>> >> >> Ever since the "--range-diff" option was added in
>> >> >> 31e2617a5f ("format-patch: add --range-diff option to embed diff in
>> >> >> cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff
>> >> >> machinery has been the one we get from "format-patch"'s own
>> >> >> setup_revisions().
>> >> >>
>> >> >> This sort of thing is unique among the log-like commands in
>> >> >> builtin/log.c, no command than format-patch will embed the output of
>> >> >> another log-like command. Since the "rev->diffopt" is reused we need
>> >> >> to munge it before we pass it to show_range_diff(). See
>> >> >> 43dafc4172 ("format-patch: don't include --stat with --range-diff
>> >> >> output", 2018-11-22) for a related regression fix which is being
>> >> >> mostly reverted here.
>> >> >>
>> >> >> Implementation notes: 1) We're not bothering with the full teardown
>> >> >> around die() and will leak memory, but it's too much boilerplate to do
>> >> >> all the frees with/without the die() and not worth it. 2) We call
>> >> >> repo_init_revisions() for "rd_rev" even though we could get away with
>> >> >> a shallow copy like the code we're replacing (and which
>> >> >> show_range_diff() itself does). This is to make this code more easily
>> >> >> understood.
>> >> >>
>> >> >> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> >> >> ---
>> >> >>  Documentation/git-format-patch.txt | 10 ++++++-
>> >> >>  builtin/log.c                      | 42 +++++++++++++++++++++++-------
>> >> >>  t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
>> >> >>  3 files changed, 82 insertions(+), 11 deletions(-)
>> >> >>
>> >> >> diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
>> >> >> index aba4c5febe..6c048f415f 100644
>> >> >> --- a/Documentation/git-format-patch.txt
>> >> >> +++ b/Documentation/git-format-patch.txt
>> >> >> @@ -24,7 +24,8 @@ SYNOPSIS
>> >> >>  		   [--to=<email>] [--cc=<email>]
>> >> >>  		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
>> >> >>  		   [--interdiff=<previous>]
>> >> >> -		   [--range-diff=<previous> [--creation-factor=<percent>]]
>> >> >> +		   [--range-diff=<previous> [--creation-factor=<percent>]
>> >> >> +		      [--range-diff<common diff option>]]
>> >> >>  		   [--progress]
>> >> >>  		   [<common diff options>]
>> >> >>  		   [ <since> | <revision range> ]
>> >> >> @@ -257,6 +258,13 @@ feeding the result to `git send-email`.
>> >> >>  	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
>> >> >>  	for details.
>> >> >>
>> >> >> +--range-diff<common diff option>::
>> >> >> +	Other options prefixed with `--range-diff` are stripped of
>> >> >> +	that prefix and passed as-is to the diff machinery used to
>> >> >> +	generate the range-diff, e.g. `--range-diff-U0` and
>> >> >> +	`--range-diff--no-color`. This allows for adjusting the format
>> >> >> +	of the range-diff independently from the patch itself.
>> >> >> +
>> >> >>  --notes[=<ref>]::
>> >> >>  	Append the notes (see linkgit:git-notes[1]) for the commit
>> >> >>  	after the three-dash line.
>> >> >> diff --git a/builtin/log.c b/builtin/log.c
>> >> >> index 02d88fa233..7658e56ecc 100644
>> >> >> --- a/builtin/log.c
>> >> >> +++ b/builtin/log.c
>> >> >> @@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
>> >> >>  	fprintf(rev->diffopt.file, "\n");
>> >> >>  }
>> >> >>
>> >> >> -static void make_cover_letter(struct rev_info *rev, int use_stdout,
>> >> >> +static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
>> >> >> +			      int use_stdout,
>> >> >>  			      struct commit *origin,
>> >> >>  			      int nr, struct commit **list,
>> >> >>  			      const char *branch_name,
>> >> >> @@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
>> >> >>  	}
>> >> >>
>> >> >>  	if (rev->rdiff1) {
>> >> >> -		struct diff_options opts;
>> >> >> -		memcpy(&opts, &rev->diffopt, sizeof(opts));
>> >> >> -		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
>> >> >> -
>> >> >>  		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
>> >> >>  		show_range_diff(rev->rdiff1, rev->rdiff2,
>> >> >> -				rev->creation_factor, 1, &opts);
>> >> >> +				rev->creation_factor, 1, &rd_rev->diffopt);
>> >> >>  	}
>> >> >>  }
>> >> >>
>> >> >> @@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >> >>  	struct commit *commit;
>> >> >>  	struct commit **list = NULL;
>> >> >>  	struct rev_info rev;
>> >> >> +	struct rev_info rd_rev;
>> >> >>  	struct setup_revision_opt s_r_opt;
>> >> >>  	int nr = 0, total, i;
>> >> >>  	int use_stdout = 0;
>> >> >> @@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >> >>  	init_log_defaults();
>> >> >>  	git_config(git_format_config, NULL);
>> >> >>  	repo_init_revisions(the_repository, &rev, prefix);
>> >> >> +	repo_init_revisions(the_repository, &rd_rev, prefix);
>> >> >>  	rev.commit_format = CMIT_FMT_EMAIL;
>> >> >>  	rev.expand_tabs_in_log_default = 0;
>> >> >>  	rev.verbose_header = 1;
>> >> >> @@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >> >>  	rev.preserve_subject = keep_subject;
>> >> >>
>> >> >>  	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
>> >> >> -	if (argc > 1)
>> >> >> -		die(_("unrecognized argument: %s"), argv[1]);
>> >> >> +	if (argc > 1) {
>> >> >> +		struct argv_array args = ARGV_ARRAY_INIT;
>> >> >> +		const char *prefix = "--range-diff";
>> >> >> +		int have_prefix = 0;
>> >> >> +
>> >> >> +		for (i = 0; i < argc; i++) {
>> >> >> +			struct strbuf sb = STRBUF_INIT;
>> >> >> +			char *str;
>> >> >> +
>> >> >> +			strbuf_addstr(&sb, argv[i]);
>> >> >> +			if (starts_with(argv[i], prefix)) {
>> >> >> +				have_prefix = 1;
>> >> >> +				strbuf_remove(&sb, 0, strlen(prefix));
>> >> >> +			}
>> >> >> +			str = strbuf_detach(&sb, NULL);
>> >> >> +			strbuf_release(&sb);
>> >> >> +
>> >> >> +			argv_array_push(&args, str);
>> >> >> +		}
>> >> >> +
>> >> >> +		if (!have_prefix)
>> >> >> +			die(_("unrecognized argument: %s"), argv[1]);
>> >> >> +		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
>> >> >> +		if (argc > 1)
>> >> >> +			die(_("unrecognized argument: %s"), argv[1]);
>> >> >> +	}
>> >> >>
>> >> >>  	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
>> >> >>  		die(_("--name-only does not make sense"));
>> >> >> @@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >> >>  	if (!use_patch_format &&
>> >> >>  		(!rev.diffopt.output_format ||
>> >> >>  		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
>> >> >> -		/* Needs to be mirrored in show_range_diff() invocation */
>> >> >>  		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
>> >> >>  	if (!rev.diffopt.stat_width)
>> >> >>  		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
>> >> >> @@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
>> >> >>  	if (cover_letter) {
>> >> >>  		if (thread)
>> >> >>  			gen_message_id(&rev, "cover");
>> >> >> -		make_cover_letter(&rev, use_stdout,
>> >> >> +		make_cover_letter(&rev, &rd_rev, use_stdout,
>> >> >>  				  origin, nr, list, branch_name, quiet);
>> >> >>  		print_bases(&bases, rev.diffopt.file);
>> >> >>  		print_signature(rev.diffopt.file);
>> >> >> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
>> >> >> index bc5facc1cd..6916103888 100755
>> >> >> --- a/t/t3206-range-diff.sh
>> >> >> +++ b/t/t3206-range-diff.sh
>> >> >> @@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
>> >> >>  		--range-diff=topic~..topic changed~..changed >actual.raw &&
>> >> >>  	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
>> >> >>  	sed -e "s|:$||" >expect <<-\EOF &&
>> >> >> +	1:  a63e992 ! 1:  d966c5c s/12/B/
>> >> >> +	    @@ -8,7 +8,7 @@
>> >> >> +	     @@
>> >> >> +	      9
>> >> >> +	      10
>> >> >> +	    - B
>> >> >> +	    + BB
>> >> >> +	     -12
>> >> >> +	     +B
>> >> >> +	      13
>> >> >> +	-- :
>> >> >> +	EOF
>> >> >> +	test_cmp expect actual.range-diff &&
>> >> >> +	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
>> >> >> +	sed -e "s|:$||" >expect <<-\EOF &&
>> >> >> +	--- a/file
>> >> >> +	+++ b/file
>> >> >> +	@@ -12 +12 @@ BB
>> >> >> +	-12
>> >> >> +	+B
>> >> >> +	-- :
>> >> >> +	EOF
>> >> >> +	test_cmp expect actual.diff &&
>> >> >> +
>> >> >> +	# -U0 & --range-diff-U0
>> >> >> +	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
>> >> >> +		--range-diff=topic~..topic changed~..changed >actual.raw &&
>> >> >> +	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
>> >> >> +	sed -e "s|:$||" >expect <<-\EOF &&
>> >> >>  	1:  a63e992 ! 1:  d966c5c s/12/B/
>> >> >>  	    @@ -11 +11 @@
>> >> >>  	    - B
>> >> >> @@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
>> >> >>  	test_cmp expect actual.diff
>> >> >>  '
>> >> >>
>> >> >> +test_expect_success 'format-patch option parsing with --range-diff-*' '
>> >> >> +	test_must_fail git format-patch --stdout --unknown \
>> >> >> +		master..unmodified 2>stderr &&
>> >> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr &&
>> >> >> +	test_must_fail git format-patch --stdout --range-diff-unknown \
>> >> >> +		master..unmodified 2>stderr &&
>> >> >> +	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
>> >> >> +	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
>> >> >> +		master..unmodified 2>stderr &&
>> >> >> +	test_i18ngrep "unrecognized argument: --unknown" stderr
>> >> >> +'
>> >> >> +
>> >> >>  test_done
>> >> >> --
>> >> >> 2.20.0.rc1.387.gf8505762e3
>> >> >>
>> >> >>
>> >>
>>

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Johannes Schindelin]

Hi Ian,

On Thu, 29 Nov 2018, Ian Jackson wrote:

> Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):
> > >  In a successful run with older git I get a reflog like this:
> > > 
> > >    4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
> > >    4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file
> > >    cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
> > >    0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
> > >    29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3
> > >    85e0c46 HEAD@{5}: debrebase: launder for new upstream
> ...
> > >  This breaks the test because my test suite is checking that I set
> > >  GIT_REFLOG_ACTION appropriately.
> > > 
> > >  If you want I can provide a minimal test case but this should suffice
> > >  to see the bug I hope...
> > 
> > This should be plenty for me to get going. Thank you!
> 
> Happy hunting.

I'll have to take a (lengthy) dinner break now, but this is what I have so
far: a regression test that verifies the breakage (see the
`fix-reflog-action` branch at https://github.com/dscho/git). I'll continue
after dinner and am confident that this bug will be fixed within the next
four hours.

> While you're looking at this, I observe that the fact that the `rebase
> finished' message also does not honour GIT_REFLOG_ACTION appears to be
> a pre-existing bug.

I noticed that, too, but at this point I am only fixing regressions. We
can try to fix this long-standing bug in the v2.20 cycle.

Ciao,
Johannes

> (In general one often can't rely on GIT_REFLOG_ACTION still being set
> because the rebase might have been interrupted and restarted, which I
> think is why my test case looks for it in the initial `checkout'
> message.)
> 
> Regards,
> Ian.
> 
> -- 
> Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.
> 
> If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
> a private address which bypasses my fierce spamfilter.
> 

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

[Ian Jackson]

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):
> I'll have to take a (lengthy) dinner break now, but this is what I have so
> far: a regression test that verifies the breakage (see the
> `fix-reflog-action` branch at https://github.com/dscho/git). I'll continue
> after dinner and am confident that this bug will be fixed within the next
> four hours.

That seems super speedy to me!

When you have a fix I will leave it up to the Debian git maintainers
to decide whether they want to cherry pick your fix into their
package, or await an updated upstream branch with rc, or what.

> [Ian:]
> > While you're looking at this, I observe that the fact that the `rebase
> > finished' message also does not honour GIT_REFLOG_ACTION appears to be
> > a pre-existing bug.
> 
> I noticed that, too, but at this point I am only fixing regressions. We
> can try to fix this long-standing bug in the v2.20 cycle.

Right.

Thanks,
Ian.

-- 
Ian Jackson <ijackson@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 4326 bytes --]

Hi Ævar,

On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:

> On Thu, Nov 29 2018, Johannes Schindelin wrote:
> 
> > On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
> >
> >> On Thu, Nov 29 2018, Johannes Schindelin wrote:
> >>
> >> > On Thu, 29 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
> >> >
> >> >> On Thu, Nov 29 2018, Johannes Schindelin wrote:
> >> >>
> >> >> > On Wed, 28 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
> >> >> >
> >> >> >> Change the semantics of the "--range-diff" option so that the regular
> >> >> >> diff options can be provided separately for the range-diff and the
> >> >> >> patch. This allows for supplying e.g. --range-diff-U0 and -U1 to
> >> >> >> "format-patch" to provide different context for the range-diff and the
> >> >> >> patch. This wasn't possible before.
> >> >> >
> >> >> > I really, really dislike the `--range-diff-<random-thing>`. We have
> >> >> > precedent for passing optional arguments that are passed to some other
> >> >> > command, so a much more logical and consistent convention would be to use
> >> >> > `--range-diff[=<diff-option>..]`, allowing all of the diff options that
> >> >> > you might want to pass to the outer diff in one go rather than having a
> >> >> > lengthy string of `--range-diff-this` and `--range-diff-that` options.
> >> >>
> >> >> Where do we pass those sorts of arguments?
> >> >>
> >> >> Reasons I did it this way:
> >> >>
> >> >>  a) Passing it as one option will require the user to double-quote those
> >> >>     options that take quoted arguments (e.g. --word-diff-regex), which I
> >> >>     thought sucked more than the prefix. On the implementation side we
> >> >>     couldn't leave the parsing of the command-line to the shell anymore.
> >> >>
> >> >>  b) I think people will want to tweak this very rarely, much more rarely
> >> >>     than e.g. -U10 in format-patch itself, so having something long-ish
> >> >>     doesn't sound bad.
> >> >
> >> > Hmm. I still don't like it. It sets a precedent, and we simply do not do
> >> > it that way in other circumstances (most obvious would be the -X merge
> >> > options). The more divergent user interfaces for the same sort of thing
> >> > are, the more brain cycles you force users to spend on navigating said
> >> > interfaces.
> >>
> >> Yeah it sucks, I just think it sucks less than the alternative :)
> >> I.e. I'm not picky about --range-diff-* prefix the name, but I think
> >> doing our own shell parsing would be nasty.
> >
> > What prevents you from using `sq_dequote_to_argv()`?
> 
> I mean not just nasty in terms of implementation, yeah we could do it,
> but also a nasty UX for things like --word-diff-regex. I.e. instead of:
> 
>     --range-diff-word-diff-regex='[0-9"]'
> 
> You need:
> 
>     --range-diff-opts="--word-diff-regex='[0-9\"]'"

Really? I think that would not work. It would pass the single quotes as
part of the regex to the diff machinery.

Or maybe not. But the extra quotes do not strike me as necessary, as there
is no shell script involved (thank deity!) after `git range-diff` parsed
the options.

> Now admittedly that in itself isn't very painful *in this case*, but in
> terms of precedent I really dislike that option, i.e. git having some
> mode where I need to work to escape input to pass to another command.
> 
> Not saying that this --range-diff-* thing is what we should go for, but
> surely we can find some way to do deal with this that doesn't involve
> the user needing to escape stuff like this.
> 
> It also has other downstream effects in the UI, e.g. it's presumably
> easy to teach the bash completion that a --foo=XYZ option is also called
> --some-prefix--foo=XYZ and to enable completion for that, less so for
> making it smart enough to complete "--some-prefix-opts="--foo=<TAB>".

These are all good points, and need proper discussion.

Sadly, all that time needed for a proper discussion is not left before
v2.20.0 is supposed to come out.

Quite honestly, I think what we will have to do is to describe in the
documentation of `format-patch`'s `--range-diff` option that the exact
user interface how to pass diff options down to `range-diff` is in flux
and not final.

That way, we can give your design the proper treatment, and work together
on making a user interface we all can be happy with.

Ciao,
Dscho

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>> What prevents you from using `sq_dequote_to_argv()`?
>
> I mean not just nasty in terms of implementation, yeah we could do it,
> but also a nasty UX for things like --word-diff-regex. I.e. instead of:
>
>     --range-diff-word-diff-regex='[0-9"]'
>
> You need:
>
>     --range-diff-opts="--word-diff-regex='[0-9\"]'"
>
> Now admittedly that in itself isn't very painful *in this case*, but in
> terms of precedent I really dislike that option, i.e. git having some
> mode where I need to work to escape input to pass to another command.

In addition, sq_dequote are meant to be used on quoted string we
internally produce; I do not think we want to promise that it is
safe to use on a random string that comes from end users.

In any case, I tend to agree with the conclusion in the downthread
by Dscho that we should just clearly mark that invocations of the
"format-patch --range-diff" command with additional diff options is
an experimental feature that may not do anything sensible in the
upcoming release, and declare that the UI to pass diff options to
affect only the range-diff part may later be invented.  IOW, I am
coming a bit stronger than Dscho's suggestion in that we should not
even pretend that we aimed to make the options used for range-diff
customizable when driven from format-patch in the upcoming release,
or aimed to make --range-diff option compatible with other diff
options given to the format-patch command.

I had to delay -rc2 to see these last minute tweaks come to some
reasonable place to stop at, and I do not think we want to delay the
final any longer or destablizing it further by piling last minute
undercooked changes on top.

[PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> In any case, I tend to agree with the conclusion in the downthread
> by Dscho that we should just clearly mark that invocations of the
> "format-patch --range-diff" command with additional diff options is
> an experimental feature that may not do anything sensible in the
> upcoming release, and declare that the UI to pass diff options to
> affect only the range-diff part may later be invented.  IOW, I am
> coming a bit stronger than Dscho's suggestion in that we should not
> even pretend that we aimed to make the options used for range-diff
> customizable when driven from format-patch in the upcoming release,
> or aimed to make --range-diff option compatible with other diff
> options given to the format-patch command.
>
> I had to delay -rc2 to see these last minute tweaks come to some
> reasonable place to stop at, and I do not think we want to delay the
> final any longer or destablizing it further by piling last minute
> undercooked changes on top.

So how about doing this on top of 'master' instead?  As this leaks
*no* information wrt how range-diff machinery should behave from the
format-patch side by not passing any diffopt, as long as the new
code I added to show_range_diff() comes up with a reasonable default
diffopts (for which I really would appreciate extra sets of eyes to
make sure), this change by definition cannot be wrong (famous last
words).

-- >8 --
Subject: format-patch: do not let its diff-options affect --range-diff

Stop leaking how the primary output of format-patch is customized to
the range-diff machinery and instead let the latter use its own
"reasonable default", in order to correct the breakage introduced by
a5170794 ("Merge branch 'ab/range-diff-no-patch'", 2018-11-18) on
the 'master' front.  "git format-patch --range-diff..." without any
weird diff option started to include the "range-diff --stat" output,
which is rather useless right now, that made the whole thing
unusable and this is probably the least disruptive way to whip the
codebase into a shippable shape.

We may want to later make the range-diff driven by format-patch more
configurable, but that would have to wait until we have a good
design.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 Documentation/git-format-patch.txt | 5 +++++
 builtin/log.c                      | 2 +-
 log-tree.c                         | 2 +-
 range-diff.c                       | 6 +++++-
 range-diff.h                       | 5 +++++
 5 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index aba4c5febe..27304428a1 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -250,6 +250,11 @@ feeding the result to `git send-email`.
 	feature/v2`), or a revision range if the two versions of the series are
 	disjoint (for example `git format-patch --cover-letter
 	--range-diff=feature/v1~3..feature/v1 -3 feature/v2`).
++
+Note that diff options passed to the command affect how the primary
+product of `format-patch` is generated, and they are not passed to
+the underlying `range-diff` machinery used to generate the cover-letter
+material (this may change in the future).
 
 --creation-factor=<percent>::
 	Used with `--range-diff`, tweak the heuristic which matches up commits
diff --git a/builtin/log.c b/builtin/log.c
index 0fe6f9ba1e..5ac18e2848 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1096,7 +1096,7 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	if (rev->rdiff1) {
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &rev->diffopt);
+				rev->creation_factor, 1, NULL);
 	}
 }
 
diff --git a/log-tree.c b/log-tree.c
index 7a83e99250..b243779a0b 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -762,7 +762,7 @@ void show_log(struct rev_info *opt)
 		next_commentary_block(opt, NULL);
 		fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
 		show_range_diff(opt->rdiff1, opt->rdiff2,
-				opt->creation_factor, 1, &opt->diffopt);
+				opt->creation_factor, 1, NULL);
 
 		memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
 	}
diff --git a/range-diff.c b/range-diff.c
index 767af8c5bb..8e52a85c19 100644
--- a/range-diff.c
+++ b/range-diff.c
@@ -460,7 +460,11 @@ int show_range_diff(const char *range1, const char *range2,
 		struct diff_options opts;
 		struct strbuf indent = STRBUF_INIT;
 
-		memcpy(&opts, diffopt, sizeof(opts));
+		if (diffopt)
+			memcpy(&opts, diffopt, sizeof(opts));
+		else
+			repo_diff_setup(the_repository, &opts);
+
 		if (!opts.output_format)
 			opts.output_format = DIFF_FORMAT_PATCH;
 		opts.flags.suppress_diff_headers = 1;
diff --git a/range-diff.h b/range-diff.h
index 190593f0c7..08a50b6e98 100644
--- a/range-diff.h
+++ b/range-diff.h
@@ -5,6 +5,11 @@
 
 #define RANGE_DIFF_CREATION_FACTOR_DEFAULT 60
 
+/*
+ * Compare series of commmits in RANGE1 and RANGE2, and emit to the
+ * standard output.  NULL can be passed to DIFFOPT to use the built-in
+ * default.
+ */
 int show_range_diff(const char *range1, const char *range2,
 		    int creation_factor, int dual_color,
 		    struct diff_options *diffopt);

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

>> I had to delay -rc2 to see these last minute tweaks come to some
>> reasonable place to stop at, and I do not think we want to delay the
>> final any longer or destablizing it further by piling last minute
>> undercooked changes on top.
>
> So how about doing this on top of 'master' instead?  As this leaks
> *no* information wrt how range-diff machinery should behave from the
> format-patch side by not passing any diffopt, as long as the new
> code I added to show_range_diff() comes up with a reasonable default
> diffopts (for which I really would appreciate extra sets of eyes to
> make sure), this change by definition cannot be wrong (famous last
> words).

As listed in today's "What's cooking" report, I've merged this to
'next' in today's pushout and planning to have it in the -rc2.  I am
not married to this exact implementation, and I'd welcome to have an
even simpler and less disruptive solution if exists, but I am hoping
that this is a good-enough interim measure for the upcoming release,
until we decide what to do with the customizability of range-diff
driven by format-patch.

In addition to this, I am planning the "rebase --stat" and "reflog
that does not say 'rebase -i' but 'rebase'" fixes merged to 'master'
before cutting -rc2.

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

[Ævar Arnfjörð Bjarmason]

On Fri, Nov 30 2018, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
>
>>> I had to delay -rc2 to see these last minute tweaks come to some
>>> reasonable place to stop at, and I do not think we want to delay the
>>> final any longer or destablizing it further by piling last minute
>>> undercooked changes on top.
>>
>> So how about doing this on top of 'master' instead?  As this leaks
>> *no* information wrt how range-diff machinery should behave from the
>> format-patch side by not passing any diffopt, as long as the new
>> code I added to show_range_diff() comes up with a reasonable default
>> diffopts (for which I really would appreciate extra sets of eyes to
>> make sure), this change by definition cannot be wrong (famous last
>> words).
>
> As listed in today's "What's cooking" report, I've merged this to
> 'next' in today's pushout and planning to have it in the -rc2.  I am
> not married to this exact implementation, and I'd welcome to have an
> even simpler and less disruptive solution if exists, but I am hoping
> that this is a good-enough interim measure for the upcoming release,
> until we decide what to do with the customizability of range-diff
> driven by format-patch.
>
> In addition to this, I am planning the "rebase --stat" and "reflog
> that does not say 'rebase -i' but 'rebase'" fixes merged to 'master'
> before cutting -rc2.

Thanks a lot, yeah having this wait looks good to me.

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

[Eric Sunshine]

On Thu, Nov 29, 2018 at 11:27 PM Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> > In any case, I tend to agree with the conclusion in the downthread
> > by Dscho that we should just clearly mark that invocations of the
> > "format-patch --range-diff" command with additional diff options is
> > an experimental feature that may not do anything sensible in the
> > upcoming release, and declare that the UI to pass diff options to
> > affect only the range-diff part may later be invented.  IOW, I am
> > coming a bit stronger than Dscho's suggestion in that we should not
> > even pretend that we aimed to make the options used for range-diff
> > customizable when driven from format-patch in the upcoming release,
> > or aimed to make --range-diff option compatible with other diff
> > options given to the format-patch command.

I agree that this way forward makes sense. It's clear that I
overlooked how there could be unexpected interactions from passing
git-format-patch's own diff_options to show_range_diff(), so taking
time to think it through without the pressure of a looming release is
preferable to rushing out some "fixes".

> So how about doing this on top of 'master' instead?  As this leaks
> *no* information wrt how range-diff machinery should behave from the
> format-patch side by not passing any diffopt, as long as the new
> code I added to show_range_diff() comes up with a reasonable default
> diffopts (for which I really would appreciate extra sets of eyes to
> make sure), this change by definition cannot be wrong (famous last
> words).

I, myself, was going to suggest this approach of leaking none of the
git-format-patch's options into range_diff(), so I think it is a good
one. Later, we can selectively pass certain _sensible_ options into
show_range_diff() once we figure out the correct UI (for instance,
--range-diff-opts=nopatch,creation-factor:60).

A couple comments on the patch itself...

> diff --git a/range-diff.c b/range-diff.c
> @@ -460,7 +460,11 @@ int show_range_diff(const char *range1, const char *range2,
> -               memcpy(&opts, diffopt, sizeof(opts));
> +               if (diffopt)
> +                       memcpy(&opts, diffopt, sizeof(opts));
> +               else
> +                       repo_diff_setup(the_repository, &opts);

The first attempt at adding --range-diff to git-format-patch invoked
the git-range-diff command, so no diff_options were passed at all.
After Dscho libified the range-diff machinery in one of his major
re-rolls, I took advantage of that to avoid the subprocess invocation.
Another benefit of calling show_range_diff() directly is that when
"git format-patch --stdout --range-diff=..." is sent to the terminal,
the range-diff gets colored output for free. I'm pleased to see that
that still works after this change.

> diff --git a/range-diff.h b/range-diff.h
> @@ -5,6 +5,11 @@
> +/*
> + * Compare series of commmits in RANGE1 and RANGE2, and emit to the
> + * standard output.  NULL can be passed to DIFFOPT to use the built-in
> + * default.
> + */

It is more correct to say that the range-diff is emitted to
diffopt->file (which may be stdout).

Thanks for working on this.

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

[Eric Sunshine]

On Thu, Nov 29, 2018 at 11:03 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> I mean not just nasty in terms of implementation, yeah we could do it,
> but also a nasty UX for things like --word-diff-regex. I.e. instead of:
>
>     --range-diff-word-diff-regex='[0-9"]'
>
> You need:
>
>     --range-diff-opts="--word-diff-regex='[0-9\"]'"
>
> Now admittedly that in itself isn't very painful *in this case*, but in
> terms of precedent I really dislike that option, i.e. git having some
> mode where I need to work to escape input to pass to another command.
>
> Not saying that this --range-diff-* thing is what we should go for, but
> surely we can find some way to do deal with this that doesn't involve
> the user needing to escape stuff like this.

I should mention that it was never the intention that
git-format-patch's --range-diff option would allow passing _all_
possible options to git-range-diff, but only those most likely to be
tweaked by the user (such as --creation-factor). It was understood
from the start (and stated by me either in the cover letter to that
series or in discussion) that the user always has the escape hatch of
running git-range-diff manually and copy/pasting its output into the
cover letter.

So, I'm not convinced that we need this sort of flexibility in
git-format-patch's --range-diff option, but we certainly can add
convenience options (such as I did with --creation-factor) as people
complain that their favorite option is missing. For a UI, I think we
already have sufficient precedent for
"--range-diff-opts=nopatch,creation-factor:60" as a possibility.

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

[Johannes Schindelin]

Hi Junio,

On Fri, 30 Nov 2018, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> >> I had to delay -rc2 to see these last minute tweaks come to some
> >> reasonable place to stop at, and I do not think we want to delay the
> >> final any longer or destablizing it further by piling last minute
> >> undercooked changes on top.
> >
> > So how about doing this on top of 'master' instead?  As this leaks
> > *no* information wrt how range-diff machinery should behave from the
> > format-patch side by not passing any diffopt, as long as the new
> > code I added to show_range_diff() comes up with a reasonable default
> > diffopts (for which I really would appreciate extra sets of eyes to
> > make sure), this change by definition cannot be wrong (famous last
> > words).
> 
> As listed in today's "What's cooking" report, I've merged this to
> 'next' in today's pushout and planning to have it in the -rc2.  I am
> not married to this exact implementation, and I'd welcome to have an
> even simpler and less disruptive solution if exists, but I am hoping
> that this is a good-enough interim measure for the upcoming release,
> until we decide what to do with the customizability of range-diff
> driven by format-patch.
> 
> In addition to this, I am planning the "rebase --stat" and "reflog
> that does not say 'rebase -i' but 'rebase'" fixes merged to 'master'
> before cutting -rc2.

Thank you for integrating them. That way, we have an -rc2 with no issues
in the built-in rebase/rebase -i that we know of.

Ciao,
Dscho

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

[Martin Ågren]

On Fri, 30 Nov 2018 at 10:32, Eric Sunshine <sunshine@sunshineco.com> wrote:
>
> On Thu, Nov 29, 2018 at 11:27 PM Junio C Hamano <gitster@pobox.com> wrote:
> > Junio C Hamano <gitster@pobox.com> writes:
> > So how about doing this on top of 'master' instead?  As this leaks
> > *no* information wrt how range-diff machinery should behave from the
> > format-patch side by not passing any diffopt, as long as the new
> > code I added to show_range_diff() comes up with a reasonable default
> > diffopts (for which I really would appreciate extra sets of eyes to
> > make sure), this change by definition cannot be wrong (famous last
> > words).

> Another benefit of calling show_range_diff() directly is that when
> "git format-patch --stdout --range-diff=..." is sent to the terminal,
> the range-diff gets colored output for free. I'm pleased to see that
> that still works after this change.

(If the patch below makes any sense to you and you know more about this
diff/color thing, the fourth bullet in the log message below might
interest you.)

> > diff --git a/range-diff.h b/range-diff.h
> > @@ -5,6 +5,11 @@
> > +/*
> > + * Compare series of commmits in RANGE1 and RANGE2, and emit to the
> > + * standard output.  NULL can be passed to DIFFOPT to use the built-in
> > + * default.
> > + */
>
> It is more correct to say that the range-diff is emitted to
> diffopt->file (which may be stdout).

This seems to be an important remark. There's a pretty bad regression
here since when `diffopt` is NULL, we've lost our original, intended
`diffopt->file` and the range-diff ends up on stdout.

Here's my whitespace-damaged WIP. I would be able to pick this up again
in about 6h, but anyone is more than welcome to pick this up and run
with it in the meantime.

This is not a corner of the code that I'm particularly familiar with...

-->8--
Subject: [PATCH] range-diff: always pass at least minimal diff options
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Commit d8981c3f88 ("format-patch: do not let its diff-options affect
--range-diff", 2018-11-30) taught `show_range_diff()` to accept a
NULL-pointer as an indication that it should use its own "reasonable
default". That fixed a regression from a5170794 ("Merge branch
'ab/range-diff-no-patch'", 2018-11-18), but unfortunately it introduced
a regression of its own.

In particular, it means we forget the `file` member of the diff options,
so rather than placing a range-diff in the cover-letter, we write it to
stdout. In order to fix this, rewrite the two callers adjusted by
d8981c3f88 to create a "dummy" set of diff options where they only fill
in which file to use.

A couple of remarks about this commit:

  * No tests. The change in builtin/log.c has been tested manually, the
    one in log-tree.c not at all, other than by running existing tests.

  * I have not convinced myself 100% that there aren't other things that
    are just as important as `file` to pass down.

  * `show_range_diff()` can still take NULL, although that is now dead
    code. If something like this here commit is deemed the proper fix
    for this, that code path could also go, either as part of this
    commit, or separately, once we've cut 2.20.

  * The range-diff is written colored regardless of destination, i.e.,
    when written to a file, it contains garbage.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
 builtin/log.c | 12 +++++++++++-
 log-tree.c    | 12 +++++++++++-
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/builtin/log.c b/builtin/log.c
index 5ac18e2848..0609e41ae5 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1094,9 +1094,19 @@ static void make_cover_letter(struct rev_info
*rev, int use_stdout,
     }

     if (rev->rdiff1) {
+        /**
+         * (At least for now) we only want to pass down
+         * the file handle where we want the range-diff
+         * to appear. Avoid any other diff options until
+         * we know how we want to handle them.
+         */
+        struct diff_options opts;
+        diff_setup(&opts);
+        opts.file = rev->diffopt.file;
+        diff_setup_done(&opts);
         fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
         show_range_diff(rev->rdiff1, rev->rdiff2,
-                rev->creation_factor, 1, NULL);
+                rev->creation_factor, 1, &opts);
     }
 }

diff --git a/log-tree.c b/log-tree.c
index b243779a0b..bc355a4e91 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -755,14 +755,24 @@ void show_log(struct rev_info *opt)

     if (cmit_fmt_is_mail(ctx.fmt) && opt->rdiff1) {
         struct diff_queue_struct dq;
+        struct diff_options opts;

         memcpy(&dq, &diff_queued_diff, sizeof(diff_queued_diff));
         DIFF_QUEUE_CLEAR(&diff_queued_diff);

         next_commentary_block(opt, NULL);
         fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
+        /**
+         * (At least for now) we only want to pass down
+         * the file handle where we want the range-diff
+         * to appear. Avoid any other diff options until
+         * we know how we want to handle them.
+        */
+        diff_setup(&opts);
+        opts.file = opt->diffopt.file;
+        diff_setup_done(&opts);
         show_range_diff(opt->rdiff1, opt->rdiff2,
-                opt->creation_factor, 1, NULL);
+                opt->creation_factor, 1, &opts);

         memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
     }
-- 
2.20.0.rc2

[PATCH v2] range-diff: always pass at least minimal diff options

[Martin Ågren]

Commit d8981c3f88 ("format-patch: do not let its diff-options affect
--range-diff", 2018-11-30) taught `show_range_diff()` to accept a
NULL-pointer as an indication that it should use its own "reasonable
default". That fixed a regression from a5170794 ("Merge branch
'ab/range-diff-no-patch'", 2018-11-18), but unfortunately it introduced
a regression of its own.

In particular, it means we forget the `file` member of the diff options,
so rather than placing a range-diff in the cover-letter, we write it to
stdout. In order to fix this, rewrite the two callers adjusted by
d8981c3f88 to instead create a "dummy" set of diff options where they
only fill in which file to use.

Plus, turn off coloring to make sure we don't write any color codes.
Maybe we could do `opts.use_color = opts.file != stdout`, but for now,
I'd much rather always write uncolored output than write color codes
where there shouldn't be any.

Modify and extend the existing tests to try and verify that the right
contents end up in the right place.

Don't revert `show_range_diff()`, i.e., let it keep accepting NULL.
Rather than removing what is dead code and figuring out it isn't
actually dead and we've broken 2.20, just leave it for now.

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
---
Here's another attempt at fixing this recent regression.

 t/t3206-range-diff.sh | 20 +++++++++++++-------
 builtin/log.c         | 13 ++++++++++++-
 log-tree.c            | 13 ++++++++++++-
 3 files changed, 37 insertions(+), 9 deletions(-)

diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index e497c1358f..048feaf6dd 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -248,18 +248,24 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
-		git format-patch --stdout --cover-letter --range-diff=$prev \
+		git format-patch --cover-letter --range-diff=$prev \
 			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+		test_when_finished "rm 000?-*" &&
+		test_line_count = 5 actual &&
+		test_i18ngrep "^Range-diff:$" 0000-* &&
+		grep "= 1: .* s/5/A" 0000-* &&
+		grep "= 2: .* s/4/A" 0000-* &&
+		grep "= 3: .* s/11/B" 0000-* &&
+		grep "= 4: .* s/12/B" 0000-*
 	'
 done
 
 test_expect_success 'format-patch --range-diff as commentary' '
-	git format-patch --stdout --range-diff=HEAD~1 HEAD~1 >actual &&
-	test_i18ngrep "^Range-diff:$" actual
+	git format-patch --range-diff=HEAD~1 HEAD~1 >actual &&
+	test_when_finished "rm 0001-*" &&
+	test_line_count = 1 actual &&
+	test_i18ngrep "^Range-diff:$" 0001-* &&
+	grep "> 1: .* new message" 0001-*
 '
 
 test_done
diff --git a/builtin/log.c b/builtin/log.c
index 5ac18e2848..e42487b46d 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1094,9 +1094,20 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
+		/*
+		 * (At least for now) we only want to pass down
+		 * the file handle where we want the range-diff
+		 * to appear. Avoid any other diff options until
+		 * we know how we want to handle them.
+		 */
+		struct diff_options opts;
+		diff_setup(&opts);
+		opts.file = rev->diffopt.file;
+		opts.use_color = 0;
+		diff_setup_done(&opts);
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, NULL);
+				rev->creation_factor, 1, &opts);
 	}
 }
 
diff --git a/log-tree.c b/log-tree.c
index b243779a0b..fd79a3ec37 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -755,14 +755,25 @@ void show_log(struct rev_info *opt)
 
 	if (cmit_fmt_is_mail(ctx.fmt) && opt->rdiff1) {
 		struct diff_queue_struct dq;
+		struct diff_options opts;
 
 		memcpy(&dq, &diff_queued_diff, sizeof(diff_queued_diff));
 		DIFF_QUEUE_CLEAR(&diff_queued_diff);
 
 		next_commentary_block(opt, NULL);
 		fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
+		/*
+		 * (At least for now) we only want to pass down
+		 * the file handle where we want the range-diff
+		 * to appear. Avoid any other diff options until
+		 * we know how we want to handle them.
+		 */
+		diff_setup(&opts);
+		opts.file = opt->diffopt.file;
+		opts.use_color = 0;
+		diff_setup_done(&opts);
 		show_range_diff(opt->rdiff1, opt->rdiff2,
-				opt->creation_factor, 1, NULL);
+				opt->creation_factor, 1, &opts);
 
 		memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
 	}
-- 
2.20.0.rc2.1.gfcc5f94f1e

[PATCH v3] range-diff: always pass at least minimal diff options

[Eric Sunshine]

From: Martin Ågren <martin.agren@gmail.com>

Commit d8981c3f88 ("format-patch: do not let its diff-options affect
--range-diff", 2018-11-30) taught `show_range_diff()` to accept a
NULL-pointer as an indication that it should use its own "reasonable
default". That fixed a regression from a5170794 ("Merge branch
'ab/range-diff-no-patch'", 2018-11-18), but unfortunately it introduced
a regression of its own.

In particular, it means we forget the `file` member of the diff options,
so rather than placing a range-diff in the cover-letter, we write it to
stdout. In order to fix this, rewrite the two callers adjusted by
d8981c3f88 to instead create a "dummy" set of diff options where they
only fill in the fields we absolutely require, such as output file and
color.

Modify and extend the existing tests to try and verify that the right
contents end up in the right place.

Don't revert `show_range_diff()`, i.e., let it keep accepting NULL.
Rather than removing what is dead code and figuring out it isn't
actually dead and we've broken 2.20, just leave it for now.

[es: retain diff coloring when going to stdout]

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---

This is a re-roll of Martin's v2[1]. The only difference from v2 is that
it retains coloring when emitting to the terminal (plus an in-code
comment was simplified).

The regression introduced by d8981c3f88, in which the range-diff only
ever gets emitted to the terminal, and never to the cover letter or
commentary section of a standalone patch, makes the --range-diff option
rather useless, so this fix probably ought to be fast-tracked. An
alternative would be to rip out all the recent "--range-diff"-related
changes and go with the --range-diff implementation which has been in
use for a few months, even if it is not perfect.

[1]: https://public-inbox.org/git/20181203200734.527341-1-martin.agren@gmail.com/

builtin/log.c         | 11 ++++++++++-
 log-tree.c            | 11 ++++++++++-
 t/t3206-range-diff.sh | 20 +++++++++++++-------
 3 files changed, 33 insertions(+), 9 deletions(-)

diff --git a/builtin/log.c b/builtin/log.c
index 5ac18e2848..e8e51068bd 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1094,9 +1094,18 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
+		/*
+		 * Pass minimum required diff-options to range-diff; others
+		 * can be added later if deemed desirable.
+		 */
+		struct diff_options opts;
+		diff_setup(&opts);
+		opts.file = rev->diffopt.file;
+		opts.use_color = rev->diffopt.use_color;
+		diff_setup_done(&opts);
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, NULL);
+				rev->creation_factor, 1, &opts);
 	}
 }
 
diff --git a/log-tree.c b/log-tree.c
index b243779a0b..10680c139e 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -755,14 +755,23 @@ void show_log(struct rev_info *opt)
 
 	if (cmit_fmt_is_mail(ctx.fmt) && opt->rdiff1) {
 		struct diff_queue_struct dq;
+		struct diff_options opts;
 
 		memcpy(&dq, &diff_queued_diff, sizeof(diff_queued_diff));
 		DIFF_QUEUE_CLEAR(&diff_queued_diff);
 
 		next_commentary_block(opt, NULL);
 		fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
+		/*
+		 * Pass minimum required diff-options to range-diff; others
+		 * can be added later if deemed desirable.
+		 */
+		diff_setup(&opts);
+		opts.file = opt->diffopt.file;
+		opts.use_color = opt->diffopt.use_color;
+		diff_setup_done(&opts);
 		show_range_diff(opt->rdiff1, opt->rdiff2,
-				opt->creation_factor, 1, NULL);
+				opt->creation_factor, 1, &opts);
 
 		memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
 	}
diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index e497c1358f..048feaf6dd 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -248,18 +248,24 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
-		git format-patch --stdout --cover-letter --range-diff=$prev \
+		git format-patch --cover-letter --range-diff=$prev \
 			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+		test_when_finished "rm 000?-*" &&
+		test_line_count = 5 actual &&
+		test_i18ngrep "^Range-diff:$" 0000-* &&
+		grep "= 1: .* s/5/A" 0000-* &&
+		grep "= 2: .* s/4/A" 0000-* &&
+		grep "= 3: .* s/11/B" 0000-* &&
+		grep "= 4: .* s/12/B" 0000-*
 	'
 done
 
 test_expect_success 'format-patch --range-diff as commentary' '
-	git format-patch --stdout --range-diff=HEAD~1 HEAD~1 >actual &&
-	test_i18ngrep "^Range-diff:$" actual
+	git format-patch --range-diff=HEAD~1 HEAD~1 >actual &&
+	test_when_finished "rm 0001-*" &&
+	test_line_count = 1 actual &&
+	test_i18ngrep "^Range-diff:$" 0001-* &&
+	grep "> 1: .* new message" 0001-*
 '
 
 test_done
-- 
2.20.0.rc2.403.gdbc3b29805

Re: [PATCH v3] range-diff: always pass at least minimal diff options

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> This is a re-roll of Martin's v2[1]. The only difference from v2 is that
> it retains coloring when emitting to the terminal (plus an in-code
> comment was simplified).
>
> The regression introduced by d8981c3f88, in which the range-diff only
> ever gets emitted to the terminal, and never to the cover letter or
> commentary section of a standalone patch, makes the --range-diff option
> rather useless, so this fix probably ought to be fast-tracked.

Yup.  Thanks.  The only thing that makes me wonder is why any of the
existing tests (among which I think I saw range-diff driven from
format-patch) did not catch this rather obvious glitch.

> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> index e497c1358f..048feaf6dd 100755
> --- a/t/t3206-range-diff.sh
> +++ b/t/t3206-range-diff.sh
> @@ -248,18 +248,24 @@ test_expect_success 'dual-coloring' '
>  for prev in topic master..topic
>  do
>  	test_expect_success "format-patch --range-diff=$prev" '
> -		git format-patch --stdout --cover-letter --range-diff=$prev \
> +		git format-patch --cover-letter --range-diff=$prev \
>  			master..unmodified >actual &&

Ah, of course.  Then the "actual" file gets the names of the output
files; we expect to see 5 of them.

But now we have lost all the range-diff tests run under "--stdout",
so next time some other change regresses only that codepath, we will
not notice (which is fine for now but would want to be addressed
before the end of the year around which time we certainly will all
forget).

Thanks.

> -		grep "= 1: .* s/5/A" actual &&
> -		grep "= 2: .* s/4/A" actual &&
> -		grep "= 3: .* s/11/B" actual &&
> -		grep "= 4: .* s/12/B" actual
> +		test_when_finished "rm 000?-*" &&
> +		test_line_count = 5 actual &&
> +		test_i18ngrep "^Range-diff:$" 0000-* &&
> +		grep "= 1: .* s/5/A" 0000-* &&
> +		grep "= 2: .* s/4/A" 0000-* &&
> +		grep "= 3: .* s/11/B" 0000-* &&
> +		grep "= 4: .* s/12/B" 0000-*
>  	'
>  done
>  
>  test_expect_success 'format-patch --range-diff as commentary' '
> -	git format-patch --stdout --range-diff=HEAD~1 HEAD~1 >actual &&
> -	test_i18ngrep "^Range-diff:$" actual
> +	git format-patch --range-diff=HEAD~1 HEAD~1 >actual &&
> +	test_when_finished "rm 0001-*" &&
> +	test_line_count = 1 actual &&
> +	test_i18ngrep "^Range-diff:$" 0001-* &&
> +	grep "> 1: .* new message" 0001-*
>  '
>  
>  test_done

Re: [PATCH v3] range-diff: always pass at least minimal diff options

[Martin Ågren]

On Mon, 3 Dec 2018 at 22:21, Eric Sunshine <sunshine@sunshineco.com> wrote:
> [es: retain diff coloring when going to stdout]
>
> Signed-off-by: Martin Ågren <martin.agren@gmail.com>
> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
> ---
>
> This is a re-roll of Martin's v2[1]. The only difference from v2 is that
> it retains coloring when emitting to the terminal (plus an in-code
> comment was simplified).

Thank you so much for this.

>         if (rev->rdiff1) {
> +               /*
> +                * Pass minimum required diff-options to range-diff; others
> +                * can be added later if deemed desirable.
> +                */

Agreed.

> +               struct diff_options opts;
> +               diff_setup(&opts);
> +               opts.file = rev->diffopt.file;
> +               opts.use_color = rev->diffopt.use_color;

Ah, s/0/rev->diffopt.use_color/, well that's obvious.

Thanks!

Martin