[PATCH 00/22] Prepare the sequencer for the upcoming rebase -i patches

[PATCH 00/22] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

This patch series marks the  '4' in the countdown to speed up rebase -i
by implementing large parts in C. It is based on the `libify-sequencer`
patch series that I submitted last week.

The patches in this series merely prepare the sequencer code for the
next patch series that actually teaches the sequencer to run an
interactive rebase.

The reason to split these two patch series is simple: to keep them at a
sensible size.

The two patch series after that are much smaller: a two-patch "series"
that switches rebase -i to use the sequencer (except with --root or
--preserve-merges), and a couple of patches to move several pretty
expensive script processing steps to C (think: autosquash).

The end game of this patch series is a git-rebase--helper that makes
rebase -i 5x faster on Windows (according to t/perf/p3404). Travis says
that even MacOSX and Linux benefit (4x and 3x, respectively).

I have been working on this since early February, whenever time allowed,
and it is time to put it into the users' hands. To that end, I will most
likely submit the remaining three patch series in the next two days, and
integrate the whole shebang into Git for Windows 2.10.0.

Therefore I would be most grateful for every in-depth review.


Johannes Schindelin (22):
  sequencer: use static initializers for replay_opts
  sequencer: use memoized sequencer directory path
  sequencer: avoid unnecessary indirection
  sequencer: future-proof remove_sequencer_state()
  sequencer: allow the sequencer to take custody of malloc()ed data
  sequencer: release memory that was allocated when reading options
  sequencer: future-proof read_populate_todo()
  sequencer: remove overzealous assumption
  sequencer: completely revamp the "todo" script parsing
  sequencer: avoid completely different messages for different actions
  sequencer: get rid of the subcommand field
  sequencer: refactor the code to obtain a short commit name
  sequencer: remember the onelines when parsing the todo file
  sequencer: prepare for rebase -i's commit functionality
  sequencer: introduce a helper to read files written by scripts
  sequencer: prepare for rebase -i's GPG settings
  sequencer: allow editing the commit message on a case-by-case basis
  sequencer: support amending commits
  sequencer: support cleaning up commit messages
  sequencer: remember do_recursive_merge()'s return value
  sequencer: left-trim the lines read from the script
  sequencer: refactor write_message()

 builtin/commit.c                |   2 +-
 builtin/revert.c                |  42 ++-
 sequencer.c                     | 573 +++++++++++++++++++++++++++-------------
 sequencer.h                     |  27 +-
 t/t3510-cherry-pick-sequence.sh |  11 -
 5 files changed, 428 insertions(+), 227 deletions(-)

Based-On: libify-sequencer at https://github.com/dscho/git
Fetch-Base-Via: git fetch https://github.com/dscho/git libify-sequencer
Published-As: https://github.com/dscho/git/releases/tag/prepare-sequencer-v1
Fetch-It-Via: git fetch https://github.com/dscho/git prepare-sequencer-v1

-- 
2.10.0.rc1.114.g2bd6b38

base-commit: 2d6d71e2a2d410b12d783f0a8edd22791f303c12

[PATCH 01/22] sequencer: use static initializers for replay_opts

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 6 ++----
 sequencer.h      | 1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 4e69380..7365559 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -178,10 +178,9 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	if (isatty(0))
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
@@ -195,10 +194,9 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 
 int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
 	parse_args(argc, argv, &opts);
diff --git a/sequencer.h b/sequencer.h
index 5ed5cb1..2ca096b 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -47,6 +47,7 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
+#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 02/22] sequencer: use memoized sequencer directory path

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/commit.c |  2 +-
 sequencer.c      | 11 ++++++-----
 sequencer.h      |  5 +----
 3 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 77e3dc8..0221190 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -173,7 +173,7 @@ static void determine_whence(struct wt_status *s)
 		whence = FROM_MERGE;
 	else if (file_exists(git_path_cherry_pick_head())) {
 		whence = FROM_CHERRY_PICK;
-		if (file_exists(git_path(SEQ_DIR)))
+		if (file_exists(git_path_seq_dir()))
 			sequencer_in_use = 1;
 	}
 	else
diff --git a/sequencer.c b/sequencer.c
index b6481bb..4d2b4e3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,10 +21,11 @@
 const char sign_off_header[] = "Signed-off-by: ";
 static const char cherry_picked_prefix[] = "(cherry picked from commit ";
 
-static GIT_PATH_FUNC(git_path_todo_file, SEQ_TODO_FILE)
-static GIT_PATH_FUNC(git_path_opts_file, SEQ_OPTS_FILE)
-static GIT_PATH_FUNC(git_path_seq_dir, SEQ_DIR)
-static GIT_PATH_FUNC(git_path_head_file, SEQ_HEAD_FILE)
+GIT_PATH_FUNC(git_path_seq_dir, "sequencer")
+
+static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
+static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
+static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
 static int is_rfc2822_line(const char *buf, int len)
 {
@@ -112,7 +113,7 @@ static void remove_sequencer_state(void)
 {
 	struct strbuf seq_dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path(SEQ_DIR));
+	strbuf_addstr(&seq_dir, git_path_seq_dir());
 	remove_dir_recursively(&seq_dir, 0);
 	strbuf_release(&seq_dir);
 }
diff --git a/sequencer.h b/sequencer.h
index 2ca096b..c955594 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -1,10 +1,7 @@
 #ifndef SEQUENCER_H
 #define SEQUENCER_H
 
-#define SEQ_DIR		"sequencer"
-#define SEQ_HEAD_FILE	"sequencer/head"
-#define SEQ_TODO_FILE	"sequencer/todo"
-#define SEQ_OPTS_FILE	"sequencer/opts"
+const char *git_path_seq_dir(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 03/22] sequencer: avoid unnecessary indirection

[Johannes Schindelin]

We really do not need the *pointer to a* pointer to the options in
the read_populate_opts() function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4d2b4e3..14ef79b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -809,11 +809,11 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
-static int read_populate_opts(struct replay_opts **opts)
+static int read_populate_opts(struct replay_opts *opts)
 {
 	if (!file_exists(git_path_opts_file()))
 		return 0;
-	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
+	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
 		return error(_("Malformed options sheet: %s"),
 			git_path_opts_file());
 	return 0;
@@ -1038,7 +1038,7 @@ static int sequencer_continue(struct replay_opts *opts)
 
 	if (!file_exists(git_path_todo_file()))
 		return continue_single_pick();
-	if (read_populate_opts(&opts) ||
+	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
 		return -1;
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 04/22] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

In a couple of commits, we will teach the sequencer to handle the
nitty gritty of the interactive rebase, which keeps its state in a
different directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 14ef79b..c4b223b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,11 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static const char *get_dir(const struct replay_opts *opts)
+{
+	return git_path_seq_dir();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -109,13 +114,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(void)
+static void remove_sequencer_state(const struct replay_opts *opts)
 {
-	struct strbuf seq_dir = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path_seq_dir());
-	remove_dir_recursively(&seq_dir, 0);
-	strbuf_release(&seq_dir);
+	strbuf_addf(&dir, "%s", get_dir(opts));
+	remove_dir_recursively(&dir, 0);
+	strbuf_release(&dir);
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -895,6 +900,9 @@ static int sequencer_rollback(struct replay_opts *opts)
 	unsigned char sha1[20];
 	struct strbuf buf = STRBUF_INIT;
 
+	if (read_and_refresh_cache(opts))
+		return -1;
+
 	f = fopen(git_path_head_file(), "r");
 	if (!f && errno == ENOENT) {
 		/*
@@ -924,7 +932,7 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	strbuf_release(&buf);
 	return 0;
 fail:
@@ -1018,7 +1026,7 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	return 0;
 }
 
@@ -1079,7 +1087,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	 * one that is being continued
 	 */
 	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state();
+		remove_sequencer_state(opts);
 		return 0;
 	}
 	if (opts->subcommand == REPLAY_ROLLBACK)
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Schindelin]

The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
like a one-shot command when it reads its configuration: memory is
allocated and released only when the command exits.

This is kind of okay for git-cherry-pick, which *is* a one-shot
command. All the work to make the sequencer its work horse was
done to allow using the functionality as a library function, though,
including proper clean-up after use.

This patch introduces an API to pass the responsibility of releasing
certain memory to the sequencer.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 +++++++++++++
 sequencer.h |  8 +++++++-
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index c4b223b..b5be0f9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -114,9 +114,22 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
+void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
+{
+	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
+	opts->owned[opts->owned_nr++] = set_me_free_after_use;
+
+	return set_me_free_after_use;
+}
+
 static void remove_sequencer_state(const struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
+	int i;
+
+	for (i = 0; i < opts->owned_nr; i++)
+		free(opts->owned[i]);
+	free(opts->owned);
 
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
diff --git a/sequencer.h b/sequencer.h
index c955594..20b708a 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -43,8 +43,14 @@ struct replay_opts {
 
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
+
+	/* malloc()ed data entrusted to the sequencer */
+	void **owned;
+	int owned_nr, owned_alloc;
 };
-#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }
+#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL, NULL, 0, 0 }
+
+void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use);
 
 int sequencer_pick_revisions(struct replay_opts *opts);
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 06/22] sequencer: release memory that was allocated when reading options

[Johannes Schindelin]

The sequencer reads options from disk and stores them in its struct
for use during sequencer's operations.

With this patch, the memory is released afterwards, plugging a
memory leak.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index b5be0f9..8d79091 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -131,6 +131,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
 		free(opts->owned[i]);
 	free(opts->owned);
 
+	free(opts->xopts);
+
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
 	strbuf_release(&dir);
@@ -811,13 +813,18 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
 	else if (!strcmp(key, "options.mainline"))
 		opts->mainline = git_config_int(key, value);
-	else if (!strcmp(key, "options.strategy"))
+	else if (!strcmp(key, "options.strategy")) {
 		git_config_string(&opts->strategy, key, value);
-	else if (!strcmp(key, "options.gpg-sign"))
+		sequencer_entrust(opts, (char *) opts->strategy);
+	}
+	else if (!strcmp(key, "options.gpg-sign")) {
 		git_config_string(&opts->gpg_sign, key, value);
+		sequencer_entrust(opts, (char *) opts->gpg_sign);
+	}
 	else if (!strcmp(key, "options.strategy-option")) {
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
-		opts->xopts[opts->xopts_nr++] = xstrdup(value);
+		opts->xopts[opts->xopts_nr++] =
+			sequencer_entrust(opts, xstrdup(value));
 	} else
 		return error(_("Invalid key: %s"), key);
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 07/22] sequencer: future-proof read_populate_todo()

[Johannes Schindelin]

Over the next commits, we will work on improving the sequencer to the
point where it can process the edit script of an interactive rebase. To
that end, we will need to teach the sequencer to read interactive
rebase's todo file. In preparation, we consolidate all places where
that todo file is needed to call a function that we will later extend.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8d79091..982b6e9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -32,6 +32,11 @@ static const char *get_dir(const struct replay_opts *opts)
 	return git_path_seq_dir();
 }
 
+static const char *get_todo_path(const struct replay_opts *opts)
+{
+	return git_path_todo_file();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -772,25 +777,24 @@ static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
 static int read_populate_todo(struct commit_list **todo_list,
 			struct replay_opts *opts)
 {
+	const char *todo_file = get_todo_path(opts);
 	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
-	fd = open(git_path_todo_file(), O_RDONLY);
+	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"),
-				   git_path_todo_file());
+		return error_errno(_("Could not open %s"), todo_file);
 	if (strbuf_read(&buf, fd, 0) < 0) {
 		close(fd);
 		strbuf_release(&buf);
-		return error(_("Could not read %s."), git_path_todo_file());
+		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(buf.buf, todo_list, opts);
 	strbuf_release(&buf);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"),
-			git_path_todo_file());
+		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
@@ -1064,7 +1068,7 @@ static int sequencer_continue(struct replay_opts *opts)
 {
 	struct commit_list *todo_list = NULL;
 
-	if (!file_exists(git_path_todo_file()))
+	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 08/22] sequencer: remove overzealous assumption

[Johannes Schindelin]

The sequencer was introduced to make the cherry-pick and revert
functionality available as library function, with the original idea
being to extend the sequencer to also implement the rebase -i
functionality.

The test to ensure that all of the commands in the script are identical
to the overall operation does not mesh well with that.

Therefore let's just get rid of the test that wants to verify that this
limitation is still in place, in preparation for the upcoming work to
teach the sequencer to do rebase -i's work.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t3510-cherry-pick-sequence.sh | 11 -----------
 1 file changed, 11 deletions(-)

diff --git a/t/t3510-cherry-pick-sequence.sh b/t/t3510-cherry-pick-sequence.sh
index 7b7a89d..6465edf 100755
--- a/t/t3510-cherry-pick-sequence.sh
+++ b/t/t3510-cherry-pick-sequence.sh
@@ -459,17 +459,6 @@ test_expect_success 'malformed instruction sheet 1' '
 	test_expect_code 128 git cherry-pick --continue
 '
 
-test_expect_success 'malformed instruction sheet 2' '
-	pristine_detach initial &&
-	test_expect_code 1 git cherry-pick base..anotherpick &&
-	echo "resolved" >foo &&
-	git add foo &&
-	git commit &&
-	sed "s/pick/revert/" .git/sequencer/todo >new_sheet &&
-	cp new_sheet .git/sequencer/todo &&
-	test_expect_code 128 git cherry-pick --continue
-'
-
 test_expect_success 'empty commit set' '
 	pristine_detach initial &&
 	test_expect_code 128 git cherry-pick base..base
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

When we came up with the "sequencer" idea, we really wanted to have
kind of a plumbing equivalent of the interactive rebase. Hence the
choice of words: the "todo" script, a "pick", etc.

However, when it came time to implement the entire shebang, somehow this
idea got lost and the sequencer was used as working horse for
cherry-pick and revert instead. So as not to interfere with the
interactive rebase, it even uses a separate directory to store its
state.

Furthermore, it also is stupidly strict about the "todo" script it
accepts: while it parses commands in a way that was *designed* to be
similar to the interactive rebase, it then goes on to *error out* if the
commands disagree with the overall action (cherry-pick or revert).

Finally, the sequencer code chose to deviate from the interactive rebase
code insofar that it *reformats* the "todo" script instead of just
writing the part of the parsed script that were not yet processed. This
is not only unnecessary churn, but might well lose information that is
valuable to the user (i.e. comments after the commands).

Let's just bite the bullet and rewrite the entire parser; the code now
becomes not only more elegant: it allows us to go on and teach the
sequencer how to parse *true* "todo" scripts as used by the interactive
rebase itself. In a way, the sequencer is about to grow up to do its
older brother's job. Better.

While at it, do not stop at the first problem, but list *all* of the
problems. This helps the user by allowing to address all issues in
one go rather than going back and forth until the todo list is valid.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 241 +++++++++++++++++++++++++++++++++---------------------------
 1 file changed, 134 insertions(+), 107 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 982b6e9..cbdce6d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -473,7 +473,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
+enum todo_command {
+	TODO_PICK,
+	TODO_REVERT
+};
+
+static const char *todo_command_strings[] = {
+	"pick",
+	"revert"
+};
+
+static const char *command_to_string(const enum todo_command command)
+{
+	if (command < ARRAY_SIZE(todo_command_strings))
+		return todo_command_strings[command];
+	die("Unknown command: %d", command);
+}
+
+
+static int do_pick_commit(enum todo_command command, struct commit *commit,
+		struct replay_opts *opts)
 {
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
@@ -535,7 +554,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		/* TRANSLATORS: The first %s will be "revert" or
 		   "cherry-pick", the second %s a SHA1 */
 		return error(_("%s: cannot parse parent commit %s"),
-			action_name(opts), oid_to_hex(&parent->object.oid));
+			command_to_string(command),
+			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
 		return error(_("Cannot get commit message for %s"),
@@ -548,7 +568,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * reverse of it if we are revert.
 	 */
 
-	if (opts->action == REPLAY_REVERT) {
+	if (command == TODO_REVERT) {
 		base = commit;
 		base_label = msg.label;
 		next = parent;
@@ -589,7 +609,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
+	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
@@ -615,17 +635,17 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * However, if the merge did not even start, then we don't want to
 	 * write it at all.
 	 */
-	if (opts->action == REPLAY_PICK && !opts->no_commit && (res == 0 || res == 1) &&
+	if (command == TODO_PICK && !opts->no_commit && (res == 0 || res == 1) &&
 	    update_ref(NULL, "CHERRY_PICK_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
-	if (opts->action == REPLAY_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
+	if (command == TODO_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
 	    update_ref(NULL, "REVERT_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
 
 	if (res) {
-		error(opts->action == REPLAY_REVERT
+		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
 		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
@@ -683,116 +703,107 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
-static int format_todo(struct strbuf *buf, struct commit_list *todo_list,
-		struct replay_opts *opts)
+struct todo_item {
+	enum todo_command command;
+	struct commit *commit;
+	size_t offset_in_buf;
+};
+
+struct todo_list {
+	struct strbuf buf;
+	struct todo_item *items;
+	int nr, alloc, current;
+};
+
+#define TODO_LIST_INIT { STRBUF_INIT, NULL, 0, 0, 0 }
+
+static void todo_list_release(struct todo_list *todo_list)
 {
-	struct commit_list *cur = NULL;
-	const char *sha1_abbrev = NULL;
-	const char *action_str = opts->action == REPLAY_REVERT ? "revert" : "pick";
-	const char *subject;
-	int subject_len;
+	strbuf_release(&todo_list->buf);
+	free(todo_list->items);
+	todo_list->items = NULL;
+	todo_list->nr = todo_list->alloc = 0;
+}
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		const char *commit_buffer = get_commit_buffer(cur->item, NULL);
-		sha1_abbrev = find_unique_abbrev(cur->item->object.oid.hash, DEFAULT_ABBREV);
-		subject_len = find_commit_subject(commit_buffer, &subject);
-		strbuf_addf(buf, "%s %s %.*s\n", action_str, sha1_abbrev,
-			subject_len, subject);
-		unuse_commit_buffer(cur->item, commit_buffer);
-	}
-	return 0;
+struct todo_item *append_todo(struct todo_list *todo_list)
+{
+	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
+	return todo_list->items + todo_list->nr++;
 }
 
-static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
+static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 {
 	unsigned char commit_sha1[20];
-	enum replay_action action;
 	char *end_of_object_name;
-	int saved, status, padding;
-
-	if (starts_with(bol, "pick")) {
-		action = REPLAY_PICK;
-		bol += strlen("pick");
-	} else if (starts_with(bol, "revert")) {
-		action = REPLAY_REVERT;
-		bol += strlen("revert");
-	} else
-		return NULL;
+	int i, saved, status, padding;
+
+	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
+		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_strings))
+		return -1;
 
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
-		return NULL;
+		return -1;
 	bol += padding;
 
-	end_of_object_name = bol + strcspn(bol, " \t\n");
+	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
-	/*
-	 * Verify that the action matches up with the one in
-	 * opts; we don't support arbitrary instructions
-	 */
-	if (action != opts->action) {
-		if (action == REPLAY_REVERT)
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot revert during another revert.")
-			    : _("Cannot revert during a cherry-pick."));
-		else
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot cherry-pick during a revert.")
-			    : _("Cannot cherry-pick during another cherry-pick."));
-		return NULL;
-	}
-
 	if (status < 0)
-		return NULL;
+		return -1;
 
-	return lookup_commit_reference(commit_sha1);
+	item->commit = lookup_commit_reference(commit_sha1);
+	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
-			struct replay_opts *opts)
+static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
-	struct commit_list **next = todo_list;
-	struct commit *commit;
+	struct todo_item *item;
 	char *p = buf;
-	int i;
+	int i, res = 0;
 
 	for (i = 1; *p; i++) {
 		char *eol = strchrnul(p, '\n');
-		commit = parse_insn_line(p, eol, opts);
-		if (!commit)
-			return error(_("Could not parse line %d."), i);
-		next = commit_list_append(commit, next);
+
+		item = append_todo(todo_list);
+		item->offset_in_buf = p - todo_list->buf.buf;
+		if (parse_insn_line(item, p, eol)) {
+			error("Invalid line: %.*s", (int)(eol - p), p);
+			res |= error(_("Could not parse line %d."), i);
+			item->command = -1;
+		}
 		p = *eol ? eol + 1 : eol;
 	}
-	if (!*todo_list)
+	if (!todo_list->nr)
 		return error(_("No commits parsed."));
-	return 0;
+	return res;
 }
 
-static int read_populate_todo(struct commit_list **todo_list,
+static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
 	const char *todo_file = get_todo_path(opts);
-	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
+	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
 		return error_errno(_("Could not open %s"), todo_file);
-	if (strbuf_read(&buf, fd, 0) < 0) {
+	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		strbuf_release(&buf);
 		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
-	res = parse_insn_buffer(buf.buf, todo_list, opts);
-	strbuf_release(&buf);
+	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
 		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
@@ -848,18 +859,33 @@ static int read_populate_opts(struct replay_opts *opts)
 	return 0;
 }
 
-static int walk_revs_populate_todo(struct commit_list **todo_list,
+static int walk_revs_populate_todo(struct todo_list *todo_list,
 				struct replay_opts *opts)
 {
+	enum todo_command command = opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT;
 	struct commit *commit;
-	struct commit_list **next;
 
 	if (prepare_revs(opts))
 		return -1;
 
-	next = todo_list;
-	while ((commit = get_revision(opts->revs)))
-		next = commit_list_append(commit, next);
+	while ((commit = get_revision(opts->revs))) {
+		struct todo_item *item = append_todo(todo_list);
+		const char *commit_buffer = get_commit_buffer(commit, NULL);
+		const char *subject;
+		int subject_len;
+
+		item->command = command;
+		item->commit = commit;
+		item->offset_in_buf = todo_list->buf.len;
+		subject_len = find_commit_subject(commit_buffer, &subject);
+		strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
+			opts->action == REPLAY_PICK ?  "pick" : "revert",
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV),
+			subject_len, subject);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
 	return 0;
 }
 
@@ -964,30 +990,24 @@ static int sequencer_rollback(struct replay_opts *opts)
 	return -1;
 }
 
-static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
+static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	static struct lock_file todo_lock;
-	struct strbuf buf = STRBUF_INIT;
-	int fd;
+	const char *todo_path = get_todo_path(opts);
+	int next = todo_list->current, offset, fd;
 
-	fd = hold_lock_file_for_update(&todo_lock, git_path_todo_file(), 0);
+	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
 		return error_errno(_("Could not lock '%s'"),
 				   git_path_todo_file());
-	if (format_todo(&buf, todo_list, opts) < 0) {
-		strbuf_release(&buf);
-		return error(_("Could not format %s."), git_path_todo_file());
-	}
-	if (write_in_full(fd, buf.buf, buf.len) < 0) {
-		strbuf_release(&buf);
-		return error_errno(_("Could not write to %s"),
-				   git_path_todo_file());
-	}
-	if (commit_lock_file(&todo_lock) < 0) {
-		strbuf_release(&buf);
-		return error(_("Error wrapping up %s."), git_path_todo_file());
-	}
-	strbuf_release(&buf);
+	offset = next < todo_list->nr ?
+		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	if (write_in_full(fd, todo_list->buf.buf + offset,
+			todo_list->buf.len - offset) < 0)
+		return error(_("Could not write to %s (%s)"),
+			todo_path, strerror(errno));
+	if (commit_lock_file(&todo_lock) < 0)
+		return error(_("Error wrapping up %s."), todo_path);
 	return 0;
 }
 
@@ -1026,9 +1046,8 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
-static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
+static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	struct commit_list *cur;
 	int res;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
@@ -1038,10 +1057,12 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		if (save_todo(cur, opts))
+	while (todo_list->current < todo_list->nr) {
+		struct todo_item *item = todo_list->items + todo_list->current;
+		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(cur->item, opts);
+		res = do_pick_commit(item->command, item->commit, opts);
+		todo_list->current++;
 		if (res)
 			return res;
 	}
@@ -1066,7 +1087,8 @@ static int continue_single_pick(void)
 
 static int sequencer_continue(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
 
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
@@ -1083,21 +1105,24 @@ static int sequencer_continue(struct replay_opts *opts)
 	}
 	if (index_differs_from("HEAD", 0))
 		return error_dirty_index(opts);
-	todo_list = todo_list->next;
-	return pick_commits(todo_list, opts);
+	todo_list.current++;
+	res = pick_commits(&todo_list, opts);
+	todo_list_release(&todo_list);
+	return res;
 }
 
 static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
-	return do_pick_commit(cmit, opts);
+	return do_pick_commit(opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT, cmit, opts);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
 	unsigned char sha1[20];
-	int i;
+	int i, res;
 
 	if (opts->subcommand == REPLAY_NONE)
 		assert(opts->revs);
@@ -1171,7 +1196,9 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	if (save_head(sha1_to_hex(sha1)) ||
 			save_opts(opts))
 		return -1;
-	return pick_commits(todo_list, opts);
+	res = pick_commits(&todo_list, opts);
+	todo_list_release(&todo_list);
+	return res;
 }
 
 void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 10/22] sequencer: avoid completely different messages for different actions

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index cbdce6d..1b65202 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -232,11 +232,8 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(action_name(opts));
 
-	/* Different translation strings for cherry-pick and revert */
-	if (opts->action == REPLAY_PICK)
-		error(_("Your local changes would be overwritten by cherry-pick."));
-	else
-		error(_("Your local changes would be overwritten by revert."));
+	error(_("Your local changes would be overwritten by %s."),
+		action_name(opts));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 11/22] sequencer: get rid of the subcommand field

[Johannes Schindelin]

The subcommands are used exactly once, at the very beginning of
sequencer_pick_revisions(), to determine what to do. This is an
unnecessary level of indirection: we can simply call the correct
function to begin with. So let's do that.

While at it, ensure that the subcommands return an error code so that
they do not have to die() all over the place (bad practice for library
functions...).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 36 ++++++++++++++++--------------------
 sequencer.c      | 35 +++++++++++------------------------
 sequencer.h      | 13 ++++---------
 3 files changed, 31 insertions(+), 53 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 7365559..c9ae4dc 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -71,7 +71,7 @@ static void verify_opt_compatible(const char *me, const char *base_opt, ...)
 		die(_("%s: %s cannot be used with %s"), me, this_opt, base_opt);
 }
 
-static void parse_args(int argc, const char **argv, struct replay_opts *opts)
+static int run_sequencer(int argc, const char **argv, struct replay_opts *opts)
 {
 	const char * const * usage_str = revert_or_cherry_pick_usage(opts);
 	const char *me = action_name(opts);
@@ -115,25 +115,15 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 	if (opts->keep_redundant_commits)
 		opts->allow_empty = 1;
 
-	/* Set the subcommand */
-	if (cmd == 'q')
-		opts->subcommand = REPLAY_REMOVE_STATE;
-	else if (cmd == 'c')
-		opts->subcommand = REPLAY_CONTINUE;
-	else if (cmd == 'a')
-		opts->subcommand = REPLAY_ROLLBACK;
-	else
-		opts->subcommand = REPLAY_NONE;
-
 	/* Check for incompatible command line arguments */
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		char *this_operation;
-		if (opts->subcommand == REPLAY_REMOVE_STATE)
+		if (cmd == 'q')
 			this_operation = "--quit";
-		else if (opts->subcommand == REPLAY_CONTINUE)
+		else if (cmd == 'c')
 			this_operation = "--continue";
 		else {
-			assert(opts->subcommand == REPLAY_ROLLBACK);
+			assert(cmd == 'a');
 			this_operation = "--abort";
 		}
 
@@ -156,7 +146,7 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 				"--edit", opts->edit,
 				NULL);
 
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		opts->revs = NULL;
 	} else {
 		struct setup_revision_opt s_r_opt;
@@ -174,6 +164,14 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 	if (argc > 1)
 		usage_with_options(usage_str, options);
+
+	if (cmd == 'q')
+		return sequencer_remove_state(opts);
+	if (cmd == 'c')
+		return sequencer_continue(opts);
+	if (cmd == 'a')
+		return sequencer_rollback(opts);
+	return sequencer_pick_revisions(opts);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
@@ -185,8 +183,7 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("revert failed"));
 	return res;
@@ -199,8 +196,7 @@ int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("cherry-pick failed"));
 	return res;
diff --git a/sequencer.c b/sequencer.c
index 1b65202..ba1fd05 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -127,7 +127,7 @@ void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
 	return set_me_free_after_use;
 }
 
-static void remove_sequencer_state(const struct replay_opts *opts)
+int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
 	int i;
@@ -141,6 +141,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
 	strbuf_release(&dir);
+
+	return 0;
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -941,7 +943,7 @@ static int rollback_single_pick(void)
 	return reset_for_rollback(head_sha1);
 }
 
-static int sequencer_rollback(struct replay_opts *opts)
+int sequencer_rollback(struct replay_opts *opts)
 {
 	FILE *f;
 	unsigned char sha1[20];
@@ -979,9 +981,8 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state(opts);
 	strbuf_release(&buf);
-	return 0;
+	return sequencer_remove_state(opts);
 fail:
 	strbuf_release(&buf);
 	return -1;
@@ -1068,8 +1069,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state(opts);
-	return 0;
+	return sequencer_remove_state(opts);
 }
 
 static int continue_single_pick(void)
@@ -1082,11 +1082,14 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
-static int sequencer_continue(struct replay_opts *opts)
+int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
 	int res;
 
+	if (read_and_refresh_cache(opts))
+		return -1;
+
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
@@ -1121,26 +1124,10 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	unsigned char sha1[20];
 	int i, res;
 
-	if (opts->subcommand == REPLAY_NONE)
-		assert(opts->revs);
-
+	assert(opts->revs);
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	/*
-	 * Decide what to do depending on the arguments; a fresh
-	 * cherry-pick should be handled differently from an existing
-	 * one that is being continued
-	 */
-	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state(opts);
-		return 0;
-	}
-	if (opts->subcommand == REPLAY_ROLLBACK)
-		return sequencer_rollback(opts);
-	if (opts->subcommand == REPLAY_CONTINUE)
-		return sequencer_continue(opts);
-
 	for (i = 0; i < opts->revs->pending.nr; i++) {
 		unsigned char sha1[20];
 		const char *name = opts->revs->pending.objects[i].name;
diff --git a/sequencer.h b/sequencer.h
index 20b708a..674f11e 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -10,16 +10,8 @@ enum replay_action {
 	REPLAY_PICK
 };
 
-enum replay_subcommand {
-	REPLAY_NONE,
-	REPLAY_REMOVE_STATE,
-	REPLAY_CONTINUE,
-	REPLAY_ROLLBACK
-};
-
 struct replay_opts {
 	enum replay_action action;
-	enum replay_subcommand subcommand;
 
 	/* Boolean options */
 	int edit;
@@ -48,11 +40,14 @@ struct replay_opts {
 	void **owned;
 	int owned_nr, owned_alloc;
 };
-#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL, NULL, 0, 0 }
+#define REPLAY_OPTS_INIT { -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL, NULL, 0, 0 }
 
 void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use);
 
 int sequencer_pick_revisions(struct replay_opts *opts);
+int sequencer_continue(struct replay_opts *opts);
+int sequencer_rollback(struct replay_opts *opts);
+int sequencer_remove_state(struct replay_opts *opts);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 12/22] sequencer: refactor the code to obtain a short commit name

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ba1fd05..06759d4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -157,13 +157,18 @@ struct commit_message {
 	const char *message;
 };
 
+static const char *short_commit_name(struct commit *commit)
+{
+	return find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+}
+
 static int get_message(struct commit *commit, struct commit_message *out)
 {
 	const char *abbrev, *subject;
 	int subject_len;
 
 	out->message = logmsg_reencode(commit, NULL, get_commit_output_encoding());
-	abbrev = find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+	abbrev = short_commit_name(commit);
 
 	subject_len = find_commit_subject(out->message, &subject);
 
@@ -647,8 +652,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
-		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
-		      msg.subject);
+		      short_commit_name(commit), msg.subject);
 		print_advice(res == 1, opts);
 		rerere(opts->allow_rerere_auto);
 		goto leave;
@@ -880,9 +884,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
 			opts->action == REPLAY_PICK ?  "pick" : "revert",
-			find_unique_abbrev(commit->object.oid.hash,
-				DEFAULT_ABBREV),
-			subject_len, subject);
+			short_commit_name(commit), subject_len, subject);
 		unuse_commit_buffer(commit, commit_buffer);
 	}
 	return 0;
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

The `git-rebase-todo` file contains a list of commands. Most of those
commands have the form

	<verb> <sha1> <oneline>

The <oneline> is displayed primarily for the user's convenience, as
rebase -i really interprets only the <verb> <sha1> part. However, there
are *some* places in interactive rebase where the <oneline> is used to
display messages, e.g. for reporting at which commit we stopped.

So let's just remember it when parsing the todo file; we keep a copy of
the entire todo file anyway (to write out the new `done` and
`git-rebase-todo` file just before processing each command), so all we
need to do is remember the begin and end offsets.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 06759d4..3398774 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	const char *arg;
+	int arg_len;
 	size_t offset_in_buf;
 };
 
@@ -760,6 +762,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
+	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_len = (int)(eol - item->arg);
+
 	if (status < 0)
 		return -1;
 
@@ -880,6 +885,8 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
+		item->arg = NULL;
+		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 14/22] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

In interactive rebases, we commit a little bit differently than the
sequencer did so far: we heed the "author-script", the "message" and
the "amend" files in the .git/rebase-merge/ subdirectory.

Likewise, we may want to edit the commit message *even* when providing
a file containing the suggested commit message. Therefore we change the
code to not even provide a default message when we do not want any, and
to call the editor explicitly.

As interactive rebase's GPG settings are configured differently from
how cherry-pick (and therefore sequencer) handles them, we will leave
support for that to the next commit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 sequencer.h |  3 ++
 2 files changed, 83 insertions(+), 12 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3398774..b124980 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,16 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+/*
+ * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
+ * GIT_AUTHOR_DATE that will be used for the commit that is currently
+ * being rebased.
+ */
+static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+
+/* We will introduce the 'interactive rebase' mode later */
+#define IS_REBASE_I() 0
+
 static const char *get_dir(const struct replay_opts *opts)
 {
 	return git_path_seq_dir();
@@ -377,20 +387,72 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static char **read_author_script(void)
+{
+	struct strbuf script = STRBUF_INIT;
+	int i, count = 0;
+	char *p, *p2, **env;
+	size_t env_size;
+
+	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
+		return NULL;
+
+	for (p = script.buf; *p; p++)
+		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
+			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
+		else if (*p == '\'')
+			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
+		else if (*p == '\n') {
+			*p = '\0';
+			count++;
+		}
+
+	env_size = (count + 1) * sizeof(*env);
+	strbuf_grow(&script, env_size);
+	memmove(script.buf + env_size, script.buf, script.len);
+	p = script.buf + env_size;
+	env = (char **)strbuf_detach(&script, NULL);
+
+	for (i = 0; i < count; i++) {
+		env[i] = p;
+		p += strlen(p) + 1;
+	}
+	env[count] = NULL;
+
+	return env;
+}
+
 /*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
  * author date and name.
  * If we are revert, or if our cherry-pick results in a hand merge,
- * we had better say that the current user is responsible for that.
+ * we had better say that the current user is responsible for that
+ * (except, of course, while running an interactive rebase).
  */
-static int run_git_commit(const char *defmsg, struct replay_opts *opts,
+int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty)
 {
+	char **env = NULL;
 	struct argv_array array;
 	int rc;
 	const char *value;
 
+	if (IS_REBASE_I()) {
+		env = read_author_script();
+		if (!env)
+			return error("You have staged changes in your working "
+				"tree. If these changes are meant to be\n"
+				"squashed into the previous commit, run:\n\n"
+				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"If they are meant to go into a new commit, "
+				"run:\n\n"
+				"  git commit $gpg_sign_opt_quoted\n\n"
+				"In both case, once you're done, continue "
+				"with:\n\n"
+				"  git rebase --continue\n");
+	}
+
 	argv_array_init(&array);
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
@@ -399,14 +461,13 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
 		argv_array_push(&array, "-s");
-	if (!opts->edit) {
-		argv_array_push(&array, "-F");
-		argv_array_push(&array, defmsg);
-		if (!opts->signoff &&
-		    !opts->record_origin &&
-		    git_config_get_value("commit.cleanup", &value))
-			argv_array_push(&array, "--cleanup=verbatim");
-	}
+	if (defmsg)
+		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (opts->edit)
+		argv_array_push(&array, "-e");
+	else if (!opts->signoff && !opts->record_origin &&
+		 git_config_get_value("commit.cleanup", &value))
+		argv_array_push(&array, "--cleanup=verbatim");
 
 	if (allow_empty)
 		argv_array_push(&array, "--allow-empty");
@@ -414,8 +475,11 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt(array.argv, RUN_GIT_CMD);
+	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+			(const char *const *)env);
 	argv_array_clear(&array);
+	free(env);
+
 	return rc;
 }
 
@@ -664,7 +728,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(git_path_merge_msg(), opts, allow);
+		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
+			opts, allow);
 
 leave:
 	free_message(commit, &msg);
@@ -859,6 +924,9 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
+	if (IS_REBASE_I())
+		return 0;
+
 	if (!file_exists(git_path_opts_file()))
 		return 0;
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
diff --git a/sequencer.h b/sequencer.h
index 674f11e..9f63c31 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -49,6 +49,9 @@ int sequencer_continue(struct replay_opts *opts);
 int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
+int sequencer_commit(const char *defmsg, struct replay_opts *opts,
+			  int allow_empty);
+
 extern const char sign_off_header[];
 
 void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 15/22] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

As we are slowly teaching the sequencer to perform the hard work for
the interactive rebase, we need to read files that were written by
shell scripts.

These files typically contain a single line and are invariably ended
by a line feed (and possibly a carriage return before that). Let's use
a helper to read such files and to remove the line ending.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index b124980..4204cc8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -239,6 +239,37 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
 	return 0;
 }
 
+/*
+ * Reads a file that was presumably written by a shell script, i.e.
+ * with an end-of-line marker that needs to be stripped.
+ *
+ * Returns 1 if the file was read, 0 if it could not be read or does not exist.
+ */
+static int read_oneliner(struct strbuf *buf,
+	const char *path, int skip_if_empty)
+{
+	int orig_len = buf->len;
+
+	if (!file_exists(path))
+		return 0;
+
+	if (strbuf_read_file(buf, path, 0) < 0) {
+		warning_errno("could not read '%s'", path);
+		return 0;
+	}
+
+	if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
+		if (--buf->len > orig_len && buf->buf[buf->len - 1] == '\r')
+			--buf->len;
+		buf->buf[buf->len] = '\0';
+	}
+
+	if (skip_if_empty && buf->len == orig_len)
+		return 0;
+
+	return 1;
+}
+
 static struct tree *empty_tree(void)
 {
 	return lookup_tree(EMPTY_TREE_SHA1_BIN);
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Johannes Schindelin]

The rebase command sports a `--gpg-sign` option that is heeded by the
interactive rebase.

This patch teaches the sequencer that trick, as part of the bigger
effort to make the sequencer the work horse of the interactive rebase.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 48 +++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 43 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4204cc8..e094ac2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -15,6 +15,7 @@
 #include "merge-recursive.h"
 #include "refs.h"
 #include "argv-array.h"
+#include "quote.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -33,6 +34,11 @@ static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
  * being rebased.
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+/*
+ * The following files are written by git-rebase just after parsing the
+ * command-line (and are only consumed, not modified, by the sequencer).
+ */
+static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
 /* We will introduce the 'interactive rebase' mode later */
 #define IS_REBASE_I() 0
@@ -129,6 +135,16 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
+static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
+{
+	static struct strbuf buf = STRBUF_INIT;
+
+	strbuf_reset(&buf);
+	if (opts->gpg_sign)
+		sq_quotef(&buf, "-S%s", opts->gpg_sign);
+	return buf.buf;
+}
+
 void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
 {
 	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
@@ -471,17 +487,20 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 
 	if (IS_REBASE_I()) {
 		env = read_author_script();
-		if (!env)
+		if (!env) {
+			const char *gpg_opt = gpg_sign_opt_quoted(opts);
+
 			return error("You have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"  git commit --amend %s\n\n"
 				"If they are meant to go into a new commit, "
 				"run:\n\n"
-				"  git commit $gpg_sign_opt_quoted\n\n"
+				"  git commit %s\n\n"
 				"In both case, once you're done, continue "
 				"with:\n\n"
-				"  git rebase --continue\n");
+				"  git rebase --continue\n", gpg_opt, gpg_opt);
+		}
 	}
 
 	argv_array_init(&array);
@@ -955,8 +974,27 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
-	if (IS_REBASE_I())
+	if (IS_REBASE_I()) {
+		struct strbuf buf = STRBUF_INIT;
+
+		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
+			if (buf.len && buf.buf[buf.len - 1] == '\n') {
+				if (--buf.len &&
+				    buf.buf[buf.len - 1] == '\r')
+					buf.len--;
+				buf.buf[buf.len] = '\0';
+			}
+
+			if (!starts_with(buf.buf, "-S"))
+				strbuf_reset(&buf);
+			else {
+				opts->gpg_sign = buf.buf + 2;
+				strbuf_detach(&buf, NULL);
+			}
+		}
+
 		return 0;
+	}
 
 	if (!file_exists(git_path_opts_file()))
 		return 0;
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 17/22] sequencer: allow editing the commit message on a case-by-case basis

[Johannes Schindelin]

In the upcoming commits, we will implement more and more of rebase
-i's functionality. One particular feature of the commands to come is
that some of them allow editing the commit message while others don't,
i.e. we cannot define in the replay_opts whether the commit message
should be edited or not.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++---
 sequencer.h | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index e094ac2..7e17d14 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -478,7 +478,7 @@ static char **read_author_script(void)
  * (except, of course, while running an interactive rebase).
  */
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty)
+			  int allow_empty, int edit)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -513,7 +513,7 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
-	if (opts->edit)
+	if (edit)
 		argv_array_push(&array, "-e");
 	else if (!opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
@@ -779,7 +779,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
-			opts, allow);
+			opts, allow, opts->edit);
 
 leave:
 	free_message(commit, &msg);
diff --git a/sequencer.h b/sequencer.h
index 9f63c31..fd02baf 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -50,7 +50,7 @@ int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty);
+			  int allow_empty, int edit);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 18/22] sequencer: support amending commits

[Johannes Schindelin]

This teaches the sequencer_commit() function to take an argument that
will allow us to implement "todo" commands that need to amend the commit
messages ("fixup", "squash" and "reword").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++--
 sequencer.h | 2 +-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 7e17d14..20f7590 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -478,7 +478,7 @@ static char **read_author_script(void)
  * (except, of course, while running an interactive rebase).
  */
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit)
+			  int allow_empty, int edit, int amend)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -507,6 +507,8 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
 
+	if (amend)
+		argv_array_push(&array, "--amend");
 	if (opts->gpg_sign)
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
@@ -779,7 +781,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
-			opts, allow, opts->edit);
+			opts, allow, opts->edit, 0);
 
 leave:
 	free_message(commit, &msg);
diff --git a/sequencer.h b/sequencer.h
index fd02baf..2106c0d 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -50,7 +50,7 @@ int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit);
+			  int allow_empty, int edit, int amend);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 19/22] sequencer: support cleaning up commit messages

[Johannes Schindelin]

The sequencer_commit() function already knows how to amend commits, and
with this new option, it can also clean up commit messages (i.e. strip
out commented lines). This is needed to implement rebase -i's 'fixup'
and 'squash' commands as sequencer commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 sequencer.h |  3 ++-
 2 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 20f7590..5ec956f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -478,7 +478,8 @@ static char **read_author_script(void)
  * (except, of course, while running an interactive rebase).
  */
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit, int amend)
+			  int allow_empty, int edit, int amend,
+			  int cleanup_commit_message)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -515,9 +516,12 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (cleanup_commit_message)
+		argv_array_push(&array, "--cleanup=strip");
 	if (edit)
 		argv_array_push(&array, "-e");
-	else if (!opts->signoff && !opts->record_origin &&
+	else if (!cleanup_commit_message &&
+		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
 		argv_array_push(&array, "--cleanup=verbatim");
 
@@ -781,7 +785,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
-			opts, allow, opts->edit, 0);
+			opts, allow, opts->edit, 0, 0);
 
 leave:
 	free_message(commit, &msg);
diff --git a/sequencer.h b/sequencer.h
index 2106c0d..e272549 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -50,7 +50,8 @@ int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit, int amend);
+			  int allow_empty, int edit, int amend,
+			  int cleanup_commit_message);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 20/22] sequencer: remember do_recursive_merge()'s return value

[Johannes Schindelin]

The return value of do_recursive_merge() may be positive (indicating merge
conflicts), se let's OR later error conditions so as not to overwrite them
with 0.

This is not yet a problem, but preparing for the patches to come: we will
teach the sequencer to do rebase -i's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5ec956f..0614b90 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -623,7 +623,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, allow;
+	int res = 0, unborn = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -734,7 +734,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
-		res = do_recursive_merge(base, next, base_label, next_label,
+		res |= do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
 			return res;
@@ -743,7 +743,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct commit_list *common = NULL;
 		struct commit_list *remotes = NULL;
 
-		res = write_message(&msgbuf, git_path_merge_msg());
+		res |= write_message(&msgbuf, git_path_merge_msg());
 
 		commit_list_insert(base, &common);
 		commit_list_insert(next, &remotes);
@@ -780,11 +780,12 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 
 	allow = allow_empty(opts, commit);
 	if (allow < 0) {
-		res = allow;
+		res |= allow;
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
+		res |= sequencer_commit(opts->edit ?
+				NULL : git_path_merge_msg(),
 			opts, allow, opts->edit, 0, 0);
 
 leave:
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 21/22] sequencer: left-trim the lines read from the script

[Johannes Schindelin]

Interactive rebase's scripts may be indented; We need to handle this
case, too, now that we prepare the sequencer to process interactive
rebases.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 0614b90..5efed2e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -864,6 +864,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	/* left-trim */
+	bol += strspn(bol, " \t");
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
-- 
2.10.0.rc1.114.g2bd6b38

[PATCH 22/22] sequencer: refactor write_message()

[Johannes Schindelin]

The write_message() function safely writes an strbuf to a file.
Sometimes this is inconvenient, though: the text to be written may not
be stored in a strbuf, or the strbuf should not be released after
writing.

Let's allow for such use cases by refactoring write_message() to allow
for a convenience function write_file_gently(). As some of the upcoming
callers of that new function will want to append a newline character,
let's just add a flag for that, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5efed2e..f5b5e5e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -239,22 +239,37 @@ static void print_advice(int show_hint, struct replay_opts *opts)
 	}
 }
 
-static int write_message(struct strbuf *msgbuf, const char *filename)
+static int write_with_lock_file(const char *filename,
+				const void *buf, size_t len, int append_eol)
 {
 	static struct lock_file msg_file;
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
 		return error_errno(_("Could not lock '%s'"), filename);
-	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
+	if (write_in_full(msg_fd, buf, len) < 0)
 		return error_errno(_("Could not write to %s"), filename);
-	strbuf_release(msgbuf);
+	if (append_eol && write(msg_fd, "\n", 1) < 0)
+		return error_errno(_("Could not write eol to %s"), filename);
 	if (commit_lock_file(&msg_file) < 0)
 		return error(_("Error wrapping up %s."), filename);
 
 	return 0;
 }
 
+static int write_message(struct strbuf *msgbuf, const char *filename)
+{
+	int res = write_with_lock_file(filename, msgbuf->buf, msgbuf->len, 0);
+	strbuf_release(msgbuf);
+	return res;
+}
+
+static int write_file_gently(const char *filename,
+			     const char *text, int append_eol)
+{
+	return write_with_lock_file(filename, text, strlen(text), append_eol);
+}
+
 /*
  * Reads a file that was presumably written by a shell script, i.e.
  * with an end-of-line marker that needs to be stripped.
-- 
2.10.0.rc1.114.g2bd6b38

Re: [PATCH 01/22] sequencer: use static initializers for replay_opts

[Dennis Kaarsemaker]

On ma, 2016-08-29 at 10:03 +0200, Johannes Schindelin wrote:

> +#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }

This looked off to me, as it replaces memset(..., 0, ...) so is not
100% equivalent. But the changed functions both set opts.action and
call parse_args which sets opts.subcommand.

D.

Re: [PATCH 04/22] sequencer: future-proof remove_sequencer_state()

[Dennis Kaarsemaker]

On ma, 2016-08-29 at 10:04 +0200, Johannes Schindelin wrote:

> +       if (read_and_refresh_cache(opts))
> +               return -1;
> +

This doesn't seem to be related to the get_dir changes?

D.

Re: [PATCH 12/22] sequencer: refactor the code to obtain a short commit name

[Dennis Kaarsemaker]

On ma, 2016-08-29 at 10:05 +0200, Johannes Schindelin wrote:

<snip actual commit>

I fail to see the point of this patch, would you mind enlightening me?

D.

Re: [PATCH 15/22] sequencer: introduce a helper to read files written by scripts

[Dennis Kaarsemaker]

On ma, 2016-08-29 at 10:06 +0200, Johannes Schindelin wrote:
> +       if (strbuf_read_file(buf, path, 0) < 0) {
> +               warning_errno("could not read '%s'", path);
> +               return 0;
> +       }
> +
> +       if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
> +               if (--buf->len > orig_len && buf->buf[buf->len - 1]
> == '\r')
> +                       --buf->len;
> +               buf->buf[buf->len] = '\0';
> +       }

Why not use open + strbuf_getline instead of hand-rolling a newline
eradicator?

D.

Re: [PATCH 20/22] sequencer: remember do_recursive_merge()'s return value

[Dennis Kaarsemaker]

On ma, 2016-08-29 at 10:06 +0200, Johannes Schindelin wrote:

> The return value of do_recursive_merge() may be positive (indicating merge
> conflicts), se let's OR later error conditions so as not to overwrite them
> with 0.

s/se/so/?

D.

Re: [PATCH 00/22] Prepare the sequencer for the upcoming rebase -i patches

[Dennis Kaarsemaker]

On ma, 2016-08-29 at 10:03 +0200, Johannes Schindelin wrote:

> Therefore I would be most grateful for every in-depth review.

Tried to do that, but could come up only with a few nits. I think the
approach is sensible.

D.

Re: [PATCH 01/22] sequencer: use static initializers for replay_opts

[Johannes Schindelin]

Hi Dennis,

On Mon, 29 Aug 2016, Dennis Kaarsemaker wrote:

> On ma, 2016-08-29 at 10:03 +0200, Johannes Schindelin wrote:
> 
> > +#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }
> 
> This looked off to me, as it replaces memset(..., 0, ...) so is not
> 100% equivalent. But the changed functions both set opts.action and
> call parse_args which sets opts.subcommand.

Okay... Do you want me to change anything?

Ciao,
Dscho

Re: [PATCH 04/22] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

Hi Dennis,

On Mon, 29 Aug 2016, Dennis Kaarsemaker wrote:

> On ma, 2016-08-29 at 10:04 +0200, Johannes Schindelin wrote:
> 
> > +       if (read_and_refresh_cache(opts))
> > +               return -1;
> > +
> 
> This doesn't seem to be related to the get_dir changes?

Good eyes.

Let me investigate why I have it here...

Ciao,
Dscho

Re: [PATCH 12/22] sequencer: refactor the code to obtain a short commit name

[Johannes Schindelin]

Hi Dennis,

On Mon, 29 Aug 2016, Dennis Kaarsemaker wrote:

> On ma, 2016-08-29 at 10:05 +0200, Johannes Schindelin wrote:
> 
> <snip actual commit>
> 
> I fail to see the point of this patch, would you mind enlightening me?

Two reasons:

1) by refactoring it into a function, the code is more DRY (with all the
advantages that come with it, such as: only a single point to change if
changing the behavior)

2) it is easier to reuse the code in upcoming patches (that would be in
the next patch series)

Will amend the commit message.

Ciao,
Dscho

Re: [PATCH 15/22] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 925 bytes --]

Hi Dennis,

On Mon, 29 Aug 2016, Dennis Kaarsemaker wrote:

> On ma, 2016-08-29 at 10:06 +0200, Johannes Schindelin wrote:
> > +       if (strbuf_read_file(buf, path, 0) < 0) {
> > +               warning_errno("could not read '%s'", path);
> > +               return 0;
> > +       }
> > +
> > +       if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
> > +               if (--buf->len > orig_len && buf->buf[buf->len - 1]
> > == '\r')
> > +                       --buf->len;
> > +               buf->buf[buf->len] = '\0';
> > +       }
> 
> Why not use open + strbuf_getline instead of hand-rolling a newline
> eradicator?

Because strbuf_getline() erases the strbuf instead of appending to it
(which is what we sometimes need when converting shell scripts to C).

Ciao,
Dscho

Re: [PATCH 20/22] sequencer: remember do_recursive_merge()'s return value

[Johannes Schindelin]

Hi Dennis,

On Mon, 29 Aug 2016, Dennis Kaarsemaker wrote:

> On ma, 2016-08-29 at 10:06 +0200, Johannes Schindelin wrote:
> 
> > The return value of do_recursive_merge() may be positive (indicating merge
> > conflicts), se let's OR later error conditions so as not to overwrite them
> > with 0.
> 
> s/se/so/?

Good eyes.

Fixed,
Dscho

Re: [PATCH 00/22] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

Hi Dennis,

On Mon, 29 Aug 2016, Dennis Kaarsemaker wrote:

> On ma, 2016-08-29 at 10:03 +0200, Johannes Schindelin wrote:
> 
> > Therefore I would be most grateful for every in-depth review.
> 
> Tried to do that, but could come up only with a few nits. I think the
> approach is sensible.

Thank you for the review!

Ciao,
Dscho

Re: [PATCH 04/22] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

Hi Dennis,

On Mon, 29 Aug 2016, Johannes Schindelin wrote:

> On Mon, 29 Aug 2016, Dennis Kaarsemaker wrote:
> 
> > On ma, 2016-08-29 at 10:04 +0200, Johannes Schindelin wrote:
> > 
> > > +       if (read_and_refresh_cache(opts))
> > > +               return -1;
> > > +
> > 
> > This doesn't seem to be related to the get_dir changes?
> 
> Good eyes.
> 
> Let me investigate why I have it here...

Unfortunately my reflogs got corrupted by the git-worktree
implementations, so I cannot back that far.

Looking at the code, and after running the tests, I am convinced that it
is a leftover of some misguided attempt to implement "git rebase -i
--abort" in sequencer_rollback().

I removed this hunk from the patch.

Again, Thank you so much for your review!
Dscho

Re: [PATCH 01/22] sequencer: use static initializers for replay_opts

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

Hi Kuba,

On Mon, 29 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 11:19, Dennis Kaarsemaker pisze:
> > On ma, 2016-08-29 at 10:03 +0200, Johannes Schindelin wrote:
> > 
> >> +#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }
> > 
> > This looked off to me, as it replaces memset(..., 0, ...) so is not
> > 100% equivalent. But the changed functions both set opts.action and
> > call parse_args which sets opts.subcommand.
> 
> This information would be nice to have in the commit message.

Clarified in the commit message.

Ciao,
Johannes

Re: [PATCH 02/22] sequencer: use memoized sequencer directory path

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]

Hi Kuba,

On Mon, 29 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  builtin/commit.c |  2 +-
> >  sequencer.c      | 11 ++++++-----
> >  sequencer.h      |  5 +----
> >  3 files changed, 8 insertions(+), 10 deletions(-)
> 
> Just a sidenote: it would be probably easier to read with *.h before
> *.c (at least this particular one).

I agree, but I did not find any way to reorder this without substantial
manual work...

> > diff --git a/builtin/commit.c b/builtin/commit.c
> > index 77e3dc8..0221190 100644
> > --- a/builtin/commit.c
> > +++ b/builtin/commit.c
> > @@ -173,7 +173,7 @@ static void determine_whence(struct wt_status *s)
> >  		whence = FROM_MERGE;
> >  	else if (file_exists(git_path_cherry_pick_head())) {
> >  		whence = FROM_CHERRY_PICK;
> > -		if (file_exists(git_path(SEQ_DIR)))
> > +		if (file_exists(git_path_seq_dir()))
> >  			sequencer_in_use = 1;
> >  	}
> >  	else
> 
> So it is more "Use memoized sequencer directory path" rather than
> "sequencer: use memoized sequencer directory path" - it replaces
> all occurrences of SEQ_DIR,... that's why it can be removed from
> 'sequencer.h'.
> 
> Though perhaps I misunderstood "sequencer: " prefix there.  Don't
> mind me then.

The idea is that this path is declared and defined in the sequencer. There
are other call sites, too, so they have to be changed at the same time...

I'd really like to keep the "sequencer:" prefix because it is semantically
correct: this change is about the sequencer, not about the other call
sites.

Ciao,
Johannes

Re: [PATCH 03/22] sequencer: avoid unnecessary indirection

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

Hi Kuba,

On Mon, 29 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> 
> > We really do not need the *pointer to a* pointer to the options in
> > the read_populate_opts() function.
> 
> Right.
>  
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/sequencer.c b/sequencer.c
> > index 4d2b4e3..14ef79b 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -809,11 +809,11 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
> >  	return 0;
> >  }
> >  
> > -static int read_populate_opts(struct replay_opts **opts)
> > +static int read_populate_opts(struct replay_opts *opts)
> 
> Especially that other *_populate_*() use 'struct replay_opts *opts':
> 
>    read_populate_todo(struct commit_list **todo_list, struct replay_opts *opts)
>    walk_revs_populate_todo(struct commit_list **todo_list, struct replay_opts *opts)
> 
> Though they use **todo_list, because they modify this list;
> maybe that was why read_populate_opts was using **opts instead
> of *opts?

I won't speculate about the reasons why it was made so.

About read_populate_todo(): it uses **todo_list, but still only *opts.

In any case, in a later patch, the todo_list parsing is completely
revamped anyway, so I did not want to "fix" anything that would get
reverted later on.

Ciao,
Johannes

Re: [PATCH 01/22] sequencer: use static initializers for replay_opts

[Jakub Narębski]

W dniu 29.08.2016 o 11:19, Dennis Kaarsemaker pisze:
> On ma, 2016-08-29 at 10:03 +0200, Johannes Schindelin wrote:
> 
>> +#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }
> 
> This looked off to me, as it replaces memset(..., 0, ...) so is not
> 100% equivalent. But the changed functions both set opts.action and
> call parse_args which sets opts.subcommand.

This information would be nice to have in the commit message.

-- 
Jakub Narębski

Re: [PATCH 02/22] sequencer: use memoized sequencer directory path

[Jakub Narębski]

W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  builtin/commit.c |  2 +-
>  sequencer.c      | 11 ++++++-----
>  sequencer.h      |  5 +----
>  3 files changed, 8 insertions(+), 10 deletions(-)

Just a sidenote: it would be probably easier to read with *.h before
*.c (at least this particular one).

> diff --git a/builtin/commit.c b/builtin/commit.c
> index 77e3dc8..0221190 100644
> --- a/builtin/commit.c
> +++ b/builtin/commit.c
> @@ -173,7 +173,7 @@ static void determine_whence(struct wt_status *s)
>  		whence = FROM_MERGE;
>  	else if (file_exists(git_path_cherry_pick_head())) {
>  		whence = FROM_CHERRY_PICK;
> -		if (file_exists(git_path(SEQ_DIR)))
> +		if (file_exists(git_path_seq_dir()))
>  			sequencer_in_use = 1;
>  	}
>  	else

So it is more "Use memoized sequencer directory path" rather than
"sequencer: use memoized sequencer directory path" - it replaces
all occurrences of SEQ_DIR,... that's why it can be removed from
'sequencer.h'.

Though perhaps I misunderstood "sequencer: " prefix there.  Don't
mind me then.

> diff --git a/sequencer.c b/sequencer.c
> index b6481bb..4d2b4e3 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -21,10 +21,11 @@
>  const char sign_off_header[] = "Signed-off-by: ";
>  static const char cherry_picked_prefix[] = "(cherry picked from commit ";
>  
> -static GIT_PATH_FUNC(git_path_todo_file, SEQ_TODO_FILE)
> -static GIT_PATH_FUNC(git_path_opts_file, SEQ_OPTS_FILE)
> -static GIT_PATH_FUNC(git_path_seq_dir, SEQ_DIR)
> -static GIT_PATH_FUNC(git_path_head_file, SEQ_HEAD_FILE)
> +GIT_PATH_FUNC(git_path_seq_dir, "sequencer")
> +
> +static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
> +static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
> +static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")

This also makes the ordering of memoized-path variables more
sensible.  Good work.

>  
>  static int is_rfc2822_line(const char *buf, int len)
>  {
> @@ -112,7 +113,7 @@ static void remove_sequencer_state(void)
>  {
>  	struct strbuf seq_dir = STRBUF_INIT;
>  
> -	strbuf_addstr(&seq_dir, git_path(SEQ_DIR));
> +	strbuf_addstr(&seq_dir, git_path_seq_dir());
>  	remove_dir_recursively(&seq_dir, 0);
>  	strbuf_release(&seq_dir);
>  }
> diff --git a/sequencer.h b/sequencer.h
> index 2ca096b..c955594 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -1,10 +1,7 @@
>  #ifndef SEQUENCER_H
>  #define SEQUENCER_H
>  
> -#define SEQ_DIR		"sequencer"
> -#define SEQ_HEAD_FILE	"sequencer/head"
> -#define SEQ_TODO_FILE	"sequencer/todo"
> -#define SEQ_OPTS_FILE	"sequencer/opts"
> +const char *git_path_seq_dir(void);

Right, I see this matches other git_path_*() functions declared in cache.h

>  
>  #define APPEND_SIGNOFF_DEDUP (1u << 0)
>  
> 

Re: [PATCH 03/22] sequencer: avoid unnecessary indirection

[Jakub Narębski]

W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:

> We really do not need the *pointer to a* pointer to the options in
> the read_populate_opts() function.

Right.
 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 4d2b4e3..14ef79b 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -809,11 +809,11 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
>  	return 0;
>  }
>  
> -static int read_populate_opts(struct replay_opts **opts)
> +static int read_populate_opts(struct replay_opts *opts)

Especially that other *_populate_*() use 'struct replay_opts *opts':

   read_populate_todo(struct commit_list **todo_list, struct replay_opts *opts)
   walk_revs_populate_todo(struct commit_list **todo_list, struct replay_opts *opts)

Though they use **todo_list, because they modify this list;
maybe that was why read_populate_opts was using **opts instead
of *opts?

>  {
>  	if (!file_exists(git_path_opts_file()))
>  		return 0;
> -	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
> +	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
>  		return error(_("Malformed options sheet: %s"),
>  			git_path_opts_file());
>  	return 0;
> @@ -1038,7 +1038,7 @@ static int sequencer_continue(struct replay_opts *opts)
>  
>  	if (!file_exists(git_path_todo_file()))
>  		return continue_single_pick();
> -	if (read_populate_opts(&opts) ||
> +	if (read_populate_opts(opts) ||
>  			read_populate_todo(&todo_list, opts))
>  		return -1;
>  
> 

Re: [PATCH 20/22] sequencer: remember do_recursive_merge()'s return value

[Jakub Narębski]

W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

> diff --git a/sequencer.c b/sequencer.c
> index 5ec956f..0614b90 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -623,7 +623,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	const char *base_label, *next_label;
>  	struct commit_message msg = { NULL, NULL, NULL, NULL };
>  	struct strbuf msgbuf = STRBUF_INIT;
> -	int res, unborn = 0, allow;
> +	int res = 0, unborn = 0, allow;

Not that I am against this part of change, making initialization
explicit, but why we are initializing automatic variables with 0,
which would be the default value anyway?  I thought our coding
guidelines discourage initializing with 0 or NULL...

Puzzled,
-- 
Jakub Narębski

Re: [PATCH 20/22] sequencer: remember do_recursive_merge()'s return value

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
>
>> diff --git a/sequencer.c b/sequencer.c
>> index 5ec956f..0614b90 100644
>> --- a/sequencer.c
>> +++ b/sequencer.c
>> @@ -623,7 +623,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>>  	const char *base_label, *next_label;
>>  	struct commit_message msg = { NULL, NULL, NULL, NULL };
>>  	struct strbuf msgbuf = STRBUF_INIT;
>> -	int res, unborn = 0, allow;
>> +	int res = 0, unborn = 0, allow;
>
> Not that I am against this part of change, making initialization
> explicit, but why we are initializing automatic variables with 0,
> which would be the default value anyway?

Because an on-stack "auto" begins its life with an undefined value,
unlike a file-scope static (and global variables) that can be in BSS
segment.

> I thought our coding
> guidelines discourage initializing with 0 or NULL...

You are confused between the two, I am afraid.

Re: [PATCH 14/22] sequencer: prepare for rebase -i's commit functionality

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +/* We will introduce the 'interactive rebase' mode later */
> +#define IS_REBASE_I() 0

I do not see a point in naming this all caps.  The use site would be
a lot more pleasant to read when the reader does not have to care if
this is implemented as a preprocessor macro or a helper function.

> @@ -377,20 +387,72 @@ static int is_index_unchanged(void)
>  	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
>  }
>  
> +static char **read_author_script(void)
> +{
> +	struct strbuf script = STRBUF_INIT;
> +	int i, count = 0;
> +	char *p, *p2, **env;
> +	size_t env_size;
> +
> +	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
> +		return NULL;
> +
> +	for (p = script.buf; *p; p++)
> +		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
> +			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
> +		else if (*p == '\'')
> +			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
> +		else if (*p == '\n') {
> +			*p = '\0';
> +			count++;
> +		}

Hmph.  What is this loop doing?  Is it decoding a sq-quoted buffer
or something?  Don't we have a helper function to do that?

> +	env_size = (count + 1) * sizeof(*env);
> +	strbuf_grow(&script, env_size);
> +	memmove(script.buf + env_size, script.buf, script.len);
> +	p = script.buf + env_size;
> +	env = (char **)strbuf_detach(&script, NULL);
> +
> +	for (i = 0; i < count; i++) {
> +		env[i] = p;
> +		p += strlen(p) + 1;
> +	}
> +	env[count] = NULL;
> +
> +	return env;
> +}
> +
>  /*
>   * If we are cherry-pick, and if the merge did not result in
>   * hand-editing, we will hit this commit and inherit the original
>   * author date and name.
>   * If we are revert, or if our cherry-pick results in a hand merge,
> - * we had better say that the current user is responsible for that.
> + * we had better say that the current user is responsible for that
> + * (except, of course, while running an interactive rebase).
>   */

The added "(except, ...)" reads as if "even if we are reverting, if
that is done as part of an interactive rebase, the authorship rule
for a revert does not apply".

If that is not what you meant, i.e. if you did not mean to imply
that "rebase -i" doing a revert is a normal thing, this needs to be
rephrased to avoid the misinterpretation.

Re: [PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Jakub Narębski]

W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:

> The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
> like a one-shot command when it reads its configuration: memory is
> allocated and released only when the command exits.
> 
> This is kind of okay for git-cherry-pick, which *is* a one-shot
> command. All the work to make the sequencer its work horse was
> done to allow using the functionality as a library function, though,
> including proper clean-up after use.
> 
> This patch introduces an API to pass the responsibility of releasing
> certain memory to the sequencer.

So how this API would be / is meant to be used?  From the following
patches (which I shouldn't have to read to understand this one)
it looks like it is about strdup'ed strings from option parsing.
Or would there be something more in the future?

Would sequencer as a library function be called multiple times,
or only once?


I'm trying to find out how this is solved in other places of Git
code, and I have stumbled upon free_util in string_list...

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 13 +++++++++++++
>  sequencer.h |  8 +++++++-
>  2 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index c4b223b..b5be0f9 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -114,9 +114,22 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
>  	return 1;
>  }
>  
> +void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
> +{
> +	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
> +	opts->owned[opts->owned_nr++] = set_me_free_after_use;
> +
> +	return set_me_free_after_use;

I was wondering what this 'set_me_free_after_use' parameter is about;
wouldn't it be more readable if this parameter was called 'owned_data'
or 'owned_ptr'?

> +}
> +
>  static void remove_sequencer_state(const struct replay_opts *opts)
>  {
>  	struct strbuf dir = STRBUF_INIT;
> +	int i;
> +
> +	for (i = 0; i < opts->owned_nr; i++)
> +		free(opts->owned[i]);

I guess you can remove owned data in any order, regardless if you
store struct or its members first...

> +	free(opts->owned);
>  
>  	strbuf_addf(&dir, "%s", get_dir(opts));
>  	remove_dir_recursively(&dir, 0);
> diff --git a/sequencer.h b/sequencer.h
> index c955594..20b708a 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -43,8 +43,14 @@ struct replay_opts {
>  
>  	/* Only used by REPLAY_NONE */
>  	struct rev_info *revs;
> +
> +	/* malloc()ed data entrusted to the sequencer */
> +	void **owned;
> +	int owned_nr, owned_alloc;

I'm not sure about naming conventions for those types of data, but
wouldn't 'owned_data' be a better name?  I could be wrong here...

>  };
> -#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }
> +#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL, NULL, 0, 0 }

Nb. it is a pity that we cannot use named initializers for structs,
so called designated inits.  It would make this macro more readable.

> +
> +void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use);
>  
>  int sequencer_pick_revisions(struct replay_opts *opts);
>  
> 

Re: [PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Sixt]

Am 29.08.2016 um 23:59 schrieb Jakub Narębski:
> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
>> -#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL }
>> +#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL, NULL, 0, 0 }
>
> Nb. it is a pity that we cannot use named initializers for structs,
> so called designated inits.  It would make this macro more readable.

It is actually pointless to add the 0's and NULL's here. This should  be 
sufficient:

#define REPLAY_OPTS_INIT { -1, -1 }

because initialization with 0 (or NULL) is the default for any omitted 
members.

-- Hannes

Re: [PATCH 14/22] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

Hi Junio,

On Mon, 29 Aug 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +/* We will introduce the 'interactive rebase' mode later */
> > +#define IS_REBASE_I() 0
> 
> I do not see a point in naming this all caps.

Old habit. Macros are all-caps.

> The use site would be a lot more pleasant to read when the reader does
> not have to care if this is implemented as a preprocessor macro or a
> helper function.

I converted this to a helper function.

> > @@ -377,20 +387,72 @@ static int is_index_unchanged(void)
> >  	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
> >  }
> >  
> > +static char **read_author_script(void)
> > +{
> > +	struct strbuf script = STRBUF_INIT;
> > +	int i, count = 0;
> > +	char *p, *p2, **env;
> > +	size_t env_size;
> > +
> > +	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
> > +		return NULL;
> > +
> > +	for (p = script.buf; *p; p++)
> > +		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
> > +			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
> > +		else if (*p == '\'')
> > +			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
> > +		else if (*p == '\n') {
> > +			*p = '\0';
> > +			count++;
> > +		}
> 
> Hmph.  What is this loop doing?  Is it decoding a sq-quoted buffer
> or something?  Don't we have a helper function to do that?

Well, it is not just decoding an sq-quoted buffer, but several lines with
definitions we sq-quoted ourselves, individually.

The quote.[ch] code currently has no code to dequote lines individually.

At this point, I would prefer to keep this code as-is, as I tested it over
the course of months and do not want to introduce regressions *now*.

> > +	env_size = (count + 1) * sizeof(*env);
> > +	strbuf_grow(&script, env_size);
> > +	memmove(script.buf + env_size, script.buf, script.len);
> > +	p = script.buf + env_size;
> > +	env = (char **)strbuf_detach(&script, NULL);
> > +
> > +	for (i = 0; i < count; i++) {
> > +		env[i] = p;
> > +		p += strlen(p) + 1;
> > +	}
> > +	env[count] = NULL;
> > +
> > +	return env;
> > +}
> > +
> >  /*
> >   * If we are cherry-pick, and if the merge did not result in
> >   * hand-editing, we will hit this commit and inherit the original
> >   * author date and name.
> >   * If we are revert, or if our cherry-pick results in a hand merge,
> > - * we had better say that the current user is responsible for that.
> > + * we had better say that the current user is responsible for that
> > + * (except, of course, while running an interactive rebase).
> >   */
> 
> The added "(except, ...)" reads as if "even if we are reverting, if
> that is done as part of an interactive rebase, the authorship rule
> for a revert does not apply".
> 
> If that is not what you meant, i.e. if you did not mean to imply
> that "rebase -i" doing a revert is a normal thing, this needs to be
> rephrased to avoid the misinterpretation.

I rephrased it.

Ciao,
Dscho

Re: [PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3032 bytes --]

Hi Kuba,

On Mon, 29 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> 
> > The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
> > like a one-shot command when it reads its configuration: memory is
> > allocated and released only when the command exits.
> > 
> > This is kind of okay for git-cherry-pick, which *is* a one-shot
> > command. All the work to make the sequencer its work horse was done to
> > allow using the functionality as a library function, though, including
> > proper clean-up after use.
> > 
> > This patch introduces an API to pass the responsibility of releasing
> > certain memory to the sequencer.
> 
> So how this API would be / is meant to be used?

I added an example to the commit message.

> Would sequencer as a library function be called multiple times,
> or only once?

The point of a library function is that it should not care.

> I'm trying to find out how this is solved in other places of Git
> code, and I have stumbled upon free_util in string_list...

I wanted this to be flexible enough to take care of any type of data, not
just strings.

And while the string_list has a void *util field, it would be rather silly
to add strings to a string list for the sole purpose of free()ing their
util fields in the end.

(That was the conclusion I came to after a search of my own.)

> > +void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
> > +{
> > +	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
> > +	opts->owned[opts->owned_nr++] = set_me_free_after_use;
> > +
> > +	return set_me_free_after_use;
> 
> I was wondering what this 'set_me_free_after_use' parameter is about;
> wouldn't it be more readable if this parameter was called 'owned_data'
> or 'owned_ptr'?

If I read "owned_ptr" as a function's parameter, I would assume that the
associated memory is owned by the caller. So I would be puzzled reading
that name.

> >  static void remove_sequencer_state(const struct replay_opts *opts)
> >  {
> >  	struct strbuf dir = STRBUF_INIT;
> > +	int i;
> > +
> > +	for (i = 0; i < opts->owned_nr; i++)
> > +		free(opts->owned[i]);
> 
> I guess you can remove owned data in any order, regardless if you
> store struct or its members first...

Indeed, this is not like a C++ destructor. It's free().

> > diff --git a/sequencer.h b/sequencer.h
> > index c955594..20b708a 100644
> > --- a/sequencer.h
> > +++ b/sequencer.h
> > @@ -43,8 +43,14 @@ struct replay_opts {
> >  
> >  	/* Only used by REPLAY_NONE */
> >  	struct rev_info *revs;
> > +
> > +	/* malloc()ed data entrusted to the sequencer */
> > +	void **owned;
> > +	int owned_nr, owned_alloc;
> 
> I'm not sure about naming conventions for those types of data, but
> wouldn't 'owned_data' be a better name?  I could be wrong here...

The convention seemed to be "void *X; int X_nr, X_alloc;", so I stuck with
it.

Thanks for your review!
Johannes

Re: [PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 871 bytes --]

Hi Hannes,

On Tue, 30 Aug 2016, Johannes Sixt wrote:

> Am 29.08.2016 um 23:59 schrieb Jakub Narębski:
> > W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> > > -#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL,
> > > NULL, NULL, 0, 0, NULL }
> > > +#define REPLAY_OPTS_INIT { -1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL,
> > > NULL, NULL, 0, 0, NULL, NULL, 0, 0 }
> >
> > Nb. it is a pity that we cannot use named initializers for structs,
> > so called designated inits.  It would make this macro more readable.
> 
> It is actually pointless to add the 0's and NULL's here. This should  be
> sufficient:
> 
> #define REPLAY_OPTS_INIT { -1, -1 }
> 
> because initialization with 0 (or NULL) is the default for any omitted
> members.

D'oh. You're right. The same applies to TODO_LIST_INIT, of course.

Fixed,
Johannes

Re: [PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Jakub Narębski]

Hello Johannes,

W dniu 30.08.2016 o 09:29, Johannes Schindelin pisze:
> On Mon, 29 Aug 2016, Jakub Narębski wrote: 
>> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:

>>> +void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
>>> +{
>>> +	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
>>> +	opts->owned[opts->owned_nr++] = set_me_free_after_use;
>>> +
>>> +	return set_me_free_after_use;
>>
>> I was wondering what this 'set_me_free_after_use' parameter is about;
>> wouldn't it be more readable if this parameter was called 'owned_data'
>> or 'owned_ptr'?
> 
> If I read "owned_ptr" as a function's parameter, I would assume that the
> associated memory is owned by the caller. So I would be puzzled reading
> that name.

Right.  Well, it is difficult to come up with a good name for this
parameter that would make sense both in a declaration as an information
for a caller, and in the function itself as information about what it
holds.

In my personal opinion 'set_me_free_after_use' is not the best name,
but I unfortunately do not have a better proposal.  Maybe 'entrust_ptr',
or 'entrusted_data' / 'entrusted_ptr' / 'entrusted'?

There are two hard things in computer science: cache invalidation, 
*naming things*, and off-by-one errors ;-)


P.S. It would be nice to have generic mechanism for taking custody
of data to help libification, either at this or at lower level (on
the level of xstrdup, etc.), but that can safely wait.  It even should
wait, so that we can see that this approach is a good one, before
trying to generalize it.  That should be not a blocker for this series,
IMVHO.

Best,
-- 
Jakub Narębski

Re: [PATCH 06/22] sequencer: release memory that was allocated when reading options

[Jakub Narębski]

W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:

> The sequencer reads options from disk and stores them in its struct
> for use during sequencer's operations.
> 
> With this patch, the memory is released afterwards, plugging a
> memory leak.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 13 ++++++++++---
>  1 file changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index b5be0f9..8d79091 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -131,6 +131,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
>  		free(opts->owned[i]);
>  	free(opts->owned);
>  
> +	free(opts->xopts);
> +

This looks like independent change, not related to using the
sequencer_entrust() to store options read from disk in replay_opts
struct to be able to free memory afterwards.

I guess you wanted to avoid one line changes...

>  	strbuf_addf(&dir, "%s", get_dir(opts));
>  	remove_dir_recursively(&dir, 0);
>  	strbuf_release(&dir);
> @@ -811,13 +813,18 @@ static int populate_opts_cb(const char *key, const char *value, void *data)

Sidenote: this patch would be easier to read if lines were reordered
as below, but I don't think any slider heuristics could help achieve
that automatically.  Also, the patch might be invalid...

>  		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
>  	else if (!strcmp(key, "options.mainline"))
>  		opts->mainline = git_config_int(key, value);
> -	else if (!strcmp(key, "options.strategy"))
> +	else if (!strcmp(key, "options.strategy")) {
>  		git_config_string(&opts->strategy, key, value);
> +		sequencer_entrust(opts, (char *) opts->strategy);

I wonder if the ability to free strings dup-ed by git_config_string()
be something that is part of replay_opts, or rather remove_sequencer_state(),
that is a list of

	free(opts->strategy);
	free(opts->gpg_sign);

And of course

	for (i = 0; i < opts->xopts_nr; i++)
		free(opts->xopts[i]);
	free(opts->xopts);

Though... free(NULL) is nop as per standard, but can we rely on it?
If it is a problem, we can create xfree(ptr) being if(ptr)free(ptr);

The *_entrust() mechanism is more generic, but do we use this general-ness?
Well, it could be xstrdup or git_config_string doing entrust'ing...


> +	}
> -	else if (!strcmp(key, "options.gpg-sign"))
> +	else if (!strcmp(key, "options.gpg-sign")) {
>  		git_config_string(&opts->gpg_sign, key, value);
> +		sequencer_entrust(opts, (char *) opts->gpg_sign);
> +	}
>  	else if (!strcmp(key, "options.strategy-option")) {
>  		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
> -		opts->xopts[opts->xopts_nr++] = xstrdup(value);
> +		opts->xopts[opts->xopts_nr++] =
> +			sequencer_entrust(opts, xstrdup(value));

Nice.

>  	} else
>  		return error(_("Invalid key: %s"), key);
>  
> 

Re: [PATCH 07/22] sequencer: future-proof read_populate_todo()

[Jakub Narębski]

W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:

> Over the next commits, we will work on improving the sequencer to the
> point where it can process the edit script of an interactive rebase. To
> that end, we will need to teach the sequencer to read interactive
> rebase's todo file. In preparation, we consolidate all places where
> that todo file is needed to call a function that we will later extend.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 18 +++++++++++-------
>  1 file changed, 11 insertions(+), 7 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 8d79091..982b6e9 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -32,6 +32,11 @@ static const char *get_dir(const struct replay_opts *opts)
>  	return git_path_seq_dir();
>  }
>  
> +static const char *get_todo_path(const struct replay_opts *opts)
> +{
> +	return git_path_todo_file();
> +}

I guess that in the future commit the return value of get_todo_path()
would change depending on what sequencer is used for, cherry-pick or
interactive rebase, that is, contents of replay_opts...

> +
>  static int is_rfc2822_line(const char *buf, int len)
>  {
>  	int i;
> @@ -772,25 +777,24 @@ static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
>  static int read_populate_todo(struct commit_list **todo_list,
>  			struct replay_opts *opts)
>  {
> +	const char *todo_file = get_todo_path(opts);

...and that's why you have added this temporary variable here, to
not repeat get_todo_path(opts) calculations...

>  	struct strbuf buf = STRBUF_INIT;
>  	int fd, res;
>  
> -	fd = open(git_path_todo_file(), O_RDONLY);
> +	fd = open(todo_file, O_RDONLY);
>  	if (fd < 0)
> -		return error_errno(_("Could not open %s"),
> -				   git_path_todo_file());
> +		return error_errno(_("Could not open %s"), todo_file);

... So that's why it is s/git_path_todo_file()/todo_file/ replacement,
and not simply...

>  	if (strbuf_read(&buf, fd, 0) < 0) {
>  		close(fd);
>  		strbuf_release(&buf);
> -		return error(_("Could not read %s."), git_path_todo_file());
> +		return error(_("Could not read %s."), todo_file);
>  	}
>  	close(fd);
>  
>  	res = parse_insn_buffer(buf.buf, todo_list, opts);
>  	strbuf_release(&buf);
>  	if (res)
> -		return error(_("Unusable instruction sheet: %s"),
> -			git_path_todo_file());
> +		return error(_("Unusable instruction sheet: %s"), todo_file);
>  	return 0;
>  }
>  
> @@ -1064,7 +1068,7 @@ static int sequencer_continue(struct replay_opts *opts)
>  {
>  	struct commit_list *todo_list = NULL;
>  
> -	if (!file_exists(git_path_todo_file()))
> +	if (!file_exists(get_todo_path(opts)))

...the s/git_path_todo_file()/git_todo_path(opts)/, isn't it?

>  		return continue_single_pick();
>  	if (read_populate_opts(opts) ||
>  			read_populate_todo(&todo_list, opts))
> 

Looks good; though I have not checked if all calling sites were converted.

Good work,
-- 
Jakub Narębski

Re: [PATCH 07/22] sequencer: future-proof read_populate_todo()

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2595 bytes --]

Hi Kuba,

On Tue, 30 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> 
> > Over the next commits, we will work on improving the sequencer to the
> > point where it can process the edit script of an interactive rebase. To
> > that end, we will need to teach the sequencer to read interactive
> > rebase's todo file. In preparation, we consolidate all places where
> > that todo file is needed to call a function that we will later extend.
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 18 +++++++++++-------
> >  1 file changed, 11 insertions(+), 7 deletions(-)
> > 
> > diff --git a/sequencer.c b/sequencer.c
> > index 8d79091..982b6e9 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -32,6 +32,11 @@ static const char *get_dir(const struct replay_opts *opts)
> >  	return git_path_seq_dir();
> >  }
> >  
> > +static const char *get_todo_path(const struct replay_opts *opts)
> > +{
> > +	return git_path_todo_file();
> > +}
> 
> I guess that in the future commit the return value of get_todo_path()
> would change depending on what sequencer is used for, cherry-pick or
> interactive rebase, that is, contents of replay_opts...

Right.

> >  static int is_rfc2822_line(const char *buf, int len)
> >  {
> >  	int i;
> > @@ -772,25 +777,24 @@ static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
> >  static int read_populate_todo(struct commit_list **todo_list,
> >  			struct replay_opts *opts)
> >  {
> > +	const char *todo_file = get_todo_path(opts);
> 
> ...and that's why you have added this temporary variable here, to
> not repeat get_todo_path(opts) calculations...

... and to repeat only 9 characters instead of 19...

> > -	fd = open(git_path_todo_file(), O_RDONLY);
> > +	fd = open(todo_file, O_RDONLY);
> >  	if (fd < 0)
> > -		return error_errno(_("Could not open %s"),
> > -				   git_path_todo_file());
> > +		return error_errno(_("Could not open %s"), todo_file);
> 
> ... So that's why it is s/git_path_todo_file()/todo_file/ replacement,
> and not simply...
> 
> > @@ -1064,7 +1068,7 @@ static int sequencer_continue(struct replay_opts *opts)
> >  {
> >  	struct commit_list *todo_list = NULL;
> >  
> > -	if (!file_exists(git_path_todo_file()))
> > +	if (!file_exists(get_todo_path(opts)))
> 
> ...the s/git_path_todo_file()/git_todo_path(opts)/, isn't it?

Correct.

> Looks good; though I have not checked if all calling sites were converted.

Thanks for the review!
Johannes

Re: [PATCH 14/22] sequencer: prepare for rebase -i's commit functionality

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> > +static char **read_author_script(void)
>> > +{
>> > +	struct strbuf script = STRBUF_INIT;
>> > +	int i, count = 0;
>> > +	char *p, *p2, **env;
>> > +	size_t env_size;
>> > +
>> > +	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
>> > +		return NULL;
>> > +
>> > +	for (p = script.buf; *p; p++)
>> > +		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
>> > +			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
>> > +		else if (*p == '\'')
>> > +			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
>> > +		else if (*p == '\n') {
>> > +			*p = '\0';
>> > +			count++;
>> > +		}
>> 
>> Hmph.  What is this loop doing?  Is it decoding a sq-quoted buffer
>> or something?  Don't we have a helper function to do that?
>
> Well, it is not just decoding an sq-quoted buffer, but several lines with
> definitions we sq-quoted ourselves, individually.
>
> The quote.[ch] code currently has no code to dequote lines individually.

There is a function with exactly the same name in builtin/am.c and I
assume that it is reading from a file with the same format, which
uses a helper to read one variable line at a time.  Hopefully that
can be refactored so that more parsing is shared between the two
users.

Two functions with the same name reading from the same format, even
when they expect to produce the same result in different internal
format, without even being aware of each other is a bad enough
"regression" in maintainability of the code.  One of them not even
using sq_dequote() helper and reinventing is even worse.

Re: [PATCH 06/22] sequencer: release memory that was allocated when reading options

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3078 bytes --]

Hi Kuba,

On Tue, 30 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> 
> > The sequencer reads options from disk and stores them in its struct
> > for use during sequencer's operations.
> > 
> > With this patch, the memory is released afterwards, plugging a
> > memory leak.
> > 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 13 ++++++++++---
> >  1 file changed, 10 insertions(+), 3 deletions(-)
> > 
> > diff --git a/sequencer.c b/sequencer.c
> > index b5be0f9..8d79091 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -131,6 +131,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
> >  		free(opts->owned[i]);
> >  	free(opts->owned);
> >  
> > +	free(opts->xopts);
> > +
> 
> This looks like independent change, not related to using the
> sequencer_entrust() to store options read from disk in replay_opts
> struct to be able to free memory afterwards.
> 
> I guess you wanted to avoid one line changes...

Actually, it is not an independent change, but it free()s memory that has
been allocated while reading the options, as the commit message says ;-)

> > @@ -811,13 +813,18 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
> 
> Sidenote: this patch would be easier to read if lines were reordered
> as below, but I don't think any slider heuristics could help achieve
> that automatically.  Also, the patch might be invalid...
> 
> >  		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
> >  	else if (!strcmp(key, "options.mainline"))
> >  		opts->mainline = git_config_int(key, value);
> > -	else if (!strcmp(key, "options.strategy"))
> > +	else if (!strcmp(key, "options.strategy")) {
> >  		git_config_string(&opts->strategy, key, value);
> > +		sequencer_entrust(opts, (char *) opts->strategy);
> 
> I wonder if the ability to free strings dup-ed by git_config_string()
> be something that is part of replay_opts, or rather remove_sequencer_state(),
> that is a list of
> 
> 	free(opts->strategy);
> 	free(opts->gpg_sign);

That is not necessarily possible because the way sequencer works, the
options may have not actually be read from the file, but may be populated
by the caller (in which case we do not necessarily want to require
strdup()ing the strings just so that the sequencer can clean stuff up
afterwards).

> Though... free(NULL) is nop as per standard, but can we rely on it?

We can, and we do.

> The *_entrust() mechanism is more generic, but do we use this general-ness?
> Well, it could be xstrdup or git_config_string doing entrust'ing...

Right, but that is exactly what I wanted to avoid, because it is rather
inelegant to strdup() strings just so that we do not have to record what
to free() and what not to free().

BTW I have no objection at all to generalize this sequencer_entrust()
mechanism further (read: to other, similar use cases), should it withstand
the test of time.

Ciao,
Johannes

Re: [PATCH 14/22] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

Hi Junio,

On Tue, 30 Aug 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> > +static char **read_author_script(void)
> >> > +{
> >> > +	struct strbuf script = STRBUF_INIT;
> >> > +	int i, count = 0;
> >> > +	char *p, *p2, **env;
> >> > +	size_t env_size;
> >> > +
> >> > +	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
> >> > +		return NULL;
> >> > +
> >> > +	for (p = script.buf; *p; p++)
> >> > +		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
> >> > +			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
> >> > +		else if (*p == '\'')
> >> > +			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
> >> > +		else if (*p == '\n') {
> >> > +			*p = '\0';
> >> > +			count++;
> >> > +		}
> >> 
> >> Hmph.  What is this loop doing?  Is it decoding a sq-quoted buffer
> >> or something?  Don't we have a helper function to do that?
> >
> > Well, it is not just decoding an sq-quoted buffer, but several lines with
> > definitions we sq-quoted ourselves, individually.
> >
> > The quote.[ch] code currently has no code to dequote lines individually.
> 
> There is a function with exactly the same name in builtin/am.c and I
> assume that it is reading from a file with the same format, which
> uses a helper to read one variable line at a time.  Hopefully that
> can be refactored so that more parsing is shared between the two
> users.
> 
> Two functions with the same name reading from the same format, even
> when they expect to produce the same result in different internal
> format, without even being aware of each other is a bad enough
> "regression" in maintainability of the code.  One of them not even
> using sq_dequote() helper and reinventing is even worse.

First of all, builtin/am's read_author_script() really, really, really
only wants to read stuff into the am_state struct.

That alone is already so incompatible that I do not think it can be
repaired.

Further, builtin/am's read_author_script() reads *only* the
GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL and GIT_AUTHOR_DATE lines (opening the
file three times for the task). And then it *requires* the corresponding
values to be sq-quoted.

I do not have a use case for this myself, but rebase -i explicitly eval's
the author script, so it is conceivable that some enterprisey user makes
use of this feature to set other environment variables. The thing is that
rebase -i cannot necessarily expect all of those files in its state
directory to be under tight control -- in stark contrast to git-am.

I could imagine that this could be fixed by abstracting the functionality
more, and use your favored sq_dequote() (which may actually fail in case
of an enterprisey usage as outlined above), and adapting builtin/am's
read_author_script() to make use of that refactored approach.

This is a huge task, make no mistake, in particular because we need to
ensure compatibility with non-standard usage. So I do not think I can
tackle that any time soon. Certainly not as part of an iteration of this
here patch series.

Ciao,
Dscho

Re: [PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

> In my personal opinion 'set_me_free_after_use' is not the best name,
> but I unfortunately do not have a better proposal.  Maybe 'entrust_ptr',
> or 'entrusted_data' / 'entrusted_ptr' / 'entrusted'?

Is this to accumulate to-be-freed pointers?

I think we often call a local variable that points at a piece of
memory to be freed "to_free", and that is an appropriate name for
what this function is trying to do.

It is a bit surprising that the careless memory management in this
codepath leaks only the dumb pieces of memory (as opposed to
pointers to structures like string list that needs _clear()
functions, in which case we cannot get away with list of
to-be-freed).  I guess we were somewhat lucky ;-)

Re: [PATCH 06/22] sequencer: release memory that was allocated when reading options

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> @@ -811,13 +813,18 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
>  		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
>  	else if (!strcmp(key, "options.mainline"))
>  		opts->mainline = git_config_int(key, value);
> -	else if (!strcmp(key, "options.strategy"))
> +	else if (!strcmp(key, "options.strategy")) {
>  		git_config_string(&opts->strategy, key, value);
> -	else if (!strcmp(key, "options.gpg-sign"))
> +		sequencer_entrust(opts, (char *) opts->strategy);
> +	}
> +	else if (!strcmp(key, "options.gpg-sign")) {
>  		git_config_string(&opts->gpg_sign, key, value);
> +		sequencer_entrust(opts, (char *) opts->gpg_sign);
> +	}
>  	else if (!strcmp(key, "options.strategy-option")) {
>  		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
> -		opts->xopts[opts->xopts_nr++] = xstrdup(value);
> +		opts->xopts[opts->xopts_nr++] =
> +			sequencer_entrust(opts, xstrdup(value));
>  	} else
>  		return error(_("Invalid key: %s"), key);

Hmm.

I would have expected a call to sequencer_opts_clear(&opts) once the
machinery is done with the options structure, and among these places
where an old value in opts->field is overwritten by a new one would
get

	free(opt->field); opt->field = ... new value ...;

Perhaps there was a good reason to do it this way (one valid reason
may be that there is _no_ good place to declare that opts is now
done and it is safe to call sequencer_opts_clear() on it), but this
looks backwards from the way things are usually done.

Re: [PATCH 06/22] sequencer: release memory that was allocated when reading options

[Johannes Sixt]

Am 30.08.2016 um 19:52 schrieb Johannes Schindelin:
> Right, but that is exactly what I wanted to avoid, because it is rather
> inelegant to strdup() strings just so that we do not have to record what
> to free() and what not to free().

Please, excuse, but when I have to choose what is more "elegant":

  1. strdup() sometimes so that I can later free() always
  2. use sequencer_entrust()

I would choose 1. at all times.

Particularly in this case: parsing options does not sound like a major 
drain of resources, neither CPU- nor memory-wise.

-- Hannes

Re: [PATCH 06/22] sequencer: release memory that was allocated when reading options

[Jakub Narębski]

W dniu 30.08.2016 o 19:52, Johannes Schindelin pisze:
> Hi Kuba,
> 
> On Tue, 30 Aug 2016, Jakub Narębski wrote:
> 
>> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
>>
>>> The sequencer reads options from disk and stores them in its struct
>>> for use during sequencer's operations.
>>>
>>> With this patch, the memory is released afterwards, plugging a
>>> memory leak.
>>>
>>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>>> ---
>>>  sequencer.c | 13 ++++++++++---
>>>  1 file changed, 10 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/sequencer.c b/sequencer.c
>>> index b5be0f9..8d79091 100644
>>> --- a/sequencer.c
>>> +++ b/sequencer.c
>>> @@ -131,6 +131,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
>>>  		free(opts->owned[i]);
>>>  	free(opts->owned);
>>>  
>>> +	free(opts->xopts);
>>> +
>>
>> This looks like independent change, not related to using the
>> sequencer_entrust() to store options read from disk in replay_opts
>> struct to be able to free memory afterwards.
>>
>> I guess you wanted to avoid one line changes...
> 
> Actually, it is not an independent change, but it free()s memory that has
> been allocated while reading the options, as the commit message says ;-)
> 
>>> @@ -811,13 +813,18 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
>>
>> Sidenote: this patch would be easier to read if lines were reordered
>> as below, but I don't think any slider heuristics could help achieve
>> that automatically.  Also, the patch might be invalid...
>>
>>>  		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
>>>  	else if (!strcmp(key, "options.mainline"))
>>>  		opts->mainline = git_config_int(key, value);
>>> -	else if (!strcmp(key, "options.strategy"))
>>> +	else if (!strcmp(key, "options.strategy")) {
>>>  		git_config_string(&opts->strategy, key, value);
>>> +		sequencer_entrust(opts, (char *) opts->strategy);
>>
>> I wonder if the ability to free strings dup-ed by git_config_string()
>> be something that is part of replay_opts, or rather remove_sequencer_state(),
>> that is a list of
>>
>> 	free(opts->strategy);
>> 	free(opts->gpg_sign);
> 
> That is not necessarily possible because the way sequencer works, the
> options may have not actually be read from the file, but may be populated
> by the caller (in which case we do not necessarily want to require
> strdup()ing the strings just so that the sequencer can clean stuff up
> afterwards).

I guess from cursory browsing through the Git code that _currently_
they are only read from the config file, where git_config_string()
strdup's them, isn't it?  And we want to prepare for the future, where
the caller would prepare replay_opts, and the caller would be responsible
for freeing data if necessary?

Would there be any sane situation where some of data should be owned
by caller (and freed by caller), and some of data should be owned by
sequencer library API (and freed in remove_sequencer_state())?  If
not, perhaps *_entrust() mechanism is overthinking it, and we simply
need 'is_strdup' boolean flag or something like that...

>> The *_entrust() mechanism is more generic, but do we use this general-ness?
>> Well, it could be xstrdup or git_config_string doing entrust'ing...
> 
> Right, but that is exactly what I wanted to avoid, because it is rather
> inelegant to strdup() strings just so that we do not have to record what
> to free() and what not to free().

Maybe inelegant, but it might be easier than inventing and implementing
*_entrust() mechanism, like Hannes wrote.

> 
> BTW I have no objection at all to generalize this sequencer_entrust()
> mechanism further (read: to other, similar use cases), should it withstand
> the test of time.

Yeah, that's my take on it too.

-- 
Jakub Narębski

Re: [PATCH 06/22] sequencer: release memory that was allocated when reading options

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Am 30.08.2016 um 19:52 schrieb Johannes Schindelin:
>> Right, but that is exactly what I wanted to avoid, because it is rather
>> inelegant to strdup() strings just so that we do not have to record what
>> to free() and what not to free().
>
> Please, excuse, but when I have to choose what is more "elegant":
>
>  1. strdup() sometimes so that I can later free() always
>  2. use sequencer_entrust()
>
> I would choose 1. at all times.

Let's not bring elegance or aesthetics in.  It is like asking if
garbage collected systems are elegant.  I do not think it is a valid
question.  GC is a good pragmatic compromise after (1) declaring
that keeping track of allocation is too cumbersome and programmers'
time is better spent elsewhere, and (2) making sure hiccups caused
by GC can be minimized to keep the latency down to acceptable
levels.  There are good compromises and bad ones.

Does the proposed entrust() thing qualify as a good pragmatic
compromise like GC does?

> Particularly in this case: parsing options does not sound like a
> major drain of resources, neither CPU- nor memory-wise.

If entrust() does not have any CPU- or memory-cost, then comparing
it with having to strdup() sometimes might be a useful comparison.

But the entrust() thing has the allocation cost in order to track of
what needs to be done at runtime, and just like strdup() needs to be
done on things that are being borrowed, entrust() needs to be
avoided on things that are being borrowed (and the caller needs to
be sure not to entrust() the same piece of memory twice), so it does
not reduce the cost on programmers' and readers' mental burden--the
programmer always has to know which piece of memory is borrowed and
which are owned by the subsystem.

So...

Re: [PATCH 14/22] sequencer: prepare for rebase -i's commit functionality

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Two functions with the same name reading from the same format, even
> when they expect to produce the same result in different internal
> format, without even being aware of each other is a bad enough
> "regression" in maintainability of the code.  One of them not even
> using sq_dequote() helper and reinventing is even worse.

So, here is what I did as a lunch-break-hack.  The "same result in
different internal format" calls for a fairly light-weight mechanism
to convey the necessary information that can be shared by callers
with different needs, and I chose string-list for that.

Totally untested, but parse_key_value_squoted() would be a good
foundation to build on.  The caller in "am" deliberately wants to be
strict (e.g. it wants to error out when the user mucked with the
file, so it insists on the three variables to appear in a known
order and rejects input that violates its assumption), but the
function does not mind if other callers want to be more lenient.

-- >8 --
From: Junio C Hamano <gitster@pobox.com>
Date: Tue, 30 Aug 2016 12:36:42 -0700
Subject: [PATCH] am: refactor read_author_script()

By splitting the part that reads from a file and the part that
parses the variable definitions from the contents, make the latter
can be more reusable in the future.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 builtin/am.c | 103 ++++++++++++++++++++++++++---------------------------------
 1 file changed, 45 insertions(+), 58 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 739b34d..b36d1f0 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -28,6 +28,7 @@
 #include "rerere.h"
 #include "prompt.h"
 #include "mailinfo.h"
+#include "string-list.h"
 
 /**
  * Returns 1 if the file is empty or does not exist, 0 otherwise.
@@ -258,38 +259,29 @@ static int read_state_file(struct strbuf *sb, const struct am_state *state,
 }
 
 /**
- * Reads a KEY=VALUE shell variable assignment from `fp`, returning the VALUE
- * as a newly-allocated string. VALUE must be a quoted string, and the KEY must
- * match `key`. Returns NULL on failure.
- *
- * This is used by read_author_script() to read the GIT_AUTHOR_* variables from
- * the author-script.
+ * Take a series of KEY='VALUE' lines where VALUE part is
+ * sq-quoted, and append <KEY, VALUE> at the end of the string list
  */
-static char *read_shell_var(FILE *fp, const char *key)
+static int parse_key_value_squoted(char *buf, struct string_list *list)
 {
-	struct strbuf sb = STRBUF_INIT;
-	const char *str;
-
-	if (strbuf_getline_lf(&sb, fp))
-		goto fail;
-
-	if (!skip_prefix(sb.buf, key, &str))
-		goto fail;
-
-	if (!skip_prefix(str, "=", &str))
-		goto fail;
-
-	strbuf_remove(&sb, 0, str - sb.buf);
-
-	str = sq_dequote(sb.buf);
-	if (!str)
-		goto fail;
-
-	return strbuf_detach(&sb, NULL);
-
-fail:
-	strbuf_release(&sb);
-	return NULL;
+	while (*buf) {
+		struct string_list_item *item;
+		char *np;
+		char *cp = strchr(buf, '=');
+		if (!cp)
+			return -1;
+		np = strchrnul(cp, '\n');
+		*cp++ = '\0';
+		item = string_list_append(list, buf);
+
+		buf = np + (*np == '\n');
+		*np = '\0';
+		cp = sq_dequote(cp);
+		if (!cp)
+			return -1;
+		item->util = xstrdup(cp);
+	}
+	return 0;
 }
 
 /**
@@ -311,44 +303,39 @@ static char *read_shell_var(FILE *fp, const char *key)
 static int read_author_script(struct am_state *state)
 {
 	const char *filename = am_path(state, "author-script");
-	FILE *fp;
+	struct strbuf buf = STRBUF_INIT;
+	struct string_list kv = STRING_LIST_INIT_DUP;
+	int retval = -1; /* assume failure */
+	int fd;
 
 	assert(!state->author_name);
 	assert(!state->author_email);
 	assert(!state->author_date);
 
-	fp = fopen(filename, "r");
-	if (!fp) {
+	fd = open(filename, O_RDONLY);
+	if (fd < 0) {
 		if (errno == ENOENT)
 			return 0;
 		die_errno(_("could not open '%s' for reading"), filename);
 	}
+	strbuf_read(&buf, fd, 0);
+	close(fd);
+	if (parse_key_value_squoted(buf.buf, &kv))
+		goto finish;
 
-	state->author_name = read_shell_var(fp, "GIT_AUTHOR_NAME");
-	if (!state->author_name) {
-		fclose(fp);
-		return -1;
-	}
-
-	state->author_email = read_shell_var(fp, "GIT_AUTHOR_EMAIL");
-	if (!state->author_email) {
-		fclose(fp);
-		return -1;
-	}
-
-	state->author_date = read_shell_var(fp, "GIT_AUTHOR_DATE");
-	if (!state->author_date) {
-		fclose(fp);
-		return -1;
-	}
-
-	if (fgetc(fp) != EOF) {
-		fclose(fp);
-		return -1;
-	}
-
-	fclose(fp);
-	return 0;
+	if (kv.nr != 3 ||
+	    strcmp(kv.items[0].string, "GIT_AUTHOR_NAME") ||
+	    strcmp(kv.items[1].string, "GIT_AUTHOR_EMAIL") ||
+	    strcmp(kv.items[2].string, "GIT_AUTHOR_DATE"))
+		goto finish;
+	state->author_name = kv.items[0].util;
+	state->author_email = kv.items[1].util;
+	state->author_date = kv.items[2].util;
+	retval = 0;
+finish:
+	string_list_clear(&kv, !!retval);
+	strbuf_release(&buf);
+	return retval;
 }
 
 /**
-- 
2.10.0-rc2-246-g4fd2c60

Re: [PATCH 06/22] sequencer: release memory that was allocated when reading options

[Johannes Schindelin]

Hi Junio,

On Tue, 30 Aug 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > @@ -811,13 +813,18 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
> >  		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
> >  	else if (!strcmp(key, "options.mainline"))
> >  		opts->mainline = git_config_int(key, value);
> > -	else if (!strcmp(key, "options.strategy"))
> > +	else if (!strcmp(key, "options.strategy")) {
> >  		git_config_string(&opts->strategy, key, value);
> > -	else if (!strcmp(key, "options.gpg-sign"))
> > +		sequencer_entrust(opts, (char *) opts->strategy);
> > +	}
> > +	else if (!strcmp(key, "options.gpg-sign")) {
> >  		git_config_string(&opts->gpg_sign, key, value);
> > +		sequencer_entrust(opts, (char *) opts->gpg_sign);
> > +	}
> >  	else if (!strcmp(key, "options.strategy-option")) {
> >  		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
> > -		opts->xopts[opts->xopts_nr++] = xstrdup(value);
> > +		opts->xopts[opts->xopts_nr++] =
> > +			sequencer_entrust(opts, xstrdup(value));
> >  	} else
> >  		return error(_("Invalid key: %s"), key);
> 
> Hmm.
> 
> I would have expected a call to sequencer_opts_clear(&opts) once the
> machinery is done with the options structure,

It is part of sequencer_remove_state().

> and among these places where an old value in opts->field is overwritten
> by a new one would get
> 
> 	free(opt->field); opt->field = ... new value ...;

That is *exactly* what we *cannot* do.

You see, the replay_opts struct is *not necessarily* populated by
read_populate_opts(). So we really cannot tell whether we are free to
free(opt->field) or whether some other code still wants to reference that
memory in the end (or for that matter, whether it was malloc()ed).

That is the *precise* reason for the existence of sequencer_entrust().

Ciao,
Dscho

Re: [PATCH 14/22] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

Hi Junio,

On Tue, 30 Aug 2016, Junio C Hamano wrote:

> From: Junio C Hamano <gitster@pobox.com>
> Date: Tue, 30 Aug 2016 12:36:42 -0700
> Subject: [PATCH] am: refactor read_author_script()
> 
> [...]

Thank you so much for that! I will use that as a starting point to
refactor the two read_author_script() functions into one.

Just don't look surprised when you see this patch as part of my patch
series, Signed-off-by me...

Ciao,
Dscho

Re: [PATCH 05/22] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

Hi Junio,

On Tue, 30 Aug 2016, Junio C Hamano wrote:

> Jakub Narębski <jnareb@gmail.com> writes:
> 
> > In my personal opinion 'set_me_free_after_use' is not the best name,
> > but I unfortunately do not have a better proposal.  Maybe 'entrust_ptr',
> > or 'entrusted_data' / 'entrusted_ptr' / 'entrusted'?
> 
> Is this to accumulate to-be-freed pointers?

Yes.

> I think we often call a local variable that points at a piece of
> memory to be freed "to_free", and that is an appropriate name for
> what this function is trying to do.

I changed it to that.

> It is a bit surprising that the careless memory management in this
> codepath leaks only the dumb pieces of memory (as opposed to
> pointers to structures like string list that needs _clear()
> functions, in which case we cannot get away with list of
> to-be-freed).  I guess we were somewhat lucky ;-)

Yeah, the carelessness is more like a convenience where the config
machinery is reused to parse the values.

Ciao,
Dscho

Re: [PATCH 08/22] sequencer: remove overzealous assumption

[Jakub Narębski]

W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> The sequencer was introduced to make the cherry-pick and revert
> functionality available as library function, with the original idea
> being to extend the sequencer to also implement the rebase -i
> functionality.
> 
> The test to ensure that all of the commands in the script are identical
> to the overall operation does not mesh well with that.

Actually the question is what does the test that got removed in this
commit actually check.  Is it high-level sanity check that todo list
for git-cherry-pick contains only 'pick', and for git-revert contains
only 'revert'?  Or does it check that at the low level sequencer
fails when instruction sheet includes only identical operations?

Only if it is the latter (we are testing too low level details of
how sequencer code works, tying too tightly test with implementation)
the test should be removed.  I see that earlier test check that
sequencer handles correctly invalid instructions in todo.

> Therefore let's just get rid of the test that wants to verify that this
> limitation is still in place, in preparation for the upcoming work to
> teach the sequencer to do rebase -i's work.

Is it "upcoming work" as in one of the patches in this series?
If so, which patch?

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  t/t3510-cherry-pick-sequence.sh | 11 -----------
>  1 file changed, 11 deletions(-)
> 
> diff --git a/t/t3510-cherry-pick-sequence.sh b/t/t3510-cherry-pick-sequence.sh
> index 7b7a89d..6465edf 100755
> --- a/t/t3510-cherry-pick-sequence.sh
> +++ b/t/t3510-cherry-pick-sequence.sh
> @@ -459,17 +459,6 @@ test_expect_success 'malformed instruction sheet 1' '
>  	test_expect_code 128 git cherry-pick --continue
>  '
>  
> -test_expect_success 'malformed instruction sheet 2' '

Hmmm... the description is somewhat lacking (especially compared to
the rest of test), anyway.

BTW. we should probably rename 'malformed instruction sheet 2'
to 'malformed instruction sheet' if there are no further such
tests after this removal, isn't it?

> -	pristine_detach initial &&
> -	test_expect_code 1 git cherry-pick base..anotherpick &&
> -	echo "resolved" >foo &&
> -	git add foo &&
> -	git commit &&
> -	sed "s/pick/revert/" .git/sequencer/todo >new_sheet &&
> -	cp new_sheet .git/sequencer/todo &&
> -	test_expect_code 128 git cherry-pick --continue
> -'
> -
>  test_expect_success 'empty commit set' '
>  	pristine_detach initial &&
>  	test_expect_code 128 git cherry-pick base..base
> 

Re: [PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Jakub Narębski]

W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:

> When we came up with the "sequencer" idea, we really wanted to have
> kind of a plumbing equivalent of the interactive rebase. Hence the
> choice of words: the "todo" script, a "pick", etc.
> 
> However, when it came time to implement the entire shebang, somehow this
> idea got lost and the sequencer was used as working horse for
> cherry-pick and revert instead. So as not to interfere with the
> interactive rebase, it even uses a separate directory to store its
> state.
> 
> Furthermore, it also is stupidly strict about the "todo" script it
> accepts: while it parses commands in a way that was *designed* to be
> similar to the interactive rebase, it then goes on to *error out* if the
> commands disagree with the overall action (cherry-pick or revert).

Does this mean that after the change you would be able to continue
"git revert" with "git cherry-pick --continue", and vice versa?  Or that
it would be possible for git-cherry-pick to do reverts (e.g. with ^<rev>)?

That's what we need to decide before becoming more lenient.
 
BTW. perhaps we would be able to continue with 'git continue', regardless
of what we have started with, I wonder...

>
> Finally, the sequencer code chose to deviate from the interactive rebase
> code insofar that it *reformats* the "todo" script instead of just
> writing the part of the parsed script that were not yet processed. This
> is not only unnecessary churn, but might well lose information that is
> valuable to the user (i.e. comments after the commands).

That's a very good change.

>
> Let's just bite the bullet and rewrite the entire parser; the code now
> becomes not only more elegant: it allows us to go on and teach the
> sequencer how to parse *true* "todo" scripts as used by the interactive
> rebase itself. In a way, the sequencer is about to grow up to do its
> older brother's job. Better.

Sidenote: this is not your fault, but Git doesn't do a good job on
changes which are mostly rewrites, trying to match stray '}' and the
like in generated diff.  I wonder if existing diff heuristic options
could help here.

> 
> While at it, do not stop at the first problem, but list *all* of the
> problems. This helps the user by allowing to address all issues in
> one go rather than going back and forth until the todo list is valid.

That is also a good change, though I wonder how often users need
to worry about this outside interactive rebase case.  If it is
preparation for rebase -i, where instruction list is written by
prone to errors human, it would be nice to have this information
in the commit message.

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 241 +++++++++++++++++++++++++++++++++---------------------------
>  1 file changed, 134 insertions(+), 107 deletions(-)

Note: I have moved some lines of diff so that the change is more
readable to humans (but it results often in --++-++ chunk).

> 
> diff --git a/sequencer.c b/sequencer.c
> index 982b6e9..cbdce6d 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -473,7 +473,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
>  		return 1;
>  }
>  
> -static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
> +enum todo_command {
> +	TODO_PICK,
> +	TODO_REVERT
> +};

Do we have a naming convention for enums elements?  Or are we explicitly
making enums and #defines interchangeable?  I wonder...

...uh, I see we don't have naming convention, but all caps snake-case
names dominate:

  $ git grep -A2 'enum .* {'
  [...]
  diff.h:enum color_diff {
  diff.h- DIFF_RESET = 0,
  diff.h- DIFF_CONTEXT = 1,
  --
  dir.c:enum path_treatment {
  dir.c-  path_none = 0,
  dir.c-  path_recurse,
  --

Shouldn't we say 'TODO_PICK = 0' explicitly, though?

> +
> +static const char *todo_command_strings[] = {
> +	"pick",
> +	"revert"
> +};

It's a bit pity that we cannot use designated inits, and hanging comma,
(from ISO C99 standard).  That is:

  +static const char *todo_command_strings[] = {
  +	[TODO_PICK]   = "pick",
  +	[TODO_REVERT] = "revert",
  +};


> +
> +static const char *command_to_string(const enum todo_command command)
> +{
> +	if (command < ARRAY_SIZE(todo_command_strings))
> +		return todo_command_strings[command];
> +	die("Unknown command: %d", command);
> +}
> +
> +
> +static int do_pick_commit(enum todo_command command, struct commit *commit,
> +		struct replay_opts *opts)
>  {
>  	unsigned char head[20];
>  	struct commit *base, *next, *parent;
> @@ -535,7 +554,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
>  		/* TRANSLATORS: The first %s will be "revert" or
>  		   "cherry-pick", the second %s a SHA1 */
>  		return error(_("%s: cannot parse parent commit %s"),

I wonder if we should not change also the error message: it is no
longer about command, but about operation in todo list (from what
I understand).  Though admittedly current message works for both...

> -			action_name(opts), oid_to_hex(&parent->object.oid));
> +			command_to_string(command),
> +			oid_to_hex(&parent->object.oid));
>  
>  	if (get_message(commit, &msg) != 0)
>  		return error(_("Cannot get commit message for %s"),
> @@ -548,7 +568,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)

From here on changes are about

  s/opts->action == REPLAY_\(PICK\|REVERT\)/command == TODO_\1/

Do we still need opts->action, or it is just needed less,
and it is 'todo' instruction that decides about command
(as it should)?

>  	 * reverse of it if we are revert.
>  	 */
>  
> -	if (opts->action == REPLAY_REVERT) {
> +	if (command == TODO_REVERT) {
>  		base = commit;
>  		base_label = msg.label;
>  		next = parent;
> @@ -589,7 +609,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
>  		}
>  	}
>  
> -	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
> +	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
>  		res = do_recursive_merge(base, next, base_label, next_label,
>  					 head, &msgbuf, opts);
>  		if (res < 0)
> @@ -615,17 +635,17 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
>  	 * However, if the merge did not even start, then we don't want to
>  	 * write it at all.
>  	 */
> -	if (opts->action == REPLAY_PICK && !opts->no_commit && (res == 0 || res == 1) &&
> +	if (command == TODO_PICK && !opts->no_commit && (res == 0 || res == 1) &&
>  	    update_ref(NULL, "CHERRY_PICK_HEAD", commit->object.oid.hash, NULL,
>  		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
>  		res = -1;
> -	if (opts->action == REPLAY_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
> +	if (command == TODO_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
>  	    update_ref(NULL, "REVERT_HEAD", commit->object.oid.hash, NULL,
>  		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
>  		res = -1;
>  
>  	if (res) {
> -		error(opts->action == REPLAY_REVERT
> +		error(command == TODO_REVERT
>  		      ? _("could not revert %s... %s")
>  		      : _("could not apply %s... %s"),
>  		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),

And here those changes end.

  s/opts->action == REPLAY_\(PICK\|REVERT\)/command == TODO_\1/

I wonder if Coccinelle / Undebt would help here; or would simple
sed or query-and-replace-regexp be enough...

> @@ -683,116 +703,107 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>  	return 0;
>  }
>  
> -static int format_todo(struct strbuf *buf, struct commit_list *todo_list,
> -		struct replay_opts *opts)
> +struct todo_item {
> +	enum todo_command command;
> +	struct commit *commit;
> +	size_t offset_in_buf;
> +};
> +
> +struct todo_list {
> +	struct strbuf buf;
> +	struct todo_item *items;
> +	int nr, alloc, current;
> +};

So there should be s/commit_list [*]{1,2}todo_list/todo_list *todo_list/
from here on?

Hmmm... commit_list is, as defined in commit.h, a linked list.
Here todo_list uses growable array implementation of list.  Which
is I guess better on current CPU architecture, with slow memory,
limited-size caches, and adjacency prefetching.

I guess using items_nr and items_alloc would be not necessary
(and a bit funny / overkill).

> +
> +#define TODO_LIST_INIT { STRBUF_INIT, NULL, 0, 0, 0 }

Same as with other patches in this series, it would be enough to

  +#define TODO_LIST_INIT { STRBUF_INIT }

You are consistent.

> +
> +static void todo_list_release(struct todo_list *todo_list)
>  {

Grh... stray '{' matched...

> -	struct commit_list *cur = NULL;
> -	const char *sha1_abbrev = NULL;
> -	const char *action_str = opts->action == REPLAY_REVERT ? "revert" : "pick";
> -	const char *subject;
> -	int subject_len;
> +	strbuf_release(&todo_list->buf);
> +	free(todo_list->items);
> +	todo_list->items = NULL;
> +	todo_list->nr = todo_list->alloc = 0;
> +}
>  
> -	for (cur = todo_list; cur; cur = cur->next) {
> -		const char *commit_buffer = get_commit_buffer(cur->item, NULL);
> -		sha1_abbrev = find_unique_abbrev(cur->item->object.oid.hash, DEFAULT_ABBREV);
> -		subject_len = find_commit_subject(commit_buffer, &subject);
> -		strbuf_addf(buf, "%s %s %.*s\n", action_str, sha1_abbrev,
> -			subject_len, subject);
> -		unuse_commit_buffer(cur->item, commit_buffer);
> -	}
> -	return 0;
> +struct todo_item *append_todo(struct todo_list *todo_list)

Errr... I don't quite understand the name of this function.
What are you appending here to the todo_list?

Compare string_list_append() and string_list_append_nodup(),
where the second parameter is item to append.

I'm not against what this function does (grow array if needed, and
return pointer to the new todo_item that is to be filled), but
I don't quite agree with the name.  Naming is hard... :-(
[See later in reply for a proposal.]

> +{
> +	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
> +	return todo_list->items + todo_list->nr++;
>  }
>  
> -static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
> +static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)

Why the change of return type?  

I guess the previous code used only opts->action out of whole replay_opts,
and now we use item->command instead; that is why replay_opts is replaced
by todo_item.

Why now struct todo_item is first when struct replay_opts was last?
Not that I say is was a bad change...

>  {
>  	unsigned char commit_sha1[20];
> -	enum replay_action action;
>  	char *end_of_object_name;
> -	int saved, status, padding;
> -
> -	if (starts_with(bol, "pick")) {
> -		action = REPLAY_PICK;
> -		bol += strlen("pick");
> -	} else if (starts_with(bol, "revert")) {
> -		action = REPLAY_REVERT;
> -		bol += strlen("revert");
> -	} else
> -		return NULL;
> +	int i, saved, status, padding;

int i or enum?  Just kidding...

> +
> +	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
> +		if (skip_prefix(bol, todo_command_strings[i], &bol)) {

skip_prefix() is such a nice abstraction...

> +			item->command = i;
> +			break;
> +		}

Nice.  Replacing if-elsif chain with loop.  

I guess any hashmap would be serious overkill, as there are and would be
only a few actions possible.

> +	if (i >= ARRAY_SIZE(todo_command_strings))
> +		return -1;
>  
>  	/* Eat up extra spaces/ tabs before object name */
>  	padding = strspn(bol, " \t");
>  	if (!padding)
> -		return NULL;
> +		return -1;
>  	bol += padding;
>  
> -	end_of_object_name = bol + strcspn(bol, " \t\n");
> +	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");

Why is this cast needed?

And do we say '(char *) bol' or '(char *)bol'?

>  	saved = *end_of_object_name;
>  	*end_of_object_name = '\0';
>  	status = get_sha1(bol, commit_sha1);
>  	*end_of_object_name = saved;
>  
> -	/*
> -	 * Verify that the action matches up with the one in
> -	 * opts; we don't support arbitrary instructions
> -	 */
> -	if (action != opts->action) {
> -		if (action == REPLAY_REVERT)
> -		      error((opts->action == REPLAY_REVERT)
> -			    ? _("Cannot revert during another revert.")

Errr... could the above ever happen?  Namely

  action != opts->action && action == REPLAY_REVERT && opts->action == REPLAY_REVERT

Surely not.

> -			    : _("Cannot revert during a cherry-pick."));
> -		else
> -		      error((opts->action == REPLAY_REVERT)
> -			    ? _("Cannot cherry-pick during a revert.")
> -			    : _("Cannot cherry-pick during another cherry-pick."));
> -		return NULL;
> -	}

Anyway, while it is / would be a good idea to prevent starting any
sequencer-based command (cherry-pick, revert, soon rebase -i) when
other command is in progress (cherry-pick, revert, soon rebase -i).
That is, if cherry-pick / revert waits for user action, you cannot
run another cherry-pick or revert.

Which I guess the above code was not about...

> -
>  	if (status < 0)
> -		return NULL;
> +		return -1;
>  
> -	return lookup_commit_reference(commit_sha1);
> +	item->commit = lookup_commit_reference(commit_sha1);
> +	return !item->commit;
>  }
>  
> -static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
> -			struct replay_opts *opts)
> +static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
>  {
> -	struct commit_list **next = todo_list;
> -	struct commit *commit;
> +	struct todo_item *item;
>  	char *p = buf;
> -	int i;
> +	int i, res = 0;
>  
>  	for (i = 1; *p; i++) {
>  		char *eol = strchrnul(p, '\n');
> -		commit = parse_insn_line(p, eol, opts);
> -		if (!commit)
> -			return error(_("Could not parse line %d."), i);
> -		next = commit_list_append(commit, next);
> +
> +		item = append_todo(todo_list);

A better name, in my personal option, would be

  +		item = todo_list_next(todo_list);

Or todo_next(todo_list).

> +		item->offset_in_buf = p - todo_list->buf.buf;
> +		if (parse_insn_line(item, p, eol)) {
> +			error("Invalid line: %.*s", (int)(eol - p), p);

This error message should, I think, be also translatable:

  +			error(_("Invalid line: %.*s"), (int)(eol - p), p);

> +			res |= error(_("Could not parse line %d."), i);

Wouldn't it make more sense to reverse order of errors, that is
first tell which line, and then show it?  

BTW. would be we able to show where exactly there was problem parsing,
that is at which character in line?  Or is it something for the future?

> +			item->command = -1;
> +		}
>  		p = *eol ? eol + 1 : eol;
>  	}
> -	if (!*todo_list)
> +	if (!todo_list->nr)
>  		return error(_("No commits parsed."));
> -	return 0;
> +	return res;

Ah, so 'res' is "was there an error" in any of lines.  Nice.

>  }
>  
> -static int read_populate_todo(struct commit_list **todo_list,
> +static int read_populate_todo(struct todo_list *todo_list,
>  			struct replay_opts *opts)
>  {
>  	const char *todo_file = get_todo_path(opts);

If I understand it correctly, replay_opts is used only to find out
correct todo_file, isn't it?

> -	struct strbuf buf = STRBUF_INIT;
>  	int fd, res;
>  
> +	strbuf_reset(&todo_list->buf);
>  	fd = open(todo_file, O_RDONLY);
>  	if (fd < 0)
>  		return error_errno(_("Could not open %s"), todo_file);
> -	if (strbuf_read(&buf, fd, 0) < 0) {
> +	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
>  		close(fd);
> -		strbuf_release(&buf);

A question: when is todo_list->buf released?

>  		return error(_("Could not read %s."), todo_file);
>  	}
>  	close(fd);
>  
> -	res = parse_insn_buffer(buf.buf, todo_list, opts);
> +	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
> -	strbuf_release(&buf);
>  	if (res)
>  		return error(_("Unusable instruction sheet: %s"), todo_file);
>  	return 0;

Nice.

> @@ -848,18 +859,33 @@ static int read_populate_opts(struct replay_opts *opts)
>  	return 0;
>  }
>  
> -static int walk_revs_populate_todo(struct commit_list **todo_list,
> +static int walk_revs_populate_todo(struct todo_list *todo_list,
>  				struct replay_opts *opts)
>  {
> +	enum todo_command command = opts->action == REPLAY_PICK ?
> +		TODO_PICK : TODO_REVERT;
>  	struct commit *commit;
> -	struct commit_list **next;
>  
>  	if (prepare_revs(opts))
>  		return -1;
>  
> -	next = todo_list;
> -	while ((commit = get_revision(opts->revs)))
> -		next = commit_list_append(commit, next);
> +	while ((commit = get_revision(opts->revs))) {
> +		struct todo_item *item = append_todo(todo_list);
> +		const char *commit_buffer = get_commit_buffer(commit, NULL);

I see that you are creating todo file contents while walking revision list,
something that was left for later in current / previous implementation
of the sequencer...

[Added: I see it was done by format_todo() called from save_todo()]

> +		const char *subject;
> +		int subject_len;
> +
> +		item->command = command;
> +		item->commit = commit;
> +		item->offset_in_buf = todo_list->buf.len;
> +		subject_len = find_commit_subject(commit_buffer, &subject);
> +		strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
> +			opts->action == REPLAY_PICK ?  "pick" : "revert",

Wouldn't it be simpler to use

  +			todo_command_strings[command],

Also, this string does not change during the loop, though I guess
compiler should be able to optimize it.

> +			find_unique_abbrev(commit->object.oid.hash,
> +				DEFAULT_ABBREV),
> +			subject_len, subject);

...Did format of the 'todo' file changed?  And if yes, was it in backward
compatible way, so that "git revert" or "git cherry-pick" started with
old version of Git can be continued with new version, and what is also
important (for somebody who sometimes uses system-installed Git, and
sometimes user-compiled one) the reverse: started with new, continued
with old?

> +		unuse_commit_buffer(commit, commit_buffer);
> +	}
>  	return 0;
>  }
>  
> @@ -964,30 +990,24 @@ static int sequencer_rollback(struct replay_opts *opts)
>  	return -1;
>  }
>  
> -static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
> +static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
>  {
>  	static struct lock_file todo_lock;
> -	struct strbuf buf = STRBUF_INIT;
> -	int fd;
> +	const char *todo_path = get_todo_path(opts);
> +	int next = todo_list->current, offset, fd;

The "next = todo_list->current" looks a bit strange.  Also, we do not
change todo_list->current, we use it in one place, so it can be used
directly without help of temporary / helper variable.  But that is
just my personal opinion.

Also, from 'next', 'offset' and 'fd', all those are different
uses of int: the index (int, rarely size_t), the offset in string
(formally ptrdiff_t, or size_t, but usually int), and the file descriptor.
I think from those the file descriptor could be kept in separate line;
it would help diff to be more readable.  But this is fairly marginal
nitpicking, and a matter of personal opinion.

>  
> -	fd = hold_lock_file_for_update(&todo_lock, git_path_todo_file(), 0);
> +	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
>  	if (fd < 0)
>  		return error_errno(_("Could not lock '%s'"),
>  				   git_path_todo_file());

We should use 'todo_path' here... and this should be done in
one of earlier patches, isn't it?

This means that

  +	const char *todo_path = get_todo_path(opts);

should better be moved to earlier patch, too.

Or maybe not.  But it looks like missed git_path_todo_file() -> get_todo_path(opts)
[-> todo_path ] change.  If it was left because of planned whole rewrite,
it should be mentioned in the commit message of that earlier commit,
isn't it?

> -	if (format_todo(&buf, todo_list, opts) < 0) {
> -		strbuf_release(&buf);
> -		return error(_("Could not format %s."), git_path_todo_file());

Can we still get this error?  Could we get this error anyway,
and under what conditions?

> -	}
> -	if (write_in_full(fd, buf.buf, buf.len) < 0) {
> -		strbuf_release(&buf);
> -		return error_errno(_("Could not write to %s"),
> -				   git_path_todo_file());
> -	}
> +	offset = next < todo_list->nr ?
> +		todo_list->items[next].offset_in_buf : todo_list->buf.len;
> +	if (write_in_full(fd, todo_list->buf.buf + offset,
> +			todo_list->buf.len - offset) < 0)
> +		return error(_("Could not write to %s (%s)"),
> +			todo_path, strerror(errno));

Ah, so it saves the remaining todo_items on todo_list, not the
whole todo_list... the name does not fully show it.

> -	if (commit_lock_file(&todo_lock) < 0) {
> -		strbuf_release(&buf);
> -		return error(_("Error wrapping up %s."), git_path_todo_file());
> -	}
> -	strbuf_release(&buf);
> +	if (commit_lock_file(&todo_lock) < 0)
> +		return error(_("Error wrapping up %s."), todo_path);

Note: this is unrelated change, but we usually put paths in quotes, like this

  +		return error(_("Error wrapping up '%s'."), todo_path);

(in this and earlier error message), so that paths containing spaces show
correctly and readably to the user.  Though this possibly is not a problem
for this path.

Also, how user is to understand "wrapping up"?

>  	return 0;
>  }
>  
> @@ -1026,9 +1046,8 @@ static int save_opts(struct replay_opts *opts)
>  	return res;
>  }
>  
> -static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
> +static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
>  {
> -	struct commit_list *cur;
>  	int res;
>  
>  	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
> @@ -1038,10 +1057,12 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
>  	if (read_and_refresh_cache(opts))
>  		return -1;
>  
> -	for (cur = todo_list; cur; cur = cur->next) {
> +	while (todo_list->current < todo_list->nr) {

Why replace for loop with while loop?  Especially that now it
looks more for-y ;-)

  +	for ( ; todo_list->current < todo_list->nr; todo_list->current++) {

Oh... I now see why.

> +		struct todo_item *item = todo_list->items + todo_list->current;
> -		if (save_todo(cur, opts))
> +		if (save_todo(todo_list, opts))
>  			return -1;
> -		res = do_pick_commit(cur->item, opts);
> +		res = do_pick_commit(item->command, item->commit, opts);

I don't quite understand what sequencer tried to do here, but the
change looks all right.

> +		todo_list->current++;
>  		if (res)
>  			return res;
>  	}
> @@ -1066,7 +1087,8 @@ static int continue_single_pick(void)
>  
>  static int sequencer_continue(struct replay_opts *opts)
>  {
> -	struct commit_list *todo_list = NULL;
> +	struct todo_list todo_list = TODO_LIST_INIT;
> +	int res;
>  
>  	if (!file_exists(get_todo_path(opts)))
>  		return continue_single_pick();
> @@ -1083,21 +1105,24 @@ static int sequencer_continue(struct replay_opts *opts)
>  	}
>  	if (index_differs_from("HEAD", 0))
>  		return error_dirty_index(opts);
> -	todo_list = todo_list->next;
> -	return pick_commits(todo_list, opts);
> +	todo_list.current++;
> +	res = pick_commits(&todo_list, opts);
> +	todo_list_release(&todo_list);
> +	return res;

Nice.  Looks correct.

>  }
>  
>  static int single_pick(struct commit *cmit, struct replay_opts *opts)
>  {
>  	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
> -	return do_pick_commit(cmit, opts);
> +	return do_pick_commit(opts->action == REPLAY_PICK ?
> +		TODO_PICK : TODO_REVERT, cmit, opts);

The ternary conditional operator here translates one enum to other enum,
isn't it?

>  }
>  
>  int sequencer_pick_revisions(struct replay_opts *opts)
>  {
> -	struct commit_list *todo_list = NULL;
> +	struct todo_list todo_list = TODO_LIST_INIT;
>  	unsigned char sha1[20];
> -	int i;
> +	int i, res;
>  
>  	if (opts->subcommand == REPLAY_NONE)
>  		assert(opts->revs);
> @@ -1171,7 +1196,9 @@ int sequencer_pick_revisions(struct replay_opts *opts)
>  	if (save_head(sha1_to_hex(sha1)) ||
>  			save_opts(opts))
>  		return -1;
> -	return pick_commits(todo_list, opts);
> +	res = pick_commits(&todo_list, opts);
> +	todo_list_release(&todo_list);
> +	return res;

Looks correct.  And consistent.

>  }
>  
>  void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
> 

Re: [PATCH 10/22] sequencer: avoid completely different messages for different actions

[Jakub Narębski]

CC-ed to Jiang Xin, L10N coordinator.

W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:

> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index cbdce6d..1b65202 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -232,11 +232,8 @@ static int error_dirty_index(struct replay_opts *opts)
>  	if (read_cache_unmerged())
>  		return error_resolve_conflict(action_name(opts));
>  
> -	/* Different translation strings for cherry-pick and revert */
> -	if (opts->action == REPLAY_PICK)
> -		error(_("Your local changes would be overwritten by cherry-pick."));
> -	else
> -		error(_("Your local changes would be overwritten by revert."));
> +	error(_("Your local changes would be overwritten by %s."),
> +		action_name(opts));

If I understand it correctly, it would make "revert" or "cherry-pick"
untranslated part of error message.  You would need to use translation
on the result with "_(action_name(opts))", you would have to mark
todo_command_strings elements for gettext lexicon with N_(...).

I am rather against this change (see also below).


From the first glance I though that there would be no problem with
translation legos / jigsaw it introduces, namely that the "revert"
and "cherry-pick" would require different rest of text:

 po/bg.po:msgid "Your local changes would be overwritten by cherry-pick."
 po/bg.po-msgstr "Локалните ви промени ще бъдат презаписани при отбирането на подавания."
 --
 po/bg.po:msgid "Your local changes would be overwritten by revert."
 po/bg.po-msgstr "Локалните ви промени ще бъдат презаписани при отмяната на подавания."

 po/de.po:msgid "Your local changes would be overwritten by cherry-pick."
 po/de.po-msgstr "Ihre lokalen Änderungen würden durch den Cherry-Pick überschrieben werden."
 --
 po/de.po:msgid "Your local changes would be overwritten by revert."
 po/de.po-msgstr "Ihre lokalen Änderungen würden durch den Revert überschrieben werden."

But it turns out that "revert" and "cherry-pick" can be of different
gender:

 po/ca.po:msgid "Your local changes would be overwritten by cherry-pick."
 po/ca.po-msgstr "Els vostres canvis locals se sobreescriurien pel recull de cireres."
 --
 po/ca.po:msgid "Your local changes would be overwritten by revert."
 po/ca.po-msgstr "Els vostres canvis locals se sobreescriurien per la reversió."

In some cases "revert" and "cherry-pick" are not translated literally
(but compare translation for similar language: po/bg.po, without this):

 po/ru.po:msgid "Your local changes would be overwritten by cherry-pick."
 po/ru.po-msgstr "Ваши локальные изменение будут перезаписаны отбором лучшего."
 --
 po/ru.po:msgid "Your local changes would be overwritten by revert."
 po/ru.po-msgstr "Ваши локальные изменение будут перезаписаны возвратом коммита."

Similar for (but here one side uses untranslated English term...):

 po/vi.po:msgid "Your local changes would be overwritten by cherry-pick."
 po/vi.po-msgstr "Các thay đổi nội bộ của bạn có thể bị ghi đè bởi lệnh cherry-pick."
 --
 po/vi.po:msgid "Your local changes would be overwritten by revert."
 po/vi.po-msgstr "Các thay đổi nội bộ của bạn có thể bị ghi đè bởi lệnh hoàn nguyên."

For some I don't know which is the case:

 po/zh_CN.po:msgid "Your local changes would be overwritten by cherry-pick."
 po/zh_CN.po-msgstr "您的本地修改将被拣选操作覆盖。"
 --
 po/zh_CN.po:msgid "Your local changes would be overwritten by revert."
 po/zh_CN.po-msgstr "您的本地修改将被还原操作覆盖。"

Unless we want to require to use English terms:

 po/sv.po:msgid "Your local changes would be overwritten by cherry-pick."
 po/sv.po-msgstr "Dina lokala ändringar skulle skrivas över av \"cherry-pick\"."
 --
 po/sv.po:msgid "Your local changes would be overwritten by revert."
 po/sv.po-msgstr "Dina lokala ändringar skulle skrivas över av \"revert\"."


>  
>  	if (advice_commit_before_merge)
>  		advise(_("Commit your changes or stash them to proceed."));
> 

Re: [PATCH 11/22] sequencer: get rid of the subcommand field

[Jakub Narębski]

W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:

> The subcommands are used exactly once, at the very beginning of
> sequencer_pick_revisions(), to determine what to do. This is an
> unnecessary level of indirection: we can simply call the correct
> function to begin with. So let's do that.

Looks good.  Parsing is moved from parse_args(), now unnecessary,
to the new run_sequencer().  Which also picked up dispatch from
sequencer_pick_revisions() - that sometimes didn't pick revisions :-o.

"All problems in computer science can be solved by another level
 of indirection, except of course for the problem of too many
 indirections." -- David John Wheeler

> 
> While at it, ensure that the subcommands return an error code so that
> they do not have to die() all over the place (bad practice for library
> functions...).

This perhaps should be moved to a separate patch, but I guess
there is a reason behind "while at it".

Also subcommand functions no longer are local to sequencer.c

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  builtin/revert.c | 36 ++++++++++++++++--------------------
>  sequencer.c      | 35 +++++++++++------------------------
>  sequencer.h      | 13 ++++---------
>  3 files changed, 31 insertions(+), 53 deletions(-)

Nice size reduction.

Re: [PATCH 08/22] sequencer: remove overzealous assumption

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2551 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
> > The sequencer was introduced to make the cherry-pick and revert
> > functionality available as library function, with the original idea
> > being to extend the sequencer to also implement the rebase -i
> > functionality.
> > 
> > The test to ensure that all of the commands in the script are identical
> > to the overall operation does not mesh well with that.
> 
> Actually the question is what does the test that got removed in this
> commit actually check.  Is it high-level sanity check that todo list
> for git-cherry-pick contains only 'pick', and for git-revert contains
> only 'revert'?

It might have been that at some stage.

But should we really check that? Or should we check the *effects*?

I am of the opinion that overzealous checking of certain implementation
details is something to be avoided.

> > Therefore let's just get rid of the test that wants to verify that this
> > limitation is still in place, in preparation for the upcoming work to
> > teach the sequencer to do rebase -i's work.
> 
> Is it "upcoming work" as in one of the patches in this series?
> If so, which patch?

I left this a little vague, didn't I? ;-)

The problem is that the `git-rebase-todo` most definitely does *not* want
to be restricted to a single command.

So if you must have a patch that disagrees with this overzealous check,
the "revamp todo parsing" one is probably the first. But it is better to
think of this at a higher level than just patches: it is wrong to limit
the todo script to contain only identical commands.

> > diff --git a/t/t3510-cherry-pick-sequence.sh b/t/t3510-cherry-pick-sequence.sh
> > index 7b7a89d..6465edf 100755
> > --- a/t/t3510-cherry-pick-sequence.sh
> > +++ b/t/t3510-cherry-pick-sequence.sh
> > @@ -459,17 +459,6 @@ test_expect_success 'malformed instruction sheet 1' '
> >  	test_expect_code 128 git cherry-pick --continue
> >  '
> >  
> > -test_expect_success 'malformed instruction sheet 2' '
> 
> Hmmm... the description is somewhat lacking (especially compared to
> the rest of test), anyway.
> 
> BTW. we should probably rename 'malformed instruction sheet 2'
> to 'malformed instruction sheet' if there are no further such
> tests after this removal, isn't it?

No, we cannot rename it after this patch because the patch removes it ;-)
(It is not a file name but really a label for a test case.)

Thanks for the review,
Johannes

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Jakub Narębski]

W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:

> The `git-rebase-todo` file contains a list of commands. Most of those
> commands have the form
> 
> 	<verb> <sha1> <oneline>
> 
> The <oneline> is displayed primarily for the user's convenience, as
> rebase -i really interprets only the <verb> <sha1> part. However, there
> are *some* places in interactive rebase where the <oneline> is used to
> display messages, e.g. for reporting at which commit we stopped.
> 
> So let's just remember it when parsing the todo file; we keep a copy of
> the entire todo file anyway (to write out the new `done` and
> `git-rebase-todo` file just before processing each command), so all we
> need to do is remember the begin and end offsets.

Actually what we remember is pointer and length, or begin offset and length,
not offset and offset.

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Nice, I'll see how it is used later (and in which commit in series).

> ---
>  sequencer.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 06759d4..3398774 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>  struct todo_item {
>  	enum todo_command command;
>  	struct commit *commit;
> +	const char *arg;
> +	int arg_len;

Why 'arg', and not 'oneline', or 'subject'?
I'm not saying it is bad name.

>  	size_t offset_in_buf;
>  };
>  
> @@ -760,6 +762,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  	status = get_sha1(bol, commit_sha1);
>  	*end_of_object_name = saved;
>  
> +	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> +	item->arg_len = (int)(eol - item->arg);
> +

Does it work correctly for line without <oneline>, that is

  	<verb> <sha1>

I think it does, but I not entirely sure.

>  	if (status < 0)
>  		return -1;
>  
> @@ -880,6 +885,8 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
>  
>  		item->command = command;
>  		item->commit = commit;
> +		item->arg = NULL;
> +		item->arg_len = 0;
>  		item->offset_in_buf = todo_list->buf.len;
>  		subject_len = find_commit_subject(commit_buffer, &subject);
>  		strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
> 

Re: [PATCH 08/22] sequencer: remove overzealous assumption

[Jakub Narębski]

Hello Johannes,

W dniu 31.08.2016 o 20:36, Johannes Schindelin pisze:
> On Wed, 31 Aug 2016, Jakub Narębski wrote: 
>> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
 
> I am of the opinion that overzealous checking of certain implementation
> details is something to be avoided.

I agree.

>>> Therefore let's just get rid of the test that wants to verify that this
>>> limitation is still in place, in preparation for the upcoming work to
>>> teach the sequencer to do rebase -i's work.
>>
>> Is it "upcoming work" as in one of the patches in this series?
>> If so, which patch?
> 
> I left this a little vague, didn't I? ;-)
> 
> The problem is that the `git-rebase-todo` most definitely does *not* want
> to be restricted to a single command.
> 
> So if you must have a patch that disagrees with this overzealous check,
> the "revamp todo parsing" one is probably the first. But it is better to
> think of this at a higher level than just patches: it is wrong to limit
> the todo script to contain only identical commands.

I see.  Right.

I wonder: would 'git cherry-pick --continue' be able to finish
'git revert', and vice versa, then?  Or 'git sequencer --continue'?

>>> diff --git a/t/t3510-cherry-pick-sequence.sh b/t/t3510-cherry-pick-sequence.sh
>>> index 7b7a89d..6465edf 100755
>>> --- a/t/t3510-cherry-pick-sequence.sh
>>> +++ b/t/t3510-cherry-pick-sequence.sh
>>> @@ -459,17 +459,6 @@ test_expect_success 'malformed instruction sheet 1' '
>>>  	test_expect_code 128 git cherry-pick --continue
>>>  '
>>>  
>>> -test_expect_success 'malformed instruction sheet 2' '
>>
>> Hmmm... the description is somewhat lacking (especially compared to
>> the rest of test), anyway.
>>
>> BTW. we should probably rename 'malformed instruction sheet 2'
>> to 'malformed instruction sheet' if there are no further such
>> tests after this removal, isn't it?
> 
> No, we cannot rename it after this patch because the patch removes it ;-)
> (It is not a file name but really a label for a test case.)

Ooops.  What I wanted to say that after removing the test case named
'malformed instruction sheet 2' we should also rename *earlier* test
case from 'malformed instruction sheet 1' to 'malformed instruction sheet',
as it is now the only 'malformed instruction sheet *' test case.

Re: [PATCH 08/22] sequencer: remove overzealous assumption

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> The problem is that the `git-rebase-todo` most definitely does *not* want
> to be restricted to a single command.

Looking at the test in question, it is about what happens when "git
cherry-pick A..B" runs, and the test assumes it gets a sequence of
"pick".  It would be a bug to see any "revert" in there if you are
doing a multi cherry-pick.

And what is checked is if "cherry-pick --continue" notices a
malformed instruction that has something other than "pick".

If the final invocation were "sequencer --continue", then I
perfectly well understand why it is not a good idea to limit the set
of instructions only to "pick" (or "revert").  But I do not think
the test performed here is for that general case.  The user knows
Git stopped in the middle of cherry-pick and expects cherry-pick to
continue.

> So if you must have a patch that disagrees with this overzealous check,
> the "revamp todo parsing" one is probably the first. But it is better to
> think of this at a higher level than just patches: it is wrong to limit
> the todo script to contain only identical commands.

So if you think of this at even higher level, the check done in
parse_insn_line() that _assumes_ that opts->action must match the
actions on each line is _WRONG_, but what this test expects to see
is perfectly reasonable, I would think.

It is a different matter if it makes sense to _verify_ that the user
didn't make nonsensical change to the generated insn and error out
when s/he did.  I tend to think it is pointless, and I wouldn't object
if this test is removed due to that reason.

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

>> diff --git a/sequencer.c b/sequencer.c
>> index 06759d4..3398774 100644
>> --- a/sequencer.c
>> +++ b/sequencer.c
>> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>>  struct todo_item {
>>  	enum todo_command command;
>>  	struct commit *commit;
>> +	const char *arg;
>> +	int arg_len;
>
> Why 'arg', and not 'oneline', or 'subject'?
> I'm not saying it is bad name.

I am not sure what the "commit" field of type "struct commit *" is
for.  It is not needed until it is the commit's turn to be picked or
reverted; if we end up stopping in the middle, parsing the commit
object for later steps will end up being wasted effort.

Also, when the sequencer becomes one sequencer to rule them all, the
command set may contain something that does not even mention any
commit at all (think: exec).

So I am not sure if we want a parsed commit there (I would not
object if we kept the texual object name read from the file,
though).  The "one sequencer to rule them all" may even have to say
"now give name ':1' to the result of the previous operation" in one
step and in another later step have an instruction "merge ':1'".
When that happens, you cannot even pre-populate the commit object
when the sequencer reads the file, as the commit has not yet been
created at that point.

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Jakub Narębski]

W dniu 31.08.2016 o 21:10, Junio C Hamano pisze:
> Jakub Narębski <jnareb@gmail.com> writes:
> 
>>> diff --git a/sequencer.c b/sequencer.c
>>> index 06759d4..3398774 100644
>>> --- a/sequencer.c
>>> +++ b/sequencer.c
>>> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>>>  struct todo_item {
>>>  	enum todo_command command;
>>>  	struct commit *commit;
>>> +	const char *arg;
>>> +	int arg_len;
 
> I am not sure what the "commit" field of type "struct commit *" is
> for.  It is not needed until it is the commit's turn to be picked or
> reverted; if we end up stopping in the middle, parsing the commit
> object for later steps will end up being wasted effort.

From what I understand this was what sequencer did before this
series, so it is not a regression (I think; the commit parsing
was in different function, but I think at the same place in
the callchain).

> 
> Also, when the sequencer becomes one sequencer to rule them all, the
> command set may contain something that does not even mention any
> commit at all (think: exec).

The "exec" line is a bit of exception, all other rebase -i commands
take commit as parameter.  It could always use NULL.

>
> So I am not sure if we want a parsed commit there (I would not
> object if we kept the texual object name read from the file,
> though).  The "one sequencer to rule them all" may even have to say
> "now give name ':1' to the result of the previous operation" in one
> step and in another later step have an instruction "merge ':1'".
> When that happens, you cannot even pre-populate the commit object
> when the sequencer reads the file, as the commit has not yet been
> created at that point.

True, --preserve-merges rebase is well, different.

Best,
-- 
Jakub Narebski

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

>>>> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>>>>  struct todo_item {
>>>>  	enum todo_command command;
>>>>  	struct commit *commit;
>>>> +	const char *arg;
>>>> +	int arg_len;
>  
>> I am not sure what the "commit" field of type "struct commit *" is
>> for.  It is not needed until it is the commit's turn to be picked or
>> reverted; if we end up stopping in the middle, parsing the commit
>> object for later steps will end up being wasted effort.
>
> From what I understand this was what sequencer did before this
> series, so it is not a regression (I think; the commit parsing
> was in different function, but I think at the same place in
> the callchain).

Yes, I agree with you and I didn't mean "this is a new bug" at all.
It just is an indication that further refactoring after this step is
needed, and it is likely to involve removal or modification of this
field.

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Jakub Narębski]

Hello Johannes,

W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

> The rebase command sports a `--gpg-sign` option that is heeded by the
> interactive rebase.

Should it be "sports" or "supports"?

> 
> This patch teaches the sequencer that trick, as part of the bigger
> effort to make the sequencer the work horse of the interactive rebase.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 48 +++++++++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 43 insertions(+), 5 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 4204cc8..e094ac2 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -15,6 +15,7 @@
>  #include "merge-recursive.h"
>  #include "refs.h"
>  #include "argv-array.h"
> +#include "quote.h"
>  
>  #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
>  
> @@ -33,6 +34,11 @@ static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
>   * being rebased.
>   */
>  static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
> +/*
> + * The following files are written by git-rebase just after parsing the
> + * command-line (and are only consumed, not modified, by the sequencer).
> + */

It is good to have this comment here.

> +static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")

I know it is not your fault, but I wonder why this file uses
snake_case_name, while all other use kebab-case-names.  That is,
why it is gpg_sign_opt and not gpg-sign-opt.

>  
>  /* We will introduce the 'interactive rebase' mode later */
>  #define IS_REBASE_I() 0
> @@ -129,6 +135,16 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
>  	return 1;
>  }
>  
> +static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
> +{
> +	static struct strbuf buf = STRBUF_INIT;
> +
> +	strbuf_reset(&buf);
> +	if (opts->gpg_sign)
> +		sq_quotef(&buf, "-S%s", opts->gpg_sign);
> +	return buf.buf;
> +}

All right, this function is quite clear.

Sidenote: it's a pity api-quote.txt is just a placeholder for proper
documentation (including sq_quotef()).  I also wonder why it is not
named sq_quotef_buf() or strbuf_addf_sq().

> +
>  void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
>  {
>  	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
> @@ -471,17 +487,20 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
>  
>  	if (IS_REBASE_I()) {
>  		env = read_author_script();
> -		if (!env)
> +		if (!env) {
> +			const char *gpg_opt = gpg_sign_opt_quoted(opts);
> +
>  			return error("You have staged changes in your working "
>  				"tree. If these changes are meant to be\n"
>  				"squashed into the previous commit, run:\n\n"
> -				"  git commit --amend $gpg_sign_opt_quoted\n\n"

How did this get expanded by error(), and why we want to replace
it if it works?

> +				"  git commit --amend %s\n\n"
>  				"If they are meant to go into a new commit, "
>  				"run:\n\n"
> -				"  git commit $gpg_sign_opt_quoted\n\n"
> +				"  git commit %s\n\n"
>  				"In both case, once you're done, continue "
>  				"with:\n\n"
> -				"  git rebase --continue\n");
> +				"  git rebase --continue\n", gpg_opt, gpg_opt);

Instead of passing option twice, why not make use of %1$s (arg reordering),
that is

  +				"  git commit --amend %1$s\n\n"
[...]
  +				"  git commit %1$s\n\n"


> +		}

So shell quoting is required only for error output.

>  	}
>  
>  	argv_array_init(&array);
> @@ -955,8 +974,27 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
>  
>  static int read_populate_opts(struct replay_opts *opts)
>  {
> -	if (IS_REBASE_I())
> +	if (IS_REBASE_I()) {
> +		struct strbuf buf = STRBUF_INIT;
> +
> +		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
> +			if (buf.len && buf.buf[buf.len - 1] == '\n') {
> +				if (--buf.len &&
> +				    buf.buf[buf.len - 1] == '\r')
> +					buf.len--;
> +				buf.buf[buf.len] = '\0';
> +			}

Isn't there some strbuf_chomp() / strbuf_strip_eof() function?
Though as strbuf_getline() uses something similar...

> +
> +			if (!starts_with(buf.buf, "-S"))
> +				strbuf_reset(&buf);

Should we signal that there was problem with a file contents?

> +			else {
> +				opts->gpg_sign = buf.buf + 2;
> +				strbuf_detach(&buf, NULL);

Wouldn't we leak 2 characters that got skipped?  Maybe xstrdup would
be better (if it is leaked, and not reattached)?

> +			}
> +		}
> +
>  		return 0;
> +	}
>  
>  	if (!file_exists(git_path_opts_file()))
>  		return 0;
> 

-- 
Jakub Narębski

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

>> +			else {
>> +				opts->gpg_sign = buf.buf + 2;
>> +				strbuf_detach(&buf, NULL);
>
> Wouldn't we leak 2 characters that got skipped?  Maybe xstrdup would
> be better (if it is leaked, and not reattached)?

An attempt to avoid leaking by calling free(opts->gpg_sign) would
make it crash, which would be even worse ;-).

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Jakub Narębski]

W dniu 31.08.2016 o 22:12, Junio C Hamano pisze:
> Jakub Narębski <jnareb@gmail.com> writes:
>> Johannes Schindelin wrote:

>>> +			else {
>>> +				opts->gpg_sign = buf.buf + 2;
>>> +				strbuf_detach(&buf, NULL);
>>
>> Wouldn't we leak 2 characters that got skipped?  Maybe xstrdup would
>> be better (if it is leaked, and not reattached)?
> 
> An attempt to avoid leaking by calling free(opts->gpg_sign) would
> make it crash, which would be even worse ;-).
 
Actually, from C standard:

"If ptr [in free(ptr)] does not point to a block of memory allocated
 with the above functions [malloc(), etc.], it causes undefined behavior."
                                                      ^^^^^^^^^

Which is even worse if it does not lead to crash.


Note that if the last line was

    +				sequencer_entrust(opts, strbuf_detach(&buf, NULL));

we can make it not leak.  A bit tricksy, though.


Though xstrdup(buf.buf + 2) followed by strbuf_release(&buf) would
make free(opts->gpg_sign) possible without crash.  That is we can
work without *_entrust() mechanism at the cost of strdups.

-- 
Jakub Narębski

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

> Though xstrdup(buf.buf + 2) followed by strbuf_release(&buf) would
> make free(opts->gpg_sign) possible without crash.  That is we can
> work without *_entrust() mechanism at the cost of strdups.

Absolutely.

It is not like entrust() thing is free of allocation cost (it needs
to allocate an array of pointers to keep track of what to free) or
programmer's mental burden (you need to be careful what to entrust()
and what not to), so "at the cost of strdup(3)" is reasonable cost
of doing business in the way normal people expect the code to work.

Re: [PATCH 17/22] sequencer: allow editing the commit message on a case-by-case basis

[Jakub Narębski]

W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

> In the upcoming commits, we will implement more and more of rebase
> -i's functionality. One particular feature of the commands to come is
> that some of them allow editing the commit message while others don't,
> i.e. we cannot define in the replay_opts whether the commit message
> should be edited or not.

It's a nice, pretty and self contained refactoring step.  Small
enough that it is easy to review.

I would like to have in the commit message that it is sequencer_commit()
function that needs to rely on new parameter, instead of on a property
of command (of its replay_opts).  And that currently it simply passes
the buck to caller, which uses opts->edit, but in the future the
caller that is rebase -i would use todo_item and replay_opts based
expression.

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 6 +++---
>  sequencer.h | 2 +-
>  2 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index e094ac2..7e17d14 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -478,7 +478,7 @@ static char **read_author_script(void)
>   * (except, of course, while running an interactive rebase).
>   */
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty)
> +			  int allow_empty, int edit)
>  {
>  	char **env = NULL;
>  	struct argv_array array;
> @@ -513,7 +513,7 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
>  		argv_array_push(&array, "-s");
>  	if (defmsg)
>  		argv_array_pushl(&array, "-F", defmsg, NULL);
> -	if (opts->edit)
> +	if (edit)
>  		argv_array_push(&array, "-e");
>  	else if (!opts->signoff && !opts->record_origin &&
>  		 git_config_get_value("commit.cleanup", &value))
> @@ -779,7 +779,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	}
>  	if (!opts->no_commit)
>  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> -			opts, allow);
> +			opts, allow, opts->edit);
>  
>  leave:
>  	free_message(commit, &msg);
> diff --git a/sequencer.h b/sequencer.h
> index 9f63c31..fd02baf 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -50,7 +50,7 @@ int sequencer_rollback(struct replay_opts *opts);
>  int sequencer_remove_state(struct replay_opts *opts);
>  
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty);
> +			  int allow_empty, int edit);
>  
>  extern const char sign_off_header[];
>  
> 

Re: [PATCH 18/22] sequencer: support amending commits

[Jakub Narębski]

W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

> This teaches the sequencer_commit() function to take an argument that
> will allow us to implement "todo" commands that need to amend the commit
> messages ("fixup", "squash" and "reword").
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 6 ++++--
>  sequencer.h | 2 +-
>  2 files changed, 5 insertions(+), 3 deletions(-)

Nice and small addition of a new feature, a scaffolding for implementing
rebase -i using the sequencer.

> 
> diff --git a/sequencer.c b/sequencer.c
> index 7e17d14..20f7590 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -478,7 +478,7 @@ static char **read_author_script(void)
>   * (except, of course, while running an interactive rebase).
>   */
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit)
> +			  int allow_empty, int edit, int amend)

I guess we won't get much more parameters; it would get unwieldy
(and hard to remember).  Five is all right.

>  {
>  	char **env = NULL;
>  	struct argv_array array;
> @@ -507,6 +507,8 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
>  	argv_array_push(&array, "commit");
>  	argv_array_push(&array, "-n");
>  
> +	if (amend)
> +		argv_array_push(&array, "--amend");
>  	if (opts->gpg_sign)
>  		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
>  	if (opts->signoff)
> @@ -779,7 +781,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	}
>  	if (!opts->no_commit)
>  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> -			opts, allow, opts->edit);
> +			opts, allow, opts->edit, 0);

... even of this makes one need to check the calling convention,
what does this 0 mean.

>  
>  leave:
>  	free_message(commit, &msg);
> diff --git a/sequencer.h b/sequencer.h
> index fd02baf..2106c0d 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -50,7 +50,7 @@ int sequencer_rollback(struct replay_opts *opts);
>  int sequencer_remove_state(struct replay_opts *opts);
>  
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit);
> +			  int allow_empty, int edit, int amend);
>  
>  extern const char sign_off_header[];
>  
> 

Re: [PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Stefan Beller]

On Wed, Aug 31, 2016 at 10:29 AM, Jakub Narębski <jnareb@gmail.com> wrote:

>
> BTW. perhaps we would be able to continue with 'git continue', regardless
> of what we have started with, I wonder...
>

git continue as a shorthand for `git <relevant-cmd> --continue` sounds great.

If we were to introduce that, I think we would need to unify the rest
as well then, e.g.
Both revert as well as cherry-pick have --quit as well as --abort,
but rebase doesn't have --quit documented, but instead an additional
--skip  and --edit-todo

Would we pull these all up as a top level command? (That sounds not so
great to me)

Re: [PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1023 bytes --]

Hi Kuba and Stefan,

On Wed, 31 Aug 2016, Stefan Beller wrote:

> On Wed, Aug 31, 2016 at 10:29 AM, Jakub Narębski <jnareb@gmail.com> wrote:
> 
> >
> > BTW. perhaps we would be able to continue with 'git continue', regardless
> > of what we have started with, I wonder...
> >
> 
> git continue as a shorthand for `git <relevant-cmd> --continue` sounds great.

Before we get ahead of ourselves:

1) this has nothing to do with the patch series at hand, and

2) if we were to introduce `git continue`, we would need to think long and
   hard about the following issues:

	I) are there potentially ambiguous <relevant-cmd>s that the user
	   may want to continue?

	II) what about options? You can say `git rebase --continue
	    --no-ff`, for example, but not `git cherry-pick --continue
	    --no-ff`...

	III) Would it not be confusing to have a subcommand `continue`
	     that does *not* serve a *single* purpose? It's kinda flying
	     into the face of the Unix philosophy.

Ciao,
Dscho

Re: [PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 23134 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:
> 
> > When we came up with the "sequencer" idea, we really wanted to have
> > kind of a plumbing equivalent of the interactive rebase. Hence the
> > choice of words: the "todo" script, a "pick", etc.
> > 
> > However, when it came time to implement the entire shebang, somehow this
> > idea got lost and the sequencer was used as working horse for
> > cherry-pick and revert instead. So as not to interfere with the
> > interactive rebase, it even uses a separate directory to store its
> > state.
> > 
> > Furthermore, it also is stupidly strict about the "todo" script it
> > accepts: while it parses commands in a way that was *designed* to be
> > similar to the interactive rebase, it then goes on to *error out* if the
> > commands disagree with the overall action (cherry-pick or revert).
> 
> Does this mean that after the change you would be able to continue
> "git revert" with "git cherry-pick --continue", and vice versa?  Or that
> it would be possible for git-cherry-pick to do reverts (e.g. with ^<rev>)?

I guess that I allow that now. Is it harmful? I dunno.

> > Let's just bite the bullet and rewrite the entire parser; the code now
> > becomes not only more elegant: it allows us to go on and teach the
> > sequencer how to parse *true* "todo" scripts as used by the interactive
> > rebase itself. In a way, the sequencer is about to grow up to do its
> > older brother's job. Better.
> 
> Sidenote: this is not your fault, but Git doesn't do a good job on
> changes which are mostly rewrites, trying to match stray '}' and the
> like in generated diff.  I wonder if existing diff heuristic options
> could help here.

I guess --patience would have helped. Or Michael's upcoming
diff-heuristics.

> > While at it, do not stop at the first problem, but list *all* of the
> > problems. This helps the user by allowing to address all issues in
> > one go rather than going back and forth until the todo list is valid.
> 
> That is also a good change, though I wonder how often users need
> to worry about this outside interactive rebase case.  If it is
> preparation for rebase -i, where instruction list is written by
> prone to errors human, it would be nice to have this information
> in the commit message.

Okay.

> > diff --git a/sequencer.c b/sequencer.c
> > index 982b6e9..cbdce6d 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -473,7 +473,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
> >  		return 1;
> >  }
> >  
> > -static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
> > +enum todo_command {
> > +	TODO_PICK,
> > +	TODO_REVERT
> > +};
> 
> Do we have a naming convention for enums elements?  Or are we explicitly
> making enums and #defines interchangeable?  I wonder...
> 
> ...uh, I see we don't have naming convention, but all caps snake-case
> names dominate:
> 
>   $ git grep -A2 'enum .* {'
>   [...]
>   diff.h:enum color_diff {
>   diff.h- DIFF_RESET = 0,
>   diff.h- DIFF_CONTEXT = 1,
>   --
>   dir.c:enum path_treatment {
>   dir.c-  path_none = 0,
>   dir.c-  path_recurse,
>   --
> 
> Shouldn't we say 'TODO_PICK = 0' explicitly, though?

Sure.

> > +static const char *todo_command_strings[] = {
> > +	"pick",
> > +	"revert"
> > +};
> 
> It's a bit pity that we cannot use designated inits, and hanging comma,
> (from ISO C99 standard).  That is:
> 
>   +static const char *todo_command_strings[] = {
>   +	[TODO_PICK]   = "pick",
>   +	[TODO_REVERT] = "revert",
>   +};

I agree, it is a pity. I could do something like I did in fsck.c:

	#define FOREACH_TODO_COMMAND(FUNC) \
		FUNC(PICK, "pick") \
		FUNC(REVERT, "revert")

	#define COMMAND_ID(id, string) TODO_##id,
	enum todo_command {
		FOREACH_TODO_COMMAND(COMMAND_ID)
		TODO_END
	};
	#undef COMMAND_ID

	#define COMMAND_ID(id, string) string,
	static const char *todo_command_string[] = {
		FOREACH_TODO_COMMAND(COMMAND_ID)
		NULL
	};
	#undef COMMAND_ID

However, this is not even readable, let alone any other type of an
improvement. So I won't.

> > @@ -548,7 +568,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
> 
> From here on changes are about
> 
>   s/opts->action == REPLAY_\(PICK\|REVERT\)/command == TODO_\1/
> 
> Do we still need opts->action, or it is just needed less,
> and it is 'todo' instruction that decides about command
> (as it should)?

We need opts->action. For example, the state directory changes depending
on it: REPLAY_INTERACTIVE_REBASE stores its stuff in
git_path("rebase-merge").

There is lots more behavior that also changes depending on opts->action.

> > [...]
> >  	if (res) {
> > -		error(opts->action == REPLAY_REVERT
> > +		error(command == TODO_REVERT
> >  		      ? _("could not revert %s... %s")
> >  		      : _("could not apply %s... %s"),
> >  		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
> 
> And here those changes end.
> 
>   s/opts->action == REPLAY_\(PICK\|REVERT\)/command == TODO_\1/
> 
> I wonder if Coccinelle / Undebt would help here; or would simple
> sed or query-and-replace-regexp be enough...

I did this by hand, to verify that I did nothing idiotic.

> > @@ -683,116 +703,107 @@ static int read_and_refresh_cache(struct replay_opts *opts)
> >  	return 0;
> >  }
> >  
> > -static int format_todo(struct strbuf *buf, struct commit_list *todo_list,
> > -		struct replay_opts *opts)
> > +struct todo_item {
> > +	enum todo_command command;
> > +	struct commit *commit;
> > +	size_t offset_in_buf;
> > +};
> > +
> > +struct todo_list {
> > +	struct strbuf buf;
> > +	struct todo_item *items;
> > +	int nr, alloc, current;
> > +};
> 
> So there should be s/commit_list [*]{1,2}todo_list/todo_list *todo_list/
> from here on?

Almost, but not quite.

> Hmmm... commit_list is, as defined in commit.h, a linked list.

That is the most prominent reason why the rest is not a mindless
conversion from commit_list to todo_list.

And we need todo_list as an array, because we need to be able to peek, or
even move, backwards from the current command.

> Here todo_list uses growable array implementation of list.  Which
> is I guess better on current CPU architecture, with slow memory,
> limited-size caches, and adjacency prefetching.

That is not the reason that an array is used here. The array allows us
much more flexibility.

One of the major performance improvements will come at the very end, for
example: the reordering of the fixup!/squash! lines. And that would be a
*major* pain to do if the todo_list were still a linked list.

> > +#define TODO_LIST_INIT { STRBUF_INIT, NULL, 0, 0, 0 }
> 
> Same as with other patches in this series, it would be enough to
> 
>   +#define TODO_LIST_INIT { STRBUF_INIT }

As it happens, after Hannes' comment about REPLAY_OPTIONS_INIT, I already
had changed TODO_LIST_INIT as indicated. I just had no time to send out
another iteration (besides, I wanted to give the sequencer-i patch series
more visibility).

> You are consistent.

Thank you!

> > -	struct commit_list *cur = NULL;
> > -	const char *sha1_abbrev = NULL;
> > -	const char *action_str = opts->action == REPLAY_REVERT ? "revert" : "pick";
> > -	const char *subject;
> > -	int subject_len;
> > +	strbuf_release(&todo_list->buf);
> > +	free(todo_list->items);
> > +	todo_list->items = NULL;
> > +	todo_list->nr = todo_list->alloc = 0;
> > +}
> >  
> > -	for (cur = todo_list; cur; cur = cur->next) {
> > -		const char *commit_buffer = get_commit_buffer(cur->item, NULL);
> > -		sha1_abbrev = find_unique_abbrev(cur->item->object.oid.hash, DEFAULT_ABBREV);
> > -		subject_len = find_commit_subject(commit_buffer, &subject);
> > -		strbuf_addf(buf, "%s %s %.*s\n", action_str, sha1_abbrev,
> > -			subject_len, subject);
> > -		unuse_commit_buffer(cur->item, commit_buffer);
> > -	}
> > -	return 0;
> > +struct todo_item *append_todo(struct todo_list *todo_list)
> 
> Errr... I don't quite understand the name of this function.
> What are you appending here to the todo_list?

A new item.

> Compare string_list_append() and string_list_append_nodup(),
> where the second parameter is item to append.

Yes, that is correct. In the case of a todo_item, things are a lot more
complicated, though. Some of the values have to be determined tediously
(such as the offset and length of the oneline after the "pick <oid>"
command). I just put those values directly into the newly allocated item,
is all.

> > +	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
> > +	return todo_list->items + todo_list->nr++;
> >  }
> >  
> > -static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
> > +static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
> 
> Why the change of return type?  

Because it makes no sense to return a commit here because not all commands
are about commits (think rebase -i's `exec`). It makes tons of sense to
return an error condition, though.

> Why now struct todo_item is first when struct replay_opts was last?

Those play very, very different roles.

The opts parameter used to provide parse_insn_line() with enough
information to complain loudly when the overall command was not identical
to the parsed command.

The item parameter is a receptacle for the parsed data. It will contain
the pointer to the commit that was previously returned, if any. But it
will also contain much more information, such as the command, the oneline,
the offset in the buffer, etc etc

So "opts" was an "in" parameter while "item" is an "out" one. Apples and
oranges.

> > +	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
> > +		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
> 
> skip_prefix() is such a nice abstraction...
> 
> > +			item->command = i;
> > +			break;
> > +		}
> 
> Nice.  Replacing if-elsif chain with loop.  
> 
> I guess any hashmap would be serious overkill, as there are and would be
> only a few actions possible.

If at all, we should use a trie here. But as you said: overkill to the
max.

> > +	if (i >= ARRAY_SIZE(todo_command_strings))
> > +		return -1;
> >  
> >  	/* Eat up extra spaces/ tabs before object name */
> >  	padding = strspn(bol, " \t");
> >  	if (!padding)
> > -		return NULL;
> > +		return -1;
> >  	bol += padding;
> >  
> > -	end_of_object_name = bol + strcspn(bol, " \t\n");
> > +	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
> 
> Why is this cast needed?

Because bol is a "const char *" and we need to put "NUL" temporarily to
*end_of_object_name:

> >  	saved = *end_of_object_name;
> >  	*end_of_object_name = '\0';
> >  	status = get_sha1(bol, commit_sha1);
> >  	*end_of_object_name = saved;

Technically, this would have made a fine excuse to teach get_sha1() a mode
where it expects a length parameter instead of relying on a NUL-terminated
string.

Practically, such fine excuses cost me months in this rebase--helper
project already, and I need to protect my time better.

> > -	/*
> > -	 * Verify that the action matches up with the one in
> > -	 * opts; we don't support arbitrary instructions
> > -	 */
> > -	if (action != opts->action) {
> > -		if (action == REPLAY_REVERT)
> > -		      error((opts->action == REPLAY_REVERT)
> > -			    ? _("Cannot revert during another revert.")
> 
> Errr... could the above ever happen?  Namely
> 
>   action != opts->action && action == REPLAY_REVERT && opts->action == REPLAY_REVERT
> 
> Surely not.

Your reply pointed to the very circumstance when this may happen: `git
cherry-pick --continue` after an interrupted `git revert`.

But then, I remove that code here, so I should not try to defend it.

> > -			    : _("Cannot revert during a cherry-pick."));
> > -		else
> > -		      error((opts->action == REPLAY_REVERT)
> > -			    ? _("Cannot cherry-pick during a revert.")
> > -			    : _("Cannot cherry-pick during another cherry-pick."));
> > -		return NULL;
> > -	}
> 
> Anyway, while it is / would be a good idea to prevent starting any
> sequencer-based command (cherry-pick, revert, soon rebase -i) when
> other command is in progress (cherry-pick, revert, soon rebase -i).
> That is, if cherry-pick / revert waits for user action, you cannot
> run another cherry-pick or revert.
> 
> Which I guess the above code was not about...

It was about that, though.

It went about it in a pretty round-about way: opts->action comes from the
name of the command ("was I called as `git revert` or `git cherry-pick`?")
and action comes from the todo script, which was assumed to be written by
a previous run of the sequencer, using the then-current value of
opts->action.

So it wrote that command into *every single line* of the todo script, *for
the sole purpose* of verifying that it was the same action when running
via --continue.

As I said earlier, I would not complain at all if an interrupted `git
revert` could be continued via `git cherry-pick --continue`.

If that is not desirable, I can reintroduce that overzealous check, but
that will have to wait until after v2.10.0. And it would require an
argument that convinces me.

> > +		item = append_todo(todo_list);
> 
> A better name, in my personal option, would be
> 
>   +		item = todo_list_next(todo_list);
> 
> Or todo_next(todo_list).

That sounds more like a function that performs the next command in the
todo_list.

While I agree that naming is hard, I still think that `append_todo()` with
the todo_list as single parameter and returning a todo_item is pretty much
self-explanatory: it appends a new item to the todo_list and returns a
pointer to it.

> > +		item->offset_in_buf = p - todo_list->buf.buf;
> > +		if (parse_insn_line(item, p, eol)) {
> > +			error("Invalid line: %.*s", (int)(eol - p), p);
> 
> This error message should, I think, be also translatable:
> 
>   +			error(_("Invalid line: %.*s"), (int)(eol - p), p);
> 
> > +			res |= error(_("Could not parse line %d."), i);

Sure. In the meantime, I consolidated those two error()s into one, and now
I also marked it translatable.

> BTW. would be we able to show where exactly there was problem parsing,
> that is at which character in line?  Or is it something for the future?

Maybe for the future.

> > -static int read_populate_todo(struct commit_list **todo_list,
> > +static int read_populate_todo(struct todo_list *todo_list,
> >  			struct replay_opts *opts)
> >  {
> >  	const char *todo_file = get_todo_path(opts);
> 
> If I understand it correctly, replay_opts is used only to find out
> correct todo_file, isn't it?

Probably. Maybe also to make certain code paths conditional on rebase -i
mode. Maybe also to figure out whether we run in verbose mode in the
future. Or something.

Think of this `read_populate_todo()` function more as if it were a method
of the "replay class", and the "opts" parameter is kind of "self" or
"this" or whatever it is called in your favorite object-oriented language.


> > -	if (strbuf_read(&buf, fd, 0) < 0) {
> > +	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
> >  		close(fd);
> > -		strbuf_release(&buf);
> 
> A question: when is todo_list->buf released?

Why, I am glad you asked! It is released in todo_list_release(), called at
the end e.g. of sequencer_continue().

> > -static int walk_revs_populate_todo(struct commit_list **todo_list,
> > +static int walk_revs_populate_todo(struct todo_list *todo_list,
> >  				struct replay_opts *opts)
> >  {
> > +	enum todo_command command = opts->action == REPLAY_PICK ?
> > +		TODO_PICK : TODO_REVERT;
> >  	struct commit *commit;
> > -	struct commit_list **next;
> >  
> >  	if (prepare_revs(opts))
> >  		return -1;
> >  
> > -	next = todo_list;
> > -	while ((commit = get_revision(opts->revs)))
> > -		next = commit_list_append(commit, next);
> > +	while ((commit = get_revision(opts->revs))) {
> > +		struct todo_item *item = append_todo(todo_list);
> > +		const char *commit_buffer = get_commit_buffer(commit, NULL);
> 
> I see that you are creating todo file contents while walking revision list,
> something that was left for later in current / previous implementation
> of the sequencer...

Not really. This function was always about generating a todo_list. It just
did not format it yet.

With the change of keeping the original formatting of the todo script
instead of re-formatting it in save_todo(), this function now has to
format the todo_list itself.

> > +		const char *subject;
> > +		int subject_len;
> > +
> > +		item->command = command;
> > +		item->commit = commit;
> > +		item->offset_in_buf = todo_list->buf.len;
> > +		subject_len = find_commit_subject(commit_buffer, &subject);
> > +		strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
> > +			opts->action == REPLAY_PICK ?  "pick" : "revert",
> 
> Wouldn't it be simpler to use
> 
>   +			todo_command_strings[command],
> 
> Also, this string does not change during the loop, though I guess
> compiler should be able to optimize it.

Sure!

> > +			find_unique_abbrev(commit->object.oid.hash,
> > +				DEFAULT_ABBREV),
> > +			subject_len, subject);
> 
> ...Did format of the 'todo' file changed?  And if yes, was it in backward
> compatible way, so that "git revert" or "git cherry-pick" started with
> old version of Git can be continued with new version, and what is also
> important (for somebody who sometimes uses system-installed Git, and
> sometimes user-compiled one) the reverse: started with new, continued
> with old?

The old format and the new format are compatible. In fact, sequencer's
format was based on rebase -i's format (which makes it all the more
surprising how much the processing deviated).

> > @@ -964,30 +990,24 @@ static int sequencer_rollback(struct replay_opts *opts)
> >  	return -1;
> >  }
> >  
> > -static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
> > +static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
> >  {
> >  	static struct lock_file todo_lock;
> > -	struct strbuf buf = STRBUF_INIT;
> > -	int fd;
> > +	const char *todo_path = get_todo_path(opts);
> > +	int next = todo_list->current, offset, fd;
> 
> The "next = todo_list->current" looks a bit strange.

Depending whether we need rebase -i processing or revert/cherry-pick's
slightly different one, the "current" position points to the next one
already...

> Also, we do not change todo_list->current, we use it in one place, so it
> can be used directly without help of temporary / helper variable.  But
> that is just my personal opinion.

No, it has nothing to do with opinion. It prepares the code to keep it
readable even when REPLAY_INTERACTIVE_REBASE is introduced.

> Also, from 'next', 'offset' and 'fd', all those are different uses of
> int: the index (int, rarely size_t), the offset in string (formally
> ptrdiff_t, or size_t, but usually int), and the file descriptor.  I
> think from those the file descriptor could be kept in separate line; it
> would help diff to be more readable.  But this is fairly marginal
> nitpicking, and a matter of personal opinion.

Right. At this point, I am really much more concerned about correctness of
code than discussing personal preferences.

> > -	fd = hold_lock_file_for_update(&todo_lock, git_path_todo_file(), 0);
> > +	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
> >  	if (fd < 0)
> >  		return error_errno(_("Could not lock '%s'"),
> >  				   git_path_todo_file());
> 
> We should use 'todo_path' here...

True.

> and this should be done in one of earlier patches, isn't it?

No. I deliberately skipped save_todo() from "future-proofing" as I planned
to rewrite it anyway. There is no point in future-proofing something you
are going to toss in a minute.

> > -	if (format_todo(&buf, todo_list, opts) < 0) {
> > -		strbuf_release(&buf);
> > -		return error(_("Could not format %s."), git_path_todo_file());
> 
> Can we still get this error?  Could we get this error anyway,
> and under what conditions?

No. We keep the original formatting. Keeping it cannot possibly result in
a formatting error.

> > -	}
> > -	if (write_in_full(fd, buf.buf, buf.len) < 0) {
> > -		strbuf_release(&buf);
> > -		return error_errno(_("Could not write to %s"),
> > -				   git_path_todo_file());
> > -	}
> > +	offset = next < todo_list->nr ?
> > +		todo_list->items[next].offset_in_buf : todo_list->buf.len;
> > +	if (write_in_full(fd, todo_list->buf.buf + offset,
> > +			todo_list->buf.len - offset) < 0)
> > +		return error(_("Could not write to %s (%s)"),
> > +			todo_path, strerror(errno));
> 
> Ah, so it saves the remaining todo_items on todo_list, not the
> whole todo_list... the name does not fully show it.

The name also does not fully show that it will write a "done" file after
the sequencer-i patch series.

> 
> > -	if (commit_lock_file(&todo_lock) < 0) {
> > -		strbuf_release(&buf);
> > -		return error(_("Error wrapping up %s."), git_path_todo_file());
> > -	}
> > -	strbuf_release(&buf);
> > +	if (commit_lock_file(&todo_lock) < 0)
> > +		return error(_("Error wrapping up %s."), todo_path);
> 
> Note: this is unrelated change, but we usually put paths in quotes, like this
> 
>   +		return error(_("Error wrapping up '%s'."), todo_path);
> 
> (in this and earlier error message), so that paths containing spaces show
> correctly and readably to the user.  Though this possibly is not a problem
> for this path.

Right.

> Also, how user is to understand "wrapping up"?

The same as before: the removed lines already had the error message,
missing the quotes, too.

Don't get me wrong: I am a big fan of consistency, and I wish that Git's
source code had more of it. So I would love to see a patch series that
makes all error messages consistently reporting paths enclosed in single
quotes.

I am also a big fan of the separation of concerns, though. And this patch
series' concern is consistency *with the existing code*.

So I won't change the error message that I inherited at this point.

> >  static int single_pick(struct commit *cmit, struct replay_opts *opts)
> >  {
> >  	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
> > -	return do_pick_commit(cmit, opts);
> > +	return do_pick_commit(opts->action == REPLAY_PICK ?
> > +		TODO_PICK : TODO_REVERT, cmit, opts);
> 
> The ternary conditional operator here translates one enum to other enum,
> isn't it?

Well, almost. Please note that the enum will receive a new value in the
sequencer-i patch series. And there is no equivalent todo_command for
REPLAY_INTERACTIVE_REBASE.

Thanks for the review!
Johannes

Re: [PATCH 10/22] sequencer: avoid completely different messages for different actions

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1832 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> CC-ed to Jiang Xin, L10N coordinator.
> 
> W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:
> 
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 7 ++-----
> >  1 file changed, 2 insertions(+), 5 deletions(-)
> > 
> > diff --git a/sequencer.c b/sequencer.c
> > index cbdce6d..1b65202 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -232,11 +232,8 @@ static int error_dirty_index(struct replay_opts *opts)
> >  	if (read_cache_unmerged())
> >  		return error_resolve_conflict(action_name(opts));
> >  
> > -	/* Different translation strings for cherry-pick and revert */
> > -	if (opts->action == REPLAY_PICK)
> > -		error(_("Your local changes would be overwritten by cherry-pick."));
> > -	else
> > -		error(_("Your local changes would be overwritten by revert."));
> > +	error(_("Your local changes would be overwritten by %s."),
> > +		action_name(opts));
> 
> If I understand it correctly, it would make "revert" or "cherry-pick"
> untranslated part of error message.  You would need to use translation
> on the result with "_(action_name(opts))", you would have to mark
> todo_command_strings elements for gettext lexicon with N_(...).
> 
> I am rather against this change (see also below).

Okay.

Unfortunately, I have to focus on the correctness of the code at the
moment (and Git for Windows does ship *without* translations for the time
being anyway, mostly to save on space, but also because users complained).

So I will take care of this after v2.10.0.

For the record, how is this supposed to be handled, in particular when I
introduce a new action whose action_name(opts) will be "rebase -i"? Do I
really need to repeat myself three times?

Ciao,
Dscho

Re: [PATCH 11/22] sequencer: get rid of the subcommand field

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 986 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:
> 
> > While at it, ensure that the subcommands return an error code so that
> > they do not have to die() all over the place (bad practice for library
> > functions...).
> 
> This perhaps should be moved to a separate patch, but I guess
> there is a reason behind "while at it".

Yes. It seemed like the logical thing to do: I already introduce a new
function, why should I shlep over a paradigm I do not want in the end?

> Also subcommand functions no longer are local to sequencer.c

They never were. All you had to do was to set a field and run the global
function.

The real problem there was that the different local functions needed
different parameters, and the round-about way to set those parameters as
fields in a struct and then call a global function with that struct just
makes it impossible to have compile-time safety.

Ciao,
Dscho

Re: [PATCH 08/22] sequencer: remove overzealous assumption

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2354 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 31.08.2016 o 20:36, Johannes Schindelin pisze:
> 
> I wonder: would 'git cherry-pick --continue' be able to finish
> 'git revert', and vice versa, then?  Or 'git sequencer --continue'?

I just tested this, via

	diff --git a/t/t3510-cherry-pick-sequence.sh
	b/t/t3510-cherry-pick-sequence.sh
	index 96c7640..085d8bc 100755
	--- a/t/t3510-cherry-pick-sequence.sh
	+++ b/t/t3510-cherry-pick-sequence.sh
	@@ -55,7 +55,7 @@ test_expect_success 'cherry-pick
	mid-cherry-pick-sequence' '
		git checkout HEAD foo &&
		git cherry-pick base &&
		git cherry-pick picked &&
	-       git cherry-pick --continue &&
	+       git revert --continue &&
		git diff --exit-code anotherpick

(Danger! Whitespace corrupted!!!)

It appears that this passes now.

Probably `git sequencer --continue` would work, too, if there was a `git
sequencer`. :0)

> > On Wed, 31 Aug 2016, Jakub Narębski wrote: 
> >> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
>  
> >>> diff --git a/t/t3510-cherry-pick-sequence.sh b/t/t3510-cherry-pick-sequence.sh
> >>> index 7b7a89d..6465edf 100755
> >>> --- a/t/t3510-cherry-pick-sequence.sh
> >>> +++ b/t/t3510-cherry-pick-sequence.sh
> >>> @@ -459,17 +459,6 @@ test_expect_success 'malformed instruction sheet 1' '
> >>>  	test_expect_code 128 git cherry-pick --continue
> >>>  '
> >>>  
> >>> -test_expect_success 'malformed instruction sheet 2' '
> >>
> >> Hmmm... the description is somewhat lacking (especially compared to
> >> the rest of test), anyway.
> >>
> >> BTW. we should probably rename 'malformed instruction sheet 2'
> >> to 'malformed instruction sheet' if there are no further such
> >> tests after this removal, isn't it?
> > 
> > No, we cannot rename it after this patch because the patch removes it ;-)
> > (It is not a file name but really a label for a test case.)
> 
> Ooops.  What I wanted to say that after removing the test case named
> 'malformed instruction sheet 2' we should also rename *earlier* test
> case from 'malformed instruction sheet 1' to 'malformed instruction sheet',
> as it is now the only 'malformed instruction sheet *' test case.

Actually, you know, I completely missed the fact that there was a
"malformed instruction sheet 3". I renumbered it.

Thanks,
Dscho

Re: [PATCH 08/22] sequencer: remove overzealous assumption

[Johannes Schindelin]

Hi Junio,

On Wed, 31 Aug 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > So if you must have a patch that disagrees with this overzealous
> > check, the "revamp todo parsing" one is probably the first. But it is
> > better to think of this at a higher level than just patches: it is
> > wrong to limit the todo script to contain only identical commands.
> 
> So if you think of this at even higher level, the check done in
> parse_insn_line() that _assumes_ that opts->action must match the
> actions on each line is _WRONG_, but what this test expects to see is
> perfectly reasonable, I would think.
> 
> It is a different matter if it makes sense to _verify_ that the user
> didn't make nonsensical change to the generated insn and error out when
> s/he did.  I tend to think it is pointless, and I wouldn't object if
> this test is removed due to that reason.

Fine. I will reintroduce that check. I guess I have the time ;-)

Ciao,
Dscho

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2115 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:
> 
> > The `git-rebase-todo` file contains a list of commands. Most of those
> > commands have the form
> > 
> > 	<verb> <sha1> <oneline>
> > 
> > The <oneline> is displayed primarily for the user's convenience, as
> > rebase -i really interprets only the <verb> <sha1> part. However, there
> > are *some* places in interactive rebase where the <oneline> is used to
> > display messages, e.g. for reporting at which commit we stopped.
> > 
> > So let's just remember it when parsing the todo file; we keep a copy of
> > the entire todo file anyway (to write out the new `done` and
> > `git-rebase-todo` file just before processing each command), so all we
> > need to do is remember the begin and end offsets.
> 
> Actually what we remember is pointer and length, or begin offset and length,
> not offset and offset.

Right. Fixed.

> > diff --git a/sequencer.c b/sequencer.c
> > index 06759d4..3398774 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
> >  struct todo_item {
> >  	enum todo_command command;
> >  	struct commit *commit;
> > +	const char *arg;
> > +	int arg_len;
> 
> Why 'arg', and not 'oneline', or 'subject'?
> I'm not saying it is bad name.

Because we will use it for `exec` commands' args, too. Clarified in the
commit message.

> > @@ -760,6 +762,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
> >  	status = get_sha1(bol, commit_sha1);
> >  	*end_of_object_name = saved;
> >  
> > +	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> > +	item->arg_len = (int)(eol - item->arg);
> > +
> 
> Does it work correctly for line without <oneline>, that is
> 
>   	<verb> <sha1>
> 
> I think it does, but I not entirely sure.

It does work correctly: in the example, *end_of_object_name would be '\n',
and strspn(end_of_object_name, " \t") would return 0.

Thanks for the review!
Dscho

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2441 bytes --]

Hi Junio,

On Wed, 31 Aug 2016, Junio C Hamano wrote:

> Jakub Narębski <jnareb@gmail.com> writes:
> 
> >> diff --git a/sequencer.c b/sequencer.c
> >> index 06759d4..3398774 100644
> >> --- a/sequencer.c
> >> +++ b/sequencer.c
> >> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
> >>  struct todo_item {
> >>  	enum todo_command command;
> >>  	struct commit *commit;
> >> +	const char *arg;
> >> +	int arg_len;
> >
> > Why 'arg', and not 'oneline', or 'subject'?
> > I'm not saying it is bad name.
> 
> I am not sure what the "commit" field of type "struct commit *" is
> for.  It is not needed until it is the commit's turn to be picked or
> reverted; if we end up stopping in the middle, parsing the commit
> object for later steps will end up being wasted effort.

No, it won't be wasted effort, as we *validate* the todo script this way.
And since we may very well need the info later (most rebases do not fail
in the middle), we store it, too.

> Also, when the sequencer becomes one sequencer to rule them all, the
> command set may contain something that does not even mention any
> commit at all (think: exec).

Right, in which case the "commit" field will have the value... *drum
roll*... NULL!

> So I am not sure if we want a parsed commit there (I would not
> object if we kept the texual object name read from the file,
> though).  The "one sequencer to rule them all" may even have to say
> "now give name ':1' to the result of the previous operation" in one
> step and in another later step have an instruction "merge ':1'".
> When that happens, you cannot even pre-populate the commit object
> when the sequencer reads the file, as the commit has not yet been
> created at that point.

These considerations are pretty hypothetical. I would even place a bet
that we will *never* have ":1" as names, not if I have anything to say...
;-)

What is not so hypothetical is that after parsing the todo_list, we will
have to act on the information contained therein. For example we will have
to cherry-pick some of the indicated commits (requiring a struct commit *
for use in do_pick_commit()). Another example: we may need to determine
the oneline for use in fixup!/squash! reordering.

So: keeping *that* aspect of the previous todo_list parsing, i.e. store a
pointer to the already-parsed commit, is the right thing to do.

Ciao,
Dscho

Re: [PATCH 19/22] sequencer: support cleaning up commit messages

[Jakub Narębski]

Hello Johannes,

W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

> The sequencer_commit() function already knows how to amend commits, and
> with this new option, it can also clean up commit messages (i.e. strip
> out commented lines). This is needed to implement rebase -i's 'fixup'
> and 'squash' commands as sequencer commands.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 10 +++++++---
>  sequencer.h |  3 ++-
>  2 files changed, 9 insertions(+), 4 deletions(-)

This looks like nice little piece of enhancement, building scaffolding
for sequencer-izing interactive rebase bit by bit.

> 
> diff --git a/sequencer.c b/sequencer.c
> index 20f7590..5ec956f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -478,7 +478,8 @@ static char **read_author_script(void)
>   * (except, of course, while running an interactive rebase).
>   */
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit, int amend)
> +			  int allow_empty, int edit, int amend,
> +			  int cleanup_commit_message)

All right, though it slowly begins coming close to the threshold
where using bitfield flags would be sensible.

>  {
>  	char **env = NULL;
>  	struct argv_array array;
> @@ -515,9 +516,12 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
>  		argv_array_push(&array, "-s");
>  	if (defmsg)
>  		argv_array_pushl(&array, "-F", defmsg, NULL);
> +	if (cleanup_commit_message)
> +		argv_array_push(&array, "--cleanup=strip");

Good.

>  	if (edit)
>  		argv_array_push(&array, "-e");
> -	else if (!opts->signoff && !opts->record_origin &&
> +	else if (!cleanup_commit_message &&

All right, explicit cleanup=strip overrides "commit.cleanup" config,
and turns off passing commit verbatim (incompatible with stripping)...

> +		 !opts->signoff && !opts->record_origin &&

...adding signoff and recording origin requires not passing commit
verbatim,...

>  		 git_config_get_value("commit.cleanup", &value))

..., and in other cases are check the "commit.cleanup"...

>  		argv_array_push(&array, "--cleanup=verbatim");

... and pass commit verbatim if it is not set.


Ah, well, the change you made looks good.

>  
> @@ -781,7 +785,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	}
>  	if (!opts->no_commit)
>  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> -			opts, allow, opts->edit, 0);
> +			opts, allow, opts->edit, 0, 0);

The calling convention begins to look unwieldy, but we have only
a single such callsite, and there are quite a bit callsites in
Git code that have similar API ("git grep ', 0, 0' -- '*.c'").
So we don't need to think about alternatives.  Yet.

It's a pity that emulation of named parameters in C requires
relying on designated inits from C99

  typedef struct {
    double pressure, moles, temp;
  } ideal_struct;

  #define ideal_pressure(...) ideal_pressure_base((ideal_struct){.pressure=1,   \
                                        .moles=1, .temp=273.15, __VA_ARGS__})

  double ideal_pressure_base(ideal_struct in)
  {
    return 8.314 * in.moles*in.temp/in.pressure;
  }

  ... ideal_pressure(.moles=2, .temp=373.15) ...

>  
>  leave:
>  	free_message(commit, &msg);
> diff --git a/sequencer.h b/sequencer.h
> index 2106c0d..e272549 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -50,7 +50,8 @@ int sequencer_rollback(struct replay_opts *opts);
>  int sequencer_remove_state(struct replay_opts *opts);
>  
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit, int amend);
> +			  int allow_empty, int edit, int amend,
> +			  int cleanup_commit_message);
>  
>  extern const char sign_off_header[];
>  
> 

Re: [PATCH 21/22] sequencer: left-trim the lines read from the script

[Jakub Narębski]

Hello Johannes,

W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

Subject: [PATCH 21/22] sequencer: left-trim the lines read from the script

In the subject, it should probably be without "the", as "lines"
are plural.

s/left-trim the lines/left-trim lines/

> Interactive rebase's scripts may be indented; We need to handle this
> case, too, now that we prepare the sequencer to process interactive
> rebases.

s/; We need/; we need/

Nice little bit of scaffolding for sequencer-izing rebase -i.

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 3 +++
>  1 file changed, 3 insertions(+)

Small change, easy to review.

> 
> diff --git a/sequencer.c b/sequencer.c
> index 0614b90..5efed2e 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -864,6 +864,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  	char *end_of_object_name;
>  	int i, saved, status, padding;
>  
> +	/* left-trim */
> +	bol += strspn(bol, " \t");
> +

Nice.  Thanks for the comment.  "left-trim" is better than "de-indent".

'bol' is beginning-of-line, isn't it (a complement to eol)?

>  	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
>  		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
>  			item->command = i;
> 

-- 
Jakub Narębski

Re: [PATCH 22/22] sequencer: refactor write_message()

[Jakub Narębski]

Hello Johannes,

W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

> The write_message() function safely writes an strbuf to a file.
> Sometimes this is inconvenient, though: the text to be written may not
> be stored in a strbuf, or the strbuf should not be released after
> writing.

By "this" you mean "using strbuf", isn't it?  It is not very obvious,
and I think it would be better to say it explicitly.

>
> Let's allow for such use cases by refactoring write_message() to allow
> for a convenience function write_file_gently(). As some of the upcoming
> callers of that new function will want to append a newline character,
> let's just add a flag for that, too.

This paragraph feels a bit convoluted.

As I understand it, you refactor "safely writing string to a file"
into write_with_lock_file(), and make write_message() use it.  The
new function makes it easy to create new convenience function 
write_file_gently(); as some of the upcoming callers of this new
function would want to append a newline character, add a flag for
it in write_file_gently(), and thus in write_with_lock_file().

Isn't it better / easier to understand?

> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 21 ++++++++++++++++++---
>  1 file changed, 18 insertions(+), 3 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 5efed2e..f5b5e5e 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -239,22 +239,37 @@ static void print_advice(int show_hint, struct replay_opts *opts)
>  	}
>  }
>  
> -static int write_message(struct strbuf *msgbuf, const char *filename)
> +static int write_with_lock_file(const char *filename,
> +				const void *buf, size_t len, int append_eol)
>  {
>  	static struct lock_file msg_file;
>  
>  	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
>  	if (msg_fd < 0)
>  		return error_errno(_("Could not lock '%s'"), filename);
> -	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
> +	if (write_in_full(msg_fd, buf, len) < 0)
>  		return error_errno(_("Could not write to %s"), filename);

You could have, for consistency, add quotes around filename (see previous
error_errno callsite), *while at it*:

  		return error_errno(_("Could not write to '%s'"), filename);


> -	strbuf_release(msgbuf);
> +	if (append_eol && write(msg_fd, "\n", 1) < 0)
> +		return error_errno(_("Could not write eol to %s"), filename);

Same here, and it wouldn't even be 'while at it'

  +		return error_errno(_("Could not write eol to '%s'"), filename);


>  	if (commit_lock_file(&msg_file) < 0)
>  		return error(_("Error wrapping up %s."), filename);

Another "while at it"... though the one that can be safely postponed
(well, the make message easier to understand part, not the quote
filename part):

  		return error(_("Error wrapping up writing to '%s'."), filename);


>  
>  	return 0;
>  }
>  
> +static int write_message(struct strbuf *msgbuf, const char *filename)
> +{
> +	int res = write_with_lock_file(filename, msgbuf->buf, msgbuf->len, 0);
> +	strbuf_release(msgbuf);
> +	return res;
> +}

Nice.

> +
> +static int write_file_gently(const char *filename,
> +			     const char *text, int append_eol)
> +{
> +	return write_with_lock_file(filename, text, strlen(text), append_eol);
> +}

Nice.  And it is static function, so we don't need to come up
with a better function name (to describe its function better).

> +
>  /*
>   * Reads a file that was presumably written by a shell script, i.e.
>   * with an end-of-line marker that needs to be stripped.
> 

And thus we got to the last patch in this series.  I have skipped
patches that already got reviewed; are there some that you would
like to have second review of?  Is there patch series that needs
to be applied earlier that needs a review?

P.S. I'll try to respond to your comments later today.

Regards,
-- 
Jakub Narębski

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2737 bytes --]

Hi Kuba & Junio,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 31.08.2016 o 21:10, Junio C Hamano pisze:
> > Jakub Narębski <jnareb@gmail.com> writes:
> > 
> >>> diff --git a/sequencer.c b/sequencer.c
> >>> index 06759d4..3398774 100644
> >>> --- a/sequencer.c
> >>> +++ b/sequencer.c
> >>> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
> >>>  struct todo_item {
> >>>  	enum todo_command command;
> >>>  	struct commit *commit;
> >>> +	const char *arg;
> >>> +	int arg_len;
>  
> > I am not sure what the "commit" field of type "struct commit *" is
> > for.  It is not needed until it is the commit's turn to be picked or
> > reverted; if we end up stopping in the middle, parsing the commit
> > object for later steps will end up being wasted effort.
> 
> From what I understand this was what sequencer did before this
> series, so it is not a regression (I think; the commit parsing
> was in different function, but I think at the same place in
> the callchain).

Exactly.

> > Also, when the sequencer becomes one sequencer to rule them all, the
> > command set may contain something that does not even mention any
> > commit at all (think: exec).
> 
> The "exec" line is a bit of exception, all other rebase -i commands
> take commit as parameter.  It could always use NULL.

There is also "noop".

> > So I am not sure if we want a parsed commit there (I would not
> > object if we kept the texual object name read from the file,
> > though).  The "one sequencer to rule them all" may even have to say
> > "now give name ':1' to the result of the previous operation" in one
> > step and in another later step have an instruction "merge ':1'".
> > When that happens, you cannot even pre-populate the commit object
> > when the sequencer reads the file, as the commit has not yet been
> > created at that point.
> 
> True, --preserve-merges rebase is well, different.

It is mis-designed. And I can be that harsh because it was my design.

In the meantime I came up with a much better design, and implemented it as
a shell script on top of rebase -i. Since shell scripts run like slow
molasses, even more so on Windows, I have a loose plan to implement its
functionality as a new --recreate-merges option, and to deprecate
--preserve-merges when that new option works.

It needs to be a new option (not a --preserve-merges=v2) because it is a
totally different beast. For starters, it does not need its own code path
that overrides pick_one, as --preserve-merges does.

But I get way ahead of myself. First we need to get these last few bits
and pieces in place to accelerate (non --preserve-merges) rebase -i.

Ciao,
Dscho

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1510 bytes --]

Hi Junio,

On Wed, 31 Aug 2016, Junio C Hamano wrote:

> Jakub Narębski <jnareb@gmail.com> writes:
> 
> >>>> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
> >>>>  struct todo_item {
> >>>>  	enum todo_command command;
> >>>>  	struct commit *commit;
> >>>> +	const char *arg;
> >>>> +	int arg_len;
> >  
> >> I am not sure what the "commit" field of type "struct commit *" is
> >> for.  It is not needed until it is the commit's turn to be picked or
> >> reverted; if we end up stopping in the middle, parsing the commit
> >> object for later steps will end up being wasted effort.
> >
> > From what I understand this was what sequencer did before this
> > series, so it is not a regression (I think; the commit parsing
> > was in different function, but I think at the same place in
> > the callchain).
> 
> Yes, I agree with you and I didn't mean "this is a new bug" at all.
> It just is an indication that further refactoring after this step is
> needed, and it is likely to involve removal or modification of this
> field.

Not so. After you review the sequencer-i patches, you will see that it
makes tons of sense to have that "commit" field: "pick" is by far the most
common case, and it would not make sense at all to validate the todo
script, only to throw away the information we got, only to re-gain it
later in the sequencer.

Read: I strongly disagree with your cursory verdict that the "commit"
field should go.

Ciao,
Dscho

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3894 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> 
> > The rebase command sports a `--gpg-sign` option that is heeded by the
> > interactive rebase.
> 
> Should it be "sports" or "supports"?

Funny. I got a PR last week that wanted to fix a similar expression.

I really meant "to sport", as in "To display; to have as a notable
feature.". See https://en.wiktionary.org/wiki/sport#Verb

> > +static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
> 
> I know it is not your fault, but I wonder why this file uses
> snake_case_name, while all other use kebab-case-names.  That is,
> why it is gpg_sign_opt and not gpg-sign-opt.

Yes, you are correct: it is not my fault ;-)

> Sidenote: it's a pity api-quote.txt is just a placeholder for proper
> documentation (including sq_quotef()).  I also wonder why it is not
> named sq_quotef_buf() or strbuf_addf_sq().

Heh. I did not even bother to check the documentation, it is my long-time
habit to dive right into the code.

> > @@ -471,17 +487,20 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> >  
> >  	if (IS_REBASE_I()) {
> >  		env = read_author_script();
> > -		if (!env)
> > +		if (!env) {
> > +			const char *gpg_opt = gpg_sign_opt_quoted(opts);
> > +
> >  			return error("You have staged changes in your working "
> >  				"tree. If these changes are meant to be\n"
> >  				"squashed into the previous commit, run:\n\n"
> > -				"  git commit --amend $gpg_sign_opt_quoted\n\n"
> 
> How did this get expanded by error(), and why we want to replace
> it if it works?

It did not work. It was a place-holder waiting for this patch ;-)

> 
> > +				"  git commit --amend %s\n\n"
> >  				"If they are meant to go into a new commit, "
> >  				"run:\n\n"
> > -				"  git commit $gpg_sign_opt_quoted\n\n"
> > +				"  git commit %s\n\n"
> >  				"In both case, once you're done, continue "
> >  				"with:\n\n"
> > -				"  git rebase --continue\n");
> > +				"  git rebase --continue\n", gpg_opt, gpg_opt);
> 
> Instead of passing option twice, why not make use of %1$s (arg reordering),
> that is
> 
>   +				"  git commit --amend %1$s\n\n"
> [...]
>   +				"  git commit %1$s\n\n"

Cute. But would this not drive the l10ners insane?

> So shell quoting is required only for error output.

Indeed.

> > @@ -955,8 +974,27 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
> >  
> >  static int read_populate_opts(struct replay_opts *opts)
> >  {
> > -	if (IS_REBASE_I())
> > +	if (IS_REBASE_I()) {
> > +		struct strbuf buf = STRBUF_INIT;
> > +
> > +		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
> > +			if (buf.len && buf.buf[buf.len - 1] == '\n') {
> > +				if (--buf.len &&
> > +				    buf.buf[buf.len - 1] == '\r')
> > +					buf.len--;
> > +				buf.buf[buf.len] = '\0';
> > +			}
> 
> Isn't there some strbuf_chomp() / strbuf_strip_eof() function?
> Though as strbuf_getline() uses something similar...

Even worse. read_oneliner() *already* does that. I just forgot to delete
this code when I introduced and used read_oneliner().

Thanks.

> > +			if (!starts_with(buf.buf, "-S"))
> > +				strbuf_reset(&buf);
> 
> Should we signal that there was problem with a file contents?

Maybe. But probably not: this file is written by git-rebase itself. I
merely safe-guarded against empty files here.

> > +			else {
> > +				opts->gpg_sign = buf.buf + 2;
> > +				strbuf_detach(&buf, NULL);
> 
> Wouldn't we leak 2 characters that got skipped?  Maybe xstrdup would
> be better (if it is leaked, and not reattached)?

We do not leak anything because I changed the code locally already to use
sequencer_entrust() (I guess in response to an earlier of your comments).

Ciao,
Dscho

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 639 bytes --]

Hi Junio,

On Wed, 31 Aug 2016, Junio C Hamano wrote:

> Jakub Narębski <jnareb@gmail.com> writes:
> 
> >> +			else {
> >> +				opts->gpg_sign = buf.buf + 2;
> >> +				strbuf_detach(&buf, NULL);
> >
> > Wouldn't we leak 2 characters that got skipped?  Maybe xstrdup would
> > be better (if it is leaked, and not reattached)?
> 
> An attempt to avoid leaking by calling free(opts->gpg_sign) would
> make it crash, which would be even worse ;-).

As I pointed out in a couple of replies yesterday: we cannot assume that
gpg_sign is free()able. That's the entire reason behind the
sequencer_entrust() dance.

Ciao,
Dscho

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 271 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> Note that if the last line was
> 
>     +				sequencer_entrust(opts, strbuf_detach(&buf, NULL));
> 
> we can make it not leak.  A bit tricksy, though.

Heh... I made it that tricksy, then.

Ciao,
Dscho

Re: [PATCH 17/22] sequencer: allow editing the commit message on a case-by-case basis

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 999 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> 
> > In the upcoming commits, we will implement more and more of rebase
> > -i's functionality. One particular feature of the commands to come is
> > that some of them allow editing the commit message while others don't,
> > i.e. we cannot define in the replay_opts whether the commit message
> > should be edited or not.
> 
> It's a nice, pretty and self contained refactoring step.  Small
> enough that it is easy to review.
> 
> I would like to have in the commit message that it is sequencer_commit()
> function that needs to rely on new parameter, instead of on a property
> of command (of its replay_opts).  And that currently it simply passes
> the buck to caller, which uses opts->edit, but in the future the
> caller that is rebase -i would use todo_item and replay_opts based
> expression.

I enhanced the commit message.

Thanks for the review,
Dscho

Re: [PATCH 18/22] sequencer: support amending commits

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1643 bytes --]

Hi Kuba,

On Wed, 31 Aug 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> 
> > diff --git a/sequencer.c b/sequencer.c
> > index 7e17d14..20f7590 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -478,7 +478,7 @@ static char **read_author_script(void)
> >   * (except, of course, while running an interactive rebase).
> >   */
> >  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> > -			  int allow_empty, int edit)
> > +			  int allow_empty, int edit, int amend)
> 
> I guess we won't get much more parameters; it would get unwieldy
> (and hard to remember).  Five is all right.

It will eventually get a sixth, cleanup_commit_message.

> >  {
> >  	char **env = NULL;
> >  	struct argv_array array;
> > @@ -507,6 +507,8 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> >  	argv_array_push(&array, "commit");
> >  	argv_array_push(&array, "-n");
> >  
> > +	if (amend)
> > +		argv_array_push(&array, "--amend");
> >  	if (opts->gpg_sign)
> >  		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
> >  	if (opts->signoff)
> > @@ -779,7 +781,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  	}
> >  	if (!opts->no_commit)
> >  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> > -			opts, allow, opts->edit);
> > +			opts, allow, opts->edit, 0);
> 
> ... even of this makes one need to check the calling convention,
> what does this 0 mean.

Yeah, but it will not remain "0", but be replaced by a variable named
"amend"...

Thanks for the review,
Dscho

Re: [PATCH 19/22] sequencer: support cleaning up commit messages

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]

Hi Kuba,

On Thu, 1 Sep 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> 
> > @@ -781,7 +785,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  	}
> >  	if (!opts->no_commit)
> >  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> > -			opts, allow, opts->edit, 0);
> > +			opts, allow, opts->edit, 0, 0);
> 
> The calling convention begins to look unwieldy, but we have only
> a single such callsite, and there are quite a bit callsites in
> Git code that have similar API ("git grep ', 0, 0' -- '*.c'").
> So we don't need to think about alternatives.  Yet.

Right.

Please note that it will make much more sense in the end, too, as the 0s
will be replaced by appropriate variables.

> It's a pity that emulation of named parameters in C requires
> relying on designated inits from C99
> 
>   typedef struct {
>     double pressure, moles, temp;
>   } ideal_struct;
> 
>   #define ideal_pressure(...) ideal_pressure_base((ideal_struct){.pressure=1,   \
>                                         .moles=1, .temp=273.15, __VA_ARGS__})
> 
>   double ideal_pressure_base(ideal_struct in)
>   {
>     return 8.314 * in.moles*in.temp/in.pressure;
>   }
> 
>   ... ideal_pressure(.moles=2, .temp=373.15) ...

Yeah, that looks unwieldy ;-)

Thanks for the review,
Dscho

Re: [PATCH 21/22] sequencer: left-trim the lines read from the script

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1688 bytes --]

Hi Kuba,

On Thu, 1 Sep 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> 
> Subject: [PATCH 21/22] sequencer: left-trim the lines read from the script
> 
> In the subject, it should probably be without "the", as "lines"
> are plural.
> 
> s/left-trim the lines/left-trim lines/

I am happy that we stepped outside of the "code correctness" land into
"grammar fix" land, as it surely means that you are convinced the code is
correct? ;-)

Fixed.

> > Interactive rebase's scripts may be indented; We need to handle this
> > case, too, now that we prepare the sequencer to process interactive
> > rebases.
> 
> s/; We need/; we need/

Hrmpf. From http://grammar.ccc.commnet.edu/grammar/marks/colon.htm:

	There is some disagreement among writing reference manuals about
	when you should capitalize an independent clause following a
	colon. Most of the manuals advise that when you have more than one
	sentence in your explanation or when your sentence(s) is a formal
	quotation, a capital is a good idea. The NYPL Writer's Guide urges
	consistency within a document; the Chicago Manual of Style says
	you may begin an independent clause with a lowercase letter unless it's
	one of those two things (a quotation or more than one sentence).
	The APA Publication Manual is the most extreme: it advises us to
	always capitalize an independent clause following a colon. The advice
	given above is consistent with the Gregg Reference Manual.

Based on that, I think that a capital is the correct case here.

> 'bol' is beginning-of-line, isn't it (a complement to eol)?

Yep. How did you guess? :-)

Ciao,
Dscho

Re: [PATCH 22/22] sequencer: refactor write_message()

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3315 bytes --]

Hi Kuba,

On Thu, 1 Sep 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> 
> > The write_message() function safely writes an strbuf to a file.
> > Sometimes this is inconvenient, though: the text to be written may not
> > be stored in a strbuf, or the strbuf should not be released after
> > writing.
> 
> By "this" you mean "using strbuf", isn't it?  It is not very obvious,
> and I think it would be better to say it explicitly.

Rephrased.

> > Let's allow for such use cases by refactoring write_message() to allow
> > for a convenience function write_file_gently(). As some of the
> > upcoming callers of that new function will want to append a newline
> > character, let's just add a flag for that, too.
> 
> This paragraph feels a bit convoluted.
> 
> As I understand it, you refactor "safely writing string to a file"
> into write_with_lock_file(), and make write_message() use it.  The
> new function makes it easy to create new convenience function 
> write_file_gently(); as some of the upcoming callers of this new
> function would want to append a newline character, add a flag for
> it in write_file_gently(), and thus in write_with_lock_file().
> 
> Isn't it better / easier to understand?

I don't know, but I took it.

> > diff --git a/sequencer.c b/sequencer.c
> > index 5efed2e..f5b5e5e 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -239,22 +239,37 @@ static void print_advice(int show_hint, struct replay_opts *opts)
> >  	}
> >  }
> >  
> > -static int write_message(struct strbuf *msgbuf, const char *filename)
> > +static int write_with_lock_file(const char *filename,
> > +				const void *buf, size_t len, int append_eol)
> >  {
> >  	static struct lock_file msg_file;
> >  
> >  	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
> >  	if (msg_fd < 0)
> >  		return error_errno(_("Could not lock '%s'"), filename);
> > -	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
> > +	if (write_in_full(msg_fd, buf, len) < 0)
> >  		return error_errno(_("Could not write to %s"), filename);
> 
> You could have, for consistency, add quotes around filename (see previous
> error_errno callsite), *while at it*:
> 
>   		return error_errno(_("Could not write to '%s'"), filename);

Done.

> > -	strbuf_release(msgbuf);
> > +	if (append_eol && write(msg_fd, "\n", 1) < 0)
> > +		return error_errno(_("Could not write eol to %s"), filename);
> 
> Same here, and it wouldn't even be 'while at it'

Done.

>   +		return error_errno(_("Could not write eol to '%s'"), filename);
> 
> 
> >  	if (commit_lock_file(&msg_file) < 0)
> >  		return error(_("Error wrapping up %s."), filename);
> 
> Another "while at it"... though the one that can be safely postponed
> (well, the make message easier to understand part, not the quote
> filename part):
> 
>   		return error(_("Error wrapping up writing to '%s'."), filename);

As I inherited this message, I'll keep it.

> And thus we got to the last patch in this series.  I have skipped
> patches that already got reviewed; are there some that you would
> like to have second review of?  Is there patch series that needs
> to be applied earlier that needs a review?

Thank you for your review!
Dscho

Re: [PATCH 21/22] sequencer: left-trim the lines read from the script

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> > Interactive rebase's scripts may be indented; We need to handle this
>> > case, too, now that we prepare the sequencer to process interactive
>> > rebases.
>> 
>> s/; We need/; we need/
>
> Hrmpf. From http://grammar.ccc.commnet.edu/grammar/marks/colon.htm:
>
> 	There is some disagreement among writing reference manuals about
> 	when you should capitalize an independent clause following a
> 	colon. Most of the manuals advise that when you have more than one
> 	sentence in your explanation or when your sentence(s) is a formal
> 	quotation, a capital is a good idea. The NYPL Writer's Guide urges
> 	consistency within a document; the Chicago Manual of Style says
> 	you may begin an independent clause with a lowercase letter unless it's
> 	one of those two things (a quotation or more than one sentence).
> 	The APA Publication Manual is the most extreme: it advises us to
> 	always capitalize an independent clause following a colon. The advice
> 	given above is consistent with the Gregg Reference Manual.
>
> Based on that, I think that a capital is the correct case here.

Does that manual have anything to say about semicolons, which is a
different thing?

Re: [PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> git continue as a shorthand for `git <relevant-cmd> --continue` sounds great.
>
> Before we get ahead of ourselves:
>
> 1) this has nothing to do with the patch series at hand, and
>
> 2) if we were to introduce `git continue`, we would need to think long and
>    hard about the following issues:
>
> 	I) are there potentially ambiguous <relevant-cmd>s that the user
> 	   may want to continue?
>
> 	II) what about options? You can say `git rebase --continue
> 	    --no-ff`, for example, but not `git cherry-pick --continue
> 	    --no-ff`...
>
> 	III) Would it not be confusing to have a subcommand `continue`
> 	     that does *not* serve a *single* purpose? It's kinda flying
> 	     into the face of the Unix philosophy.

The above reasoning applies equally to "git abort".  I do not think
"git continue" would help.

If it were that anything you can do with Git can be --continue'ed
the same way (e.g. all uses one sequencer to rule them all), it
might be achievable, but I do not think it isn't, and will never be.

"git commit" may say "You haven't added anything yet" and refuse to
do anything.  Should "git continue" do "git commit -a" by noticing
that the last thing you tried to do was "git commit" and guess that
it is likely you wanted to commit all changes?  I think not.

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> though).  The "one sequencer to rule them all" may even have to say
>> "now give name ':1' to the result of the previous operation" in one
>> step and in another later step have an instruction "merge ':1'".
>> When that happens, you cannot even pre-populate the commit object
>> when the sequencer reads the file, as the commit has not yet been
>> created at that point.
>
> These considerations are pretty hypothetical. I would even place a bet
> that we will *never* have ":1" as names, not if I have anything to say...
> ;-)

If you can always work with pre-existing commit, then you can
validate all object references that appear in the instructions
upfront.

I was sort of expecting that, when you do the preserve-merges mode
of "rebase -i", you would need to jump around, doing "we have
reconstructed the side branch on a new 'onto', let's give the result
this temporary name ':1', and then switch to the trunk (which would
call for 'reset <commit>' instruction) and merge that thing (which
would be 'merge :1' or perhaps called 'pick :1')", and at that point
you no longer validate the object references upfront.

If you do not have to have such a "mark this point" and a "refer to
that point we previously marked", then I agree that you should be
able to pre-validate and keep the result in the structure.

Re: [PATCH 08/22] sequencer: remove overzealous assumption

[Jakub Narębski]

Hello Johannes,

W dniu 01.09.2016 o 10:01, Johannes Schindelin pisze:
> On Wed, 31 Aug 2016, Jakub Narębski wrote:
>> W dniu 31.08.2016 o 20:36, Johannes Schindelin pisze:
>>
>> I wonder: would 'git cherry-pick --continue' be able to finish
>> 'git revert', and vice versa, then?  Or 'git sequencer --continue'?
> 
> I just tested this, via
> 
> 	diff --git a/t/t3510-cherry-pick-sequence.sh
> 	b/t/t3510-cherry-pick-sequence.sh
> 	index 96c7640..085d8bc 100755
> 	--- a/t/t3510-cherry-pick-sequence.sh
> 	+++ b/t/t3510-cherry-pick-sequence.sh
> 	@@ -55,7 +55,7 @@ test_expect_success 'cherry-pick
> 	mid-cherry-pick-sequence' '
> 		git checkout HEAD foo &&
> 		git cherry-pick base &&
> 		git cherry-pick picked &&
> 	-       git cherry-pick --continue &&
> 	+       git revert --continue &&
> 		git diff --exit-code anotherpick
> 
> (Danger! Whitespace corrupted!!!)
> 
> It appears that this passes now.

I'm now not sure if it is such a great idea.  As was said somewhere else
in this thread, different sequencer-based commands sports different
options, and you can add options to the "git <command> --continue".
For example you can say "git cherry-pick --continue -x", but you
cannot say "git revert --continue -x", as '-x' is a cherry-pick only
option.  Or you can, theoretically, use "git am --continue --no-3way".

One option is to temporarily relax the test (test_expect_failure),
then fix it at the end.


BTW. how git-am uses sequencer?  I have seen "revert" etc., and "pick"
etc., but no git-am related constants or strings...

> Probably `git sequencer --continue` would work, too, if there was a `git
> sequencer`. :0)

Right.

> 
>>> On Wed, 31 Aug 2016, Jakub Narębski wrote: 
>>>> W dniu 29.08.2016 o 10:04, Johannes Schindelin pisze:
>>  
>>>>> diff --git a/t/t3510-cherry-pick-sequence.sh b/t/t3510-cherry-pick-sequence.sh
>>>>> index 7b7a89d..6465edf 100755
>>>>> --- a/t/t3510-cherry-pick-sequence.sh
>>>>> +++ b/t/t3510-cherry-pick-sequence.sh
>>>>> @@ -459,17 +459,6 @@ test_expect_success 'malformed instruction sheet 1' '
>>>>>  	test_expect_code 128 git cherry-pick --continue
>>>>>  '
>>>>>  
>>>>> -test_expect_success 'malformed instruction sheet 2' '
>>>>
>>>> Hmmm... the description is somewhat lacking (especially compared to
>>>> the rest of test), anyway.
>>>>
>>>> BTW. we should probably rename 'malformed instruction sheet 2'
>>>> to 'malformed instruction sheet' if there are no further such
>>>> tests after this removal, isn't it?
>>>
>>> No, we cannot rename it after this patch because the patch removes it ;-)
>>> (It is not a file name but really a label for a test case.)
>>
>> Ooops.  What I wanted to say that after removing the test case named
>> 'malformed instruction sheet 2' we should also rename *earlier* test
>> case from 'malformed instruction sheet 1' to 'malformed instruction sheet',
>> as it is now the only 'malformed instruction sheet *' test case.
> 
> Actually, you know, I completely missed the fact that there was a
> "malformed instruction sheet 3". I renumbered it.

Ooops.  I have missed it too, having looked only at the test after the
one removed (which is not about malformed instruction sheet).

Best,
-- 
Jakub Narębski

Re: [PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Jakub Narębski]

Hello Johannes,

W dniu 01.09.2016 o 09:49, Johannes Schindelin pisze:
> On Wed, 31 Aug 2016, Jakub Narębski wrote: 
>> W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:
[...]

>> Hmmm... commit_list is, as defined in commit.h, a linked list.
> 
> That is the most prominent reason why the rest is not a mindless
> conversion from commit_list to todo_list.
> 
> And we need todo_list as an array, because we need to be able to peek, or
> even move, backwards from the current command.
> 
>> Here todo_list uses growable array implementation of list.  Which
>> is I guess better on current CPU architecture, with slow memory,
>> limited-size caches, and adjacency prefetching.
> 
> That is not the reason that an array is used here. The array allows us
> much more flexibility.

It would be nice if this reasoning (behind the change from linked list
to growable array) was mentioned in appropriate commit message, and
perhaps also in the cover letter for the series.  It is IMVHO quite
important information (that you thought obvious).

> 
> One of the major performance improvements will come at the very end, for
> example: the reordering of the fixup!/squash! lines. And that would be a
> *major* pain to do if the todo_list were still a linked list.

Actually deletion from and insertion into single linked list are
not that hard, and O(1) after finding place, O(N) with finding
included.  Moving elements in array is O(N),... and arguably a bit
simpler - but at high level, with appropriate primitives, they are
about the same.

Yes, array is easier for permutation and reordering.

>>> +struct todo_item *append_todo(struct todo_list *todo_list)
>>
>> Errr... I don't quite understand the name of this function.
>> What are you appending here to the todo_list?
> 
> A new item.
> 
>> Compare string_list_append() and string_list_append_nodup(),
>> where the second parameter is item to append.
> 
> Yes, that is correct. In the case of a todo_item, things are a lot more
> complicated, though. Some of the values have to be determined tediously
> (such as the offset and length of the oneline after the "pick <oid>"
> command). I just put those values directly into the newly allocated item,
> is all.

I would expect sth_append command to take a list (or other collection),
an element, and return [modified] collection with the new element added.
Such API would require temporary variable in caller and memcopy in the
sth_append() function.

This is not it.  It creates a new element, expanding a list (a collection),
and then expose this element.  Which spares us memcopy... on non-critical
path.

I don't know how to name operation "grow list and return new element".
But "append" it is not.
 
>>> +	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
>>> +	return todo_list->items + todo_list->nr++;
>>>  }
>>>  
>>> -static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
>>> +static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>>
>> Why the change of return type?  
> 
> Because it makes no sense to return a commit here because not all commands
> are about commits (think rebase -i's `exec`). It makes tons of sense to
> return an error condition, though.

All right.

I have not checked / not remember what caller of parse_insn_line() used
it's return value for.  I guess that we save it into todo_item, moving
that operation from caller to callee.

>> Why now struct todo_item is first when struct replay_opts was last?
> 
> Those play very, very different roles.
> 
> The opts parameter used to provide parse_insn_line() with enough
> information to complain loudly when the overall command was not identical
> to the parsed command.
> 
> The item parameter is a receptacle for the parsed data. It will contain
> the pointer to the commit that was previously returned, if any. But it
> will also contain much more information, such as the command, the oneline,
> the offset in the buffer, etc etc
> 
> So "opts" was an "in" parameter while "item" is an "out" one. Apples and
> oranges.

All right.  Good explanation.  And the one that is too low-level
detail to put in the commit message, I think.

>>> -	end_of_object_name = bol + strcspn(bol, " \t\n");
>>> +	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
>>
>> Why is this cast needed?
> 
> Because bol is a "const char *" and we need to put "NUL" temporarily to
> *end_of_object_name:

Would compiler complain without this const'ness-stripping cast?

> 
>>>  	saved = *end_of_object_name;
>>>  	*end_of_object_name = '\0';
>>>  	status = get_sha1(bol, commit_sha1);
>>>  	*end_of_object_name = saved;
> 
> Technically, this would have made a fine excuse to teach get_sha1() a mode
> where it expects a length parameter instead of relying on a NUL-terminated
> string.
> 
> Practically, such fine excuses cost me months in this rebase--helper
> project already, and I need to protect my time better.

Put it in TODO list (and perhaps add a TODO comment) ;-).

BTW. open-source produces better software, and development bandwidth
is good, but the latency :-(((( 

> 
>>> -	/*
>>> -	 * Verify that the action matches up with the one in
>>> -	 * opts; we don't support arbitrary instructions
>>> -	 */
>>> -	if (action != opts->action) {
>>> -		if (action == REPLAY_REVERT)
>>> -		      error((opts->action == REPLAY_REVERT)
>>> -			    ? _("Cannot revert during another revert.")
>>
>> Errr... could the above ever happen?  Namely
>>
>>   action != opts->action && action == REPLAY_REVERT && opts->action == REPLAY_REVERT
>>
>> Surely not.
> 
> Your reply pointed to the very circumstance when this may happen: `git
> cherry-pick --continue` after an interrupted `git revert`.

I was talking about "Cannot revert during another revert." and
"Cannot cherry-pick during another cherry-pick." errors, which can
never happen because of the outermost if.

   x == A && y == A && x != Y

can never happen, because equality is transitive^*

   x == A && y == A  =>  x == y

*) except NaN (NaN != NaN), but we don't deal with floating point here.


From those error messages it looks like they were originally intended
to prevent from starting another revert or cherry-pick if sequencer
based operation is already in progress (to not stomp on the latter
internal state).  But for that it would need to examine the
opts->subcommand if it is REPLAY_NONE (or equivalent), check that
sequencer file already exists, and read it to fond which operation
is in progress.

This is about checking that command in todo-list agrees with
the git command used (we probably checked that it is --continue,
or maybe --skip).  But the error message does not spell that;
it is misleading.

[Nb. git-blame shows Vasco Almeida as author of those lines]

> 
> But then, I remove that code here, so I should not try to defend it.
> 

But that was / is another reason for removing code: it is slightly
wrong.

>>> -			    : _("Cannot revert during a cherry-pick."));
>>> -		else
>>> -		      error((opts->action == REPLAY_REVERT)
>>> -			    ? _("Cannot cherry-pick during a revert.")
>>> -			    : _("Cannot cherry-pick during another cherry-pick."));
>>> -		return NULL;
>>> -	}
>>
>> Anyway, while it is / would be a good idea to prevent starting any
>> sequencer-based command (cherry-pick, revert, soon rebase -i) when
>> other command is in progress (cherry-pick, revert, soon rebase -i).
>> That is, if cherry-pick / revert waits for user action, you cannot
>> run another cherry-pick or revert.
>>
>> Which I guess the above code was not about...
> 
> It was about that, though.

No it was not, see the reasoning above.

> 
> It went about it in a pretty round-about way: opts->action comes from the
> name of the command ("was I called as `git revert` or `git cherry-pick`?")
> and action comes from the todo script, which was assumed to be written by
> a previous run of the sequencer, using the then-current value of
> opts->action.

It did different check, that we continue with the same command
as we began with.

> 
> So it wrote that command into *every single line* of the todo script, *for
> the sole purpose* of verifying that it was the same action when running
> via --continue.
> 
> As I said earlier, I would not complain at all if an interrupted `git
> revert` could be continued via `git cherry-pick --continue`.
> 
> If that is not desirable, I can reintroduce that overzealous check, but
> that will have to wait until after v2.10.0. And it would require an
> argument that convinces me.

One argument is that you can add options to --continue, and among
options for cherry-pick, revert and rebase -i, there are options
that apply only to some of them.  So we need at least decide what
to do if we started rebase, and try to continue with cherry-pick:
do we accept rebase-only options?  What we do with cherry-pick-only
options, that rebase does not understand?

It is easier to just check that revert is continued with revert
(by checking command in todo file), cherry-pick with cherry-pick
(same), rebase -i with rebase (by checking another file), and
am with am.

> 
>>> +		item = append_todo(todo_list);
>>
>> A better name, in my personal option, would be
>>
>>   +		item = todo_list_next(todo_list);
>>
>> Or todo_next(todo_list).
> 
> That sounds more like a function that performs the next command in the
> todo_list.
> 
> While I agree that naming is hard, I still think that `append_todo()` with
> the todo_list as single parameter and returning a todo_item is pretty much
> self-explanatory: it appends a new item to the todo_list and returns a
> pointer to it.

It creates new item in the todo_list at the end, or it grows todo_list,
but the function does not append anything...

[...]
>>> -	if (strbuf_read(&buf, fd, 0) < 0) {
>>> +	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
>>>  		close(fd);
>>> -		strbuf_release(&buf);
>>
>> A question: when is todo_list->buf released?
> 
> Why, I am glad you asked! It is released in todo_list_release(), called at
> the end e.g. of sequencer_continue().

All right.  I could have guessed that.

>>> -static int walk_revs_populate_todo(struct commit_list **todo_list,
>>> +static int walk_revs_populate_todo(struct todo_list *todo_list,
>>>  				struct replay_opts *opts)
>>>  {
>>> +	enum todo_command command = opts->action == REPLAY_PICK ?
>>> +		TODO_PICK : TODO_REVERT;
>>>  	struct commit *commit;
>>> -	struct commit_list **next;
>>>  
>>>  	if (prepare_revs(opts))
>>>  		return -1;
>>>  
>>> -	next = todo_list;
>>> -	while ((commit = get_revision(opts->revs)))
>>> -		next = commit_list_append(commit, next);
>>> +	while ((commit = get_revision(opts->revs))) {
>>> +		struct todo_item *item = append_todo(todo_list);
>>> +		const char *commit_buffer = get_commit_buffer(commit, NULL);
>>
>> I see that you are creating todo file contents while walking revision list,
>> something that was left for later in current / previous implementation
>> of the sequencer...
> 
> Not really. This function was always about generating a todo_list. It just
> did not format it yet.
> 
> With the change of keeping the original formatting of the todo script
> instead of re-formatting it in save_todo(), this function now has to
> format the todo_list itself.

So the commit parsing was moved from save_todo(), which contrary to
the name also re-generated todo list, to walk_revs_populate_todo().
I guess it is the same callchain.

Additional question, answer to which should address Junio's complaint
about early/eager commit parsing (if I understand it correctly):
do information from parsing commit is needed for creating a new
commit, i.e. doing a pick or revert?  If not, then perhaps parsing
only those commits that are left, at the time of saving todo file
(with probably less commit that what we have started with), would
be better?

[...]
>>> -static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
>>> +static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
>>>  {
>>>  	static struct lock_file todo_lock;
>>> -	struct strbuf buf = STRBUF_INIT;
>>> -	int fd;
>>> +	const char *todo_path = get_todo_path(opts);
>>> +	int next = todo_list->current, offset, fd;
>>
>> The "next = todo_list->current" looks a bit strange.
> 
> Depending whether we need rebase -i processing or revert/cherry-pick's
> slightly different one, the "current" position points to the next one
> already...
> 
>> Also, we do not change todo_list->current, we use it in one place, so it
>> can be used directly without help of temporary / helper variable.  But
>> that is just my personal opinion.
> 
> No, it has nothing to do with opinion. It prepares the code to keep it
> readable even when REPLAY_INTERACTIVE_REBASE is introduced.

Ah, all right.

[...]
>> and this should be done in one of earlier patches, isn't it?
> 
> No. I deliberately skipped save_todo() from "future-proofing" as I planned
> to rewrite it anyway. There is no point in future-proofing something you
> are going to toss in a minute.
> 

All right.  Though I wonder if it should not be mentioned in
the commit message of said previous patch (though I have missed
that not all sites were "future-proofed" in review; I'm sorry).

>>> -	if (commit_lock_file(&todo_lock) < 0) {
>>> -		strbuf_release(&buf);
>>> -		return error(_("Error wrapping up %s."), git_path_todo_file());
>>> -	}
>>> -	strbuf_release(&buf);
>>> +	if (commit_lock_file(&todo_lock) < 0)
>>> +		return error(_("Error wrapping up %s."), todo_path);
>>
>> Note: this is unrelated change, but we usually put paths in quotes, like this
>>
>>   +		return error(_("Error wrapping up '%s'."), todo_path);
>>
>> (in this and earlier error message), so that paths containing spaces show
>> correctly and readably to the user.  Though this possibly is not a problem
>> for this path.
> 
> Right.
> 
>> Also, how user is to understand "wrapping up"?
> 
> The same as before: the removed lines already had the error message,
> missing the quotes, too.
> 
> Don't get me wrong: I am a big fan of consistency, and I wish that Git's
> source code had more of it. So I would love to see a patch series that
> makes all error messages consistently reporting paths enclosed in single
> quotes.
> 
> I am also a big fan of the separation of concerns, though. And this patch
> series' concern is consistency *with the existing code*.
> 
> So I won't change the error message that I inherited at this point.

All right, I can understand that.  But I think it won't cost
much to do _while at it_ adding of quotes around pathnames in
error messages, where you notice this problem.
 
>>>  static int single_pick(struct commit *cmit, struct replay_opts *opts)
>>>  {
>>>  	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
>>> -	return do_pick_commit(cmit, opts);
>>> +	return do_pick_commit(opts->action == REPLAY_PICK ?
>>> +		TODO_PICK : TODO_REVERT, cmit, opts);
>>
>> The ternary conditional operator here translates one enum to other enum,
>> isn't it?
> 
> Well, almost. Please note that the enum will receive a new value in the
> sequencer-i patch series. And there is no equivalent todo_command for
> REPLAY_INTERACTIVE_REBASE.

All right.  And casting one enum to other, relying on the same
order, is tricky to the extreme and brittle.

> 
> Thanks for the review!

You are welcome.

Best,
-- 
Jakub Narębski

Re: [PATCH 10/22] sequencer: avoid completely different messages for different actions

[Jakub Narębski]

Hello Johannes

W dniu 01.09.2016 o 09:52, Johannes Schindelin pisze:
> On Wed, 31 Aug 2016, Jakub Narębski wrote:
>> CC-ed to Jiang Xin, L10N coordinator.
>> W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:

[...]
>>> -	/* Different translation strings for cherry-pick and revert */
>>> -	if (opts->action == REPLAY_PICK)
>>> -		error(_("Your local changes would be overwritten by cherry-pick."));
>>> -	else
>>> -		error(_("Your local changes would be overwritten by revert."));
>>> +	error(_("Your local changes would be overwritten by %s."),
>>> +		action_name(opts));
>>
>> If I understand it correctly, it would make "revert" or "cherry-pick"
>> untranslated part of error message.  You would need to use translation
>> on the result with "_(action_name(opts))", you would have to mark
>> todo_command_strings elements for gettext lexicon with N_(...).
>>
>> I am rather against this change (see also below).
> 
> Okay.
> 
> Unfortunately, I have to focus on the correctness of the code at the
> moment (and Git for Windows does ship *without* translations for the time
> being anyway, mostly to save on space, but also because users complained).

Users complained about having translations, or not having easy way to
switch them or switch them off?

> 
> So I will take care of this after v2.10.0.
> 
> For the record, how is this supposed to be handled, in particular when I
> introduce a new action whose action_name(opts) will be "rebase -i"? Do I
> really need to repeat myself three times?

I think you should be able to mark strings to be translated,
without translating them at the time of definition,

  static const char *todo_command_strings[] = {
  	N_("pick"),
  	N_("revert")
  };

then translate at the point of use

  	error(_("Your local changes would be overwritten by %s."),
		_(action_name(opts)));

I assume that action_name(opts) returns one of todo_command_strings.
If not, there should be array with possible actions.


Assuming that such lego l10n is preferable to multiple translations,
more free-formt.

-- 
Jakub Narębski

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Jakub Narębski]

W dniu 01.09.2016 o 11:37, Johannes Schindelin pisze:
> On Wed, 31 Aug 2016, Junio C Hamano wrote:
>> Jakub Narębski <jnareb@gmail.com> writes:
>>
>>>> diff --git a/sequencer.c b/sequencer.c
>>>> index 06759d4..3398774 100644
>>>> --- a/sequencer.c
>>>> +++ b/sequencer.c
>>>> @@ -709,6 +709,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>>>>  struct todo_item {
>>>>  	enum todo_command command;
>>>>  	struct commit *commit;
>>>> +	const char *arg;
>>>> +	int arg_len;
>>>
>>> Why 'arg', and not 'oneline', or 'subject'?
>>> I'm not saying it is bad name.
>>
>> I am not sure what the "commit" field of type "struct commit *" is
>> for.  It is not needed until it is the commit's turn to be picked or
>> reverted; if we end up stopping in the middle, parsing the commit
>> object for later steps will end up being wasted effort.
> 
> No, it won't be wasted effort, as we *validate* the todo script this way.
> And since we may very well need the info later (most rebases do not fail
> in the middle), we store it, too.

The question was (I think) whether we should do eager parsing of
commits, or whether we can do lazy parsing by postponing full parsing
"until it is the commit's turn to be picked or reverted", and possibly
when saving todo file.

I wonder how probable is situation where we save instruction sheet
for interactive rebase, with shortened SHA-1, and during rebase
shortened SHA-1 stops being unambiguous...

>                                [...] after parsing the todo_list, we will
> have to act on the information contained therein. For example we will have
> to cherry-pick some of the indicated commits (requiring a struct commit *
> for use in do_pick_commit()). Another example: we may need to determine
> the oneline for use in fixup!/squash! reordering.
> 
> So: keeping *that* aspect of the previous todo_list parsing, i.e. store a
> pointer to the already-parsed commit, is the right thing to do.

The above probably means that eager eval is better

Best,
-- 
Jakub Narębski

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Jakub Narębski]

Hello,

W dniu 01.09.2016 o 15:12, Johannes Schindelin pisze:
> On Wed, 31 Aug 2016, Jakub Narębski wrote:
>> W dniu 31.08.2016 o 21:10, Junio C Hamano pisze:

>>> So I am not sure if we want a parsed commit there (I would not
>>> object if we kept the texual object name read from the file,
>>> though).  The "one sequencer to rule them all" may even have to say
>>> "now give name ':1' to the result of the previous operation" in one
>>> step and in another later step have an instruction "merge ':1'".
>>> When that happens, you cannot even pre-populate the commit object
>>> when the sequencer reads the file, as the commit has not yet been
>>> created at that point.
>>
>> True, --preserve-merges rebase is well, different.
> 
> It is mis-designed. And I can be that harsh because it was my design.
> 
> In the meantime I came up with a much better design, and implemented it as
> a shell script on top of rebase -i. Since shell scripts run like slow
> molasses, even more so on Windows, I have a loose plan to implement its
> functionality as a new --recreate-merges option, and to deprecate
> --preserve-merges when that new option works.
> 
> It needs to be a new option (not a --preserve-merges=v2) because it is a
> totally different beast. For starters, it does not need its own code path
> that overrides pick_one, as --preserve-merges does.

Better preserving for merges (with cleanly defined sematics)
would be certainly nice to have.

> But I get way ahead of myself. First we need to get these last few bits
> and pieces in place to accelerate (non --preserve-merges) rebase -i.

But it can wait, right.

-- 
Jakub Narębski

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

> I wonder how probable is situation where we save instruction sheet
> for interactive rebase, with shortened SHA-1, and during rebase
> shortened SHA-1 stops being unambiguous...

It is my understanding that the shortened ones are only for end-user
consumption.  The insn sheet internally uses fully expanded form for
this exact reason, and then abbreviated back at each step before the
updated one is presented to the end-user.  Uniqueness guarantee is
enforced with new objects created during each step taken into
account by doing it this way.

Re: [PATCH 16/22] sequencer: prepare for rebase -i's GPG settings

[Jakub Narębski]

Hello Johannes,

W dniu 01.09.2016 o 15:33, Johannes Schindelin pisze:
> On Wed, 31 Aug 2016, Jakub Narębski wrote:
>> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

>>> @@ -471,17 +487,20 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
>>>  
>>>  	if (IS_REBASE_I()) {
>>>  		env = read_author_script();
>>> -		if (!env)
>>> +		if (!env) {
>>> +			const char *gpg_opt = gpg_sign_opt_quoted(opts);
>>> +
>>>  			return error("You have staged changes in your working "
>>>  				"tree. If these changes are meant to be\n"
>>>  				"squashed into the previous commit, run:\n\n"
>>> -				"  git commit --amend $gpg_sign_opt_quoted\n\n"
>>
>> How did this get expanded by error(), and why we want to replace
>> it if it works?

After writing this email, I got an idea on how it could work:
git-rebase script calls some C helper, which outputs above, and
output of this helper is eval'ed by script (with gpg_sign_opt_quoted
variable present in the environment)...

> 
> It did not work. It was a place-holder waiting for this patch ;-)
> 

... but it might have been simply copy'n'pasted from shell script
to C, literally.

>>
>>> +				"  git commit --amend %s\n\n"
>>>  				"If they are meant to go into a new commit, "
>>>  				"run:\n\n"
>>> -				"  git commit $gpg_sign_opt_quoted\n\n"
>>> +				"  git commit %s\n\n"
>>>  				"In both case, once you're done, continue "
>>>  				"with:\n\n"
>>> -				"  git rebase --continue\n");
>>> +				"  git rebase --continue\n", gpg_opt, gpg_opt);
>>
>> Instead of passing option twice, why not make use of %1$s (arg reordering),
>> that is
>>
>>   +				"  git commit --amend %1$s\n\n"
>> [...]
>>   +				"  git commit %1$s\n\n"
> 
> Cute. But would this not drive the l10ners insane?
> 

Shouldn't, as l10ners need to deal with arg reordering, because in different
languages the order of words might be different: %s %s in English may be
%2$s %1$s in other language, see example in
  https://www.gnu.org/software/gettext/manual/gettext.html#c_002dformat-Flag

Best,
-- 
Jakub Narębski

Re: [PATCH 19/22] sequencer: support cleaning up commit messages

[Jakub Narębski]

Hello Johannes,

W dniu 01.09.2016 o 15:56, Johannes Schindelin pisze: 
> On Thu, 1 Sep 2016, Jakub Narębski wrote:

>> It's a pity that emulation of named parameters in C requires
>> relying on designated inits from C99
>>
>>   typedef struct {
>>     double pressure, moles, temp;
>>   } ideal_struct;
>>
>>   #define ideal_pressure(...) ideal_pressure_base((ideal_struct){.pressure=1,   \
>>                                         .moles=1, .temp=273.15, __VA_ARGS__})
>>
>>   double ideal_pressure_base(ideal_struct in)
>>   {
>>     return 8.314 * in.moles*in.temp/in.pressure;
>>   }
>>
>>   ... ideal_pressure(.moles=2, .temp=373.15) ...

Forgot to add citation:

[1] Ben Klemens "21st Century C: C Tips from the New School", 2nd Ed. (2014),
    O'Reilly Media, chapter 10. "Better Structures", subsection
    "Optional and Named Arguments"

> 
> Yeah, that looks unwieldy ;-)
>

Declaration needs some trickery, but use is much, much more readable
(if we cannot use sensibly named variables for passing arguments):

  ideal_pressure()
  ideal_pressure(.temp=373.15)
  ideal_pressure(.moles=2)
  ideal_pressure(.moles=2, .temp=373.15) 

It is even better if there are large amount of parameters:

  res = amortization(.amount=200000, .inflation=3,
                     .show_table=0, .extra_payoff=100)

vs

  double amortize(double amt, double rate, double inflation, int months,
            int selloff_month, double extra_payoff, int verbose,
            double *interest_pv, double *duration, double *monthly_payment);

 
But we can't use it in Git, anyway
-- 
Jakub Narębski

Re: [PATCH 21/22] sequencer: left-trim the lines read from the script

[Jakub Narębski]

Hello Johannes,

W dniu 01.09.2016 o 16:13, Johannes Schindelin pisze: 
> On Thu, 1 Sep 2016, Jakub Narębski wrote:
 
>> 'bol' is beginning-of-line, isn't it (a complement to eol)?
> 
> Yep. How did you guess? :-)

Wouldn't 'beg' and 'end' instead of 'bol' and 'eol' be easier
to understand, thus more readable?

-- 
Jakub Narębski

Re: [PATCH 22/22] sequencer: refactor write_message()

[Jakub Narębski]

Hello Johannes,

W dniu 01.09.2016 o 16:20, Johannes Schindelin pisze:
> On Thu, 1 Sep 2016, Jakub Narębski wrote: 
>> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:

>>>  	if (commit_lock_file(&msg_file) < 0)
>>>  		return error(_("Error wrapping up %s."), filename);
>>
>> Another "while at it"... though the one that can be safely postponed
>> (well, the make message easier to understand part, not the quote
>> filename part):
>>
>>   		return error(_("Error wrapping up writing to '%s'."), filename);
> 
> As I inherited this message, I'll keep it.

Well, please then add quotes while at it, at least, for consistency

  		return error(_("Error wrapping up '%s'."), filename);

Best,
-- 
Jakub Narębski

Re: [PATCH 00/22] Prepare the sequencer for the upcoming rebase -i patches

[Jakub Narębski]

W dniu 29.08.2016 o 10:03, Johannes Schindelin pisze:

> This patch series marks the  '4' in the countdown to speed up rebase -i
> by implementing large parts in C. It is based on the `libify-sequencer`
> patch series that I submitted last week.

Which of those got reviewed (and perhaps accepted), and which of those
needs review still?  What is subject of their cover letter?

> 
> The patches in this series merely prepare the sequencer code for the
> next patch series that actually teaches the sequencer to run an
> interactive rebase.
> 
> The reason to split these two patch series is simple: to keep them at a
> sensible size.

That's good.

> 
> The two patch series after that are much smaller: a two-patch "series"
> that switches rebase -i to use the sequencer (except with --root or
> --preserve-merges), and a couple of patches to move several pretty
> expensive script processing steps to C (think: autosquash).

I can understand --preserve-merges, but what is the problem with --root?

> 
> The end game of this patch series is a git-rebase--helper that makes
> rebase -i 5x faster on Windows (according to t/perf/p3404). Travis says
> that even MacOSX and Linux benefit (4x and 3x, respectively).

So do I understand correctly that end goal for *this* series is to move
most of processing to git-rebase--helper, but full builtin-ification
(and retiring git-rebase.sh to contrib/examples/) would have to wait
for later?

[...]

I'd like here to summarize the discussion (my review, Dennis review,
Johannes Sixt and Junio comments).

If there are no comments, it means no problems or minor changes.

> Johannes Schindelin (22):
>   sequencer: use static initializers for replay_opts
There is no need for putting zeros in static initializer.  Commit
message expanded.

>   sequencer: use memoized sequencer directory path
>   sequencer: avoid unnecessary indirection
>   sequencer: future-proof remove_sequencer_state()
Leftover unrelated chunk removed.

>   sequencer: allow the sequencer to take custody of malloc()ed data
Is introducing new *_entrust() mechanism (which needs docs, at least
as comments) worth it, instead of just strdup everything and free?
If it is: naming of function parameter + example in commit message.

>   sequencer: release memory that was allocated when reading options
See above.

>   sequencer: future-proof read_populate_todo()
Possibly mention which functions were not future-proofed because
of planned for the subsequent patch full rewrite.

>   sequencer: remove overzealous assumption
Overzealous assumptions, or a worthy check?  Perhaps just remove check
for rebase -i in future commit, and keep test.  Perhaps remove test
temporarily.

>   sequencer: completely revamp the "todo" script parsing
This removes check; it should return if it was worthy.  Some discussion
about eager versus lazy parsing of commits, but IMHO it should be left
for later, if considered worth it.

>   sequencer: avoid completely different messages for different actions
Fix l10n or drop (and not introduce lego translation).

>   sequencer: get rid of the subcommand field
>   sequencer: refactor the code to obtain a short commit name
Explain reason behind this change in the commit mesage.

>   sequencer: remember the onelines when parsing the todo file
Lazy or eager again; "exec", "noop" and --preserve-merges.

>   sequencer: prepare for rebase -i's commit functionality
Add helper function, possibly extract helper function.  Rephrase block
comment.

"[PATCH] am: refactor read_author_script()" from Junio.

>   sequencer: introduce a helper to read files written by scripts
Perhaps add why not use open + strbuf_getline to commit message...

>   sequencer: prepare for rebase -i's GPG settings
Possibly fixes bug.  Use *_entrust() or strdup to not leak memory
(and to not crash when freeing memory).

>   sequencer: allow editing the commit message on a case-by-case basis
Enhance the commit message.

>   sequencer: support amending commits
>   sequencer: support cleaning up commit messages
>   sequencer: remember do_recursive_merge()'s return value
>   sequencer: left-trim the lines read from the script
>   sequencer: refactor write_message()
Enhance the commit message.  Quote path in messages while at it.


HTH

> 
>  builtin/commit.c                |   2 +-
>  builtin/revert.c                |  42 ++-
>  sequencer.c                     | 573 +++++++++++++++++++++++++++-------------
>  sequencer.h                     |  27 +-
>  t/t3510-cherry-pick-sequence.sh |  11 -
>  5 files changed, 428 insertions(+), 227 deletions(-)
> 
> Based-On: libify-sequencer at https://github.com/dscho/git
> Fetch-Base-Via: git fetch https://github.com/dscho/git libify-sequencer
> Published-As: https://github.com/dscho/git/releases/tag/prepare-sequencer-v1
> Fetch-It-Via: git fetch https://github.com/dscho/git prepare-sequencer-v1

An unrelated question: Dscho, how are you generating above lines?

-- 
Jakub Narębski
 

Re: [PATCH 00/22] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 11176 bytes --]

Hi Kuba,

On Fri, 2 Sep 2016, Jakub Narębski wrote:

> W dniu 29.08.2016 o 10:03, Johannes Schindelin pisze:
> 
> > This patch series marks the  '4' in the countdown to speed up rebase -i
> > by implementing large parts in C. It is based on the `libify-sequencer`
> > patch series that I submitted last week.
> 
> Which of those got reviewed (and perhaps accepted), and which of those
> needs review still?  What is subject of their cover letter?

Most of the patch series I sent before last week got accepted. Only one
got rejected, IIRC, and replaced by a better solution (3727318 (Merge
branch 'jk/test-send-sh-x-trace-elsewhere', 2016-05-17)).

The patch series I submitted as part of my rebase--helper work that were
accepted:

b232439 (Merge branch 'js/t3404-typofix', 2016-05-17)
7b02771 (Merge branch 'js/perf-rebase-i', 2016-05-23)
3437017 (Merge branch 'js/perf-on-apple', 2016-07-06)
62e5e83 (Merge branch 'js/find-commit-subject-ignore-leading-blanks', 2016-07-11)
c510926 (Merge branch 'js/sign-empty-commit-fix', 2016-07-13)
6c35952 (Merge branch 'js/t3404-grammo-fix', 2016-07-13)
63641fb (Merge branch 'js/log-to-diffopt-file', 2016-07-19)
3d55eea (Merge branch 'js/am-call-theirs-theirs-in-fallback-3way', 2016-07-19)
c97268c (Merge branch 'js/rebase-i-tests', 2016-07-28)
1a5f1a3 (Merge branch 'js/am-3-merge-recursive-direct', 2016-08-10)

You will note that I broke out a couple of patch series that do not
strictly have anything to do with the rebase--helper, such as
perf-on-apple. Nevertheless, they were part of a 99-strong patch series
that was my initial working rebase--helper, which I have used ever since
to perform all of my interactive rebases.

There are still a couple of patch series in flight. Let me list them by
the tags created by my mail-patch-series.sh script:

https://github.com/dscho/git/releases/tag/libify-sequencer-v2
https://github.com/dscho/git/releases/tag/require-clean-work-tree-v1
https://github.com/dscho/git/releases/tag/prepare-sequencer-v1
https://github.com/dscho/git/releases/tag/sequencer-i-v1
https://github.com/dscho/git/releases/tag/rebase--helper-v1

These tags all contain links to the cover letter as stored on
public-inbox.org, identified by the Message-ID.

Please note that the first four of this batch of five already saw
substantial work-after-review, thanks in part to your helpful comments.
You may appreciate the fact that a link of the form

https://github.com/dscho/git/compare/libify-sequencer-v2...libify-sequencer

shows you where I am at, although it cannot give you a real interdiff
because I rebased to a newer version of upstream/master in the meantime.

Finally, there is one last patch series that I did not yet submit: the
'rebase-i-extra' patch series. However, as I continuously update the
overall 'interactive-rebase' branch thicket (and have done so since the
very beginning of my work on the rebase--helper), it is relatively easy to
see what is left:

https://github.com/dscho/git/compare/rebase--helper...interactive-rebase

BTW thanks for making me dig out all of this information (it did take a
while to uncover it...), as I am so totally going to use that in a blog
post.

> > The reason to split these two patch series is simple: to keep them at a
> > sensible size.
> 
> That's good.

Thanks. I really try to be sensible with other people's time.

Even more so after being so offended by the talk at the most recent Git
Merge that stated that some people deliberately waste contributors' time
because they value their own time so much more. I am *really* offended by
that.

As a maintainer of Git for Windows, I do everything in my power to strike
a sensible balance between how much time I spend on improving the software
and how much time I ask others to do so.

> > The two patch series after that are much smaller: a two-patch "series"
> > that switches rebase -i to use the sequencer (except with --root or
> > --preserve-merges), and a couple of patches to move several pretty
> > expensive script processing steps to C (think: autosquash).
> 
> I can understand --preserve-merges, but what is the problem with --root?

The problem with --root is that it *creates* an initial commit. It is
empty, and will be amended. It would most likely not be a lot of work, but
I really wanted this work to be incremental, focusing on the most
important aspects first.

In fact, I do hope that somebody with the need for --root will take the
baton and run with it.

> > The end game of this patch series is a git-rebase--helper that makes
> > rebase -i 5x faster on Windows (according to t/perf/p3404). Travis
> > says that even MacOSX and Linux benefit (4x and 3x, respectively).
> 
> So do I understand correctly that end goal for *this* series is to move
> most of processing to git-rebase--helper, but full builtin-ification
> (and retiring git-rebase.sh to contrib/examples/) would have to wait for
> later?

Oh yes!

Retiring git-rebase.sh is *far, far, far* in the future. We really missed
the boat a *looooong* time ago to turn this from a hacky shell script into
a proper C builtin.

There is so much more to do before git-rebase.sh can be retired.

For starters, git-rebase.sh is actually just a glorified command-line
option parser and front-end to git-rebase--am.sh,
git-rebase--interactive.sh and git-rebase--merge.sh.

To retire it, those three shell scripts need to be *completely* built-in
first.

(Actually, for the --preserve-merges case, I could imagine that we simply
refactor it into its own shell script and call that from a builtin
git-rebase, until we retire --preserve-merges, but that's a couple of
years down the road.)

So the first goal would be to retire git-rebase--interactive.sh. For that
to happen, --root needs to be supported first. Then the --preserve-merges
stuff needs to be refactored into its own shell script. And then the
command-line option parsing needs to be moved to rebase--helper, too. And
*then* git-rebase--interactive.sh can be retired.

As I stated earlier, my hope is that the rebase--helper work is only an
initial step, opening the door for other contributors to tackle
independent parts of making git-rebase a builtin.

> [...]
> 
> I'd like here to summarize the discussion (my review, Dennis review,
> Johannes Sixt and Junio comments).
> 
> If there are no comments, it means no problems or minor changes.

Please keep in mind that my current state (local, and pushed to my GitHub
repository) has advanced substantially. I am reluctant to send it out yet
because I still need to send out rebase-i-extra first, so that it gets
*some* visibility before v2.10.0.

> > Johannes Schindelin (22):
> >   sequencer: use static initializers for replay_opts
> There is no need for putting zeros in static initializer.  Commit
> message expanded.
> 
> >   sequencer: use memoized sequencer directory path
> >   sequencer: avoid unnecessary indirection
> >   sequencer: future-proof remove_sequencer_state()
> Leftover unrelated chunk removed.
> 
> >   sequencer: allow the sequencer to take custody of malloc()ed data
> Is introducing new *_entrust() mechanism (which needs docs, at least
> as comments) worth it, instead of just strdup everything and free?
> If it is: naming of function parameter + example in commit message.
> 
> >   sequencer: release memory that was allocated when reading options
> See above.
> 
> >   sequencer: future-proof read_populate_todo()
> Possibly mention which functions were not future-proofed because
> of planned for the subsequent patch full rewrite.

Note that this commit is about read_populate_todo(), not about
save_todo(). So I do not think that we should mention anything in this
commit's message about other functions that may be rewritten instead of
being future-proofed..

> >   sequencer: remove overzealous assumption
> Overzealous assumptions, or a worthy check?  Perhaps just remove check
> for rebase -i in future commit, and keep test.  Perhaps remove test
> temporarily.

As mentioned earlier, I bit the bullet and reimplemented that logic.
Mostly to fend off more comments in this direction.

> >   sequencer: completely revamp the "todo" script parsing
> This removes check; it should return if it was worthy.  Some discussion
> about eager versus lazy parsing of commits, but IMHO it should be left
> for later, if considered worth it.

Again, it was reintroduced. The test to check for the overzealous
assumption was not removed, and it passes, to prove that I did it right.

> >   sequencer: avoid completely different messages for different actions
> Fix l10n or drop (and not introduce lego translation).
> 
> >   sequencer: get rid of the subcommand field
> >   sequencer: refactor the code to obtain a short commit name
> Explain reason behind this change in the commit mesage.
> 
> >   sequencer: remember the onelines when parsing the todo file
> Lazy or eager again; "exec", "noop" and --preserve-merges.
> 
> >   sequencer: prepare for rebase -i's commit functionality
> Add helper function, possibly extract helper function.  Rephrase block
> comment.
> 
> "[PATCH] am: refactor read_author_script()" from Junio.
> 
> >   sequencer: introduce a helper to read files written by scripts
> Perhaps add why not use open + strbuf_getline to commit message...
> 
> >   sequencer: prepare for rebase -i's GPG settings
> Possibly fixes bug.  Use *_entrust() or strdup to not leak memory
> (and to not crash when freeing memory).
> 
> >   sequencer: allow editing the commit message on a case-by-case basis
> Enhance the commit message.
> 
> >   sequencer: support amending commits
> >   sequencer: support cleaning up commit messages
> >   sequencer: remember do_recursive_merge()'s return value
> >   sequencer: left-trim the lines read from the script
> >   sequencer: refactor write_message()
> Enhance the commit message.  Quote path in messages while at it.

Apart from the l10n issues, I think I addressed them all locally, and
pushed the result out to my GitHub repository, although I plan to send out
additional iterations only after releasing Git for Windows v2.10.0.

> > Based-On: libify-sequencer at https://github.com/dscho/git
> > Fetch-Base-Via: git fetch https://github.com/dscho/git libify-sequencer
> > Published-As: https://github.com/dscho/git/releases/tag/prepare-sequencer-v1
> > Fetch-It-Via: git fetch https://github.com/dscho/git prepare-sequencer-v1
> 
> An unrelated question: Dscho, how are you generating above lines?

It's the `mail-patch-series.sh` script, in conjunction with setting the
config variable mail.publishremoteto to point to my GitHub remote. You can
find the `mail-patch-series.sh` script here:

	https://github.com/dscho/mail-patch-series

I probably forgot to adjust the README to reflect the most recent changes
(such as the `--basedon` feature)... PRs welcome [*1*].

Ciao,
Dscho

Footnote *1*:
https://raw.githubusercontent.com/dscho/images/master/i-can-haz-pull-request.png

Re: [PATCH 09/22] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3209 bytes --]

Hi Kuba,

On Fri, 2 Sep 2016, Jakub Narębski wrote:

> W dniu 01.09.2016 o 09:49, Johannes Schindelin pisze:
> > On Wed, 31 Aug 2016, Jakub Narębski wrote: 
> 
> >> Here todo_list uses growable array implementation of list.  Which is
> >> I guess better on current CPU architecture, with slow memory,
> >> limited-size caches, and adjacency prefetching.
> > 
> > That is not the reason that an array is used here. The array allows us
> > much more flexibility.
> 
> It would be nice if this reasoning (behind the change from linked list
> to growable array) was mentioned in appropriate commit message, and
> perhaps also in the cover letter for the series.  It is IMVHO quite
> important information (that you thought obvious).

Amended.

> >>> +struct todo_item *append_todo(struct todo_list *todo_list)
> >>
> >> Errr... I don't quite understand the name of this function.
> >> What are you appending here to the todo_list?
> > 
> > A new item.
> > 
> >> Compare string_list_append() and string_list_append_nodup(),
> >> where the second parameter is item to append.
> > 
> > Yes, that is correct. In the case of a todo_item, things are a lot more
> > complicated, though. Some of the values have to be determined tediously
> > (such as the offset and length of the oneline after the "pick <oid>"
> > command). I just put those values directly into the newly allocated item,
> > is all.
> 
> I would expect sth_append command to take a list (or other collection),
> an element, and return [modified] collection with the new element added.
> Such API would require temporary variable in caller and memcopy in the
> sth_append() function.
> 
> This is not it.  It creates a new element, expanding a list (a collection),
> and then expose this element.  Which spares us memcopy... on non-critical
> path.
> 
> I don't know how to name operation "grow list and return new element".
> But "append" it is not.

I renamed it to append_new_todo().

> >>> -	end_of_object_name = bol + strcspn(bol, " \t\n");
> >>> +	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
> >>
> >> Why is this cast needed?
> > 
> > Because bol is a "const char *" and we need to put "NUL" temporarily to
> > *end_of_object_name:
> 
> Would compiler complain without this const'ness-stripping cast?

Yes. I would not have added it otherwise.

Please note that this is only necessary because I changed the parameter
from "char *" to "const char *" (which was The Right Thing To Do).

> >>>  	saved = *end_of_object_name;
> >>>  	*end_of_object_name = '\0';
> >>>  	status = get_sha1(bol, commit_sha1);
> >>>  	*end_of_object_name = saved;
> > 
> > Technically, this would have made a fine excuse to teach get_sha1() a
> > mode where it expects a length parameter instead of relying on a
> > NUL-terminated string.
> > 
> > Practically, such fine excuses cost me months in this rebase--helper
> > project already, and I need to protect my time better.
> 
> Put it in TODO list (and perhaps add a TODO comment) ;-).

I am also a realist: I won't be able to do anything about this. If you
care enough, please go right to town.

Thanks again,
Dscho

Re: [PATCH 10/22] sequencer: avoid completely different messages for different actions

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1005 bytes --]

Hi Kuba,

On Fri, 2 Sep 2016, Jakub Narębski wrote:

> W dniu 01.09.2016 o 09:52, Johannes Schindelin pisze:
> > On Wed, 31 Aug 2016, Jakub Narębski wrote:
> >> CC-ed to Jiang Xin, L10N coordinator.
> >> W dniu 29.08.2016 o 10:05, Johannes Schindelin pisze:
> 
> [...]
> >>> -	/* Different translation strings for cherry-pick and revert */
> >>> -	if (opts->action == REPLAY_PICK)
> >>> -		error(_("Your local changes would be overwritten by cherry-pick."));
> >>> -	else
> >>> -		error(_("Your local changes would be overwritten by revert."));
> >>> +	error(_("Your local changes would be overwritten by %s."),
> >>> +		action_name(opts));
> >>
> >> If I understand it correctly, it would make "revert" or "cherry-pick"
> >> untranslated part of error message.  You would need to use translation
> >> on the result with "_(action_name(opts))", you would have to mark
> >> todo_command_strings elements for gettext lexicon with N_(...).

Okay, that is easy enough.

Ciao,
Dscho

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 823 bytes --]

Hi Junio,

On Thu, 1 Sep 2016, Junio C Hamano wrote:

> Jakub Narębski <jnareb@gmail.com> writes:
> 
> > I wonder how probable is situation where we save instruction sheet
> > for interactive rebase, with shortened SHA-1, and during rebase
> > shortened SHA-1 stops being unambiguous...
> 
> It is my understanding that the shortened ones are only for end-user
> consumption.  The insn sheet internally uses fully expanded form for
> this exact reason, and then abbreviated back at each step before the
> updated one is presented to the end-user.  Uniqueness guarantee is
> enforced with new objects created during each step taken into
> account by doing it this way.

Indeed, the rebase -i shortens the SHA-1s just before letting the user
edit git-rebase-todo and then expands them back.

Ciao,
Dscho

Re: [PATCH 21/22] sequencer: left-trim the lines read from the script

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 513 bytes --]

Hi Kuba,

On Fri, 2 Sep 2016, Jakub Narębski wrote:

> Hello Johannes,
> 
> W dniu 01.09.2016 o 16:13, Johannes Schindelin pisze: 
> > On Thu, 1 Sep 2016, Jakub Narębski wrote:
>  
> >> 'bol' is beginning-of-line, isn't it (a complement to eol)?
> > 
> > Yep. How did you guess? :-)
> 
> Wouldn't 'beg' and 'end' instead of 'bol' and 'eol' be easier
> to understand, thus more readable?

It is just consistency with the code I inherited: sequencer.c used 'bol'
and 'eol' before.

Ciao,
Dscho

Re: [PATCH 22/22] sequencer: refactor write_message()

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 950 bytes --]

Hi Kuba,

On Fri, 2 Sep 2016, Jakub Narębski wrote:

> W dniu 01.09.2016 o 16:20, Johannes Schindelin pisze:
> > On Thu, 1 Sep 2016, Jakub Narębski wrote: 
> >> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> 
> >>>  	if (commit_lock_file(&msg_file) < 0)
> >>>  		return error(_("Error wrapping up %s."), filename);
> >>
> >> Another "while at it"... though the one that can be safely postponed
> >> (well, the make message easier to understand part, not the quote
> >> filename part):
> >>
> >>   		return error(_("Error wrapping up writing to '%s'."), filename);
> > 
> > As I inherited this message, I'll keep it.
> 
> Well, please then add quotes while at it, at least, for consistency
> 
>   		return error(_("Error wrapping up '%s'."), filename);

I may do that as a final patch, once all the other concerns are addressed.
I really do not want to change the error message during the conversion.

Ciao,
Dscho

Re: [PATCH 21/22] sequencer: left-trim the lines read from the script

[Johannes Schindelin]

Hi Junio,

On Thu, 1 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> > Interactive rebase's scripts may be indented; We need to handle this
> >> > case, too, now that we prepare the sequencer to process interactive
> >> > rebases.
> >> 
> >> s/; We need/; we need/
> >
> > Hrmpf. From http://grammar.ccc.commnet.edu/grammar/marks/colon.htm:
> >
> > 	There is some disagreement among writing reference manuals about
> > 	when you should capitalize an independent clause following a
> > 	colon. Most of the manuals advise that when you have more than one
> > 	sentence in your explanation or when your sentence(s) is a formal
> > 	quotation, a capital is a good idea. The NYPL Writer's Guide urges
> > 	consistency within a document; the Chicago Manual of Style says
> > 	you may begin an independent clause with a lowercase letter unless it's
> > 	one of those two things (a quotation or more than one sentence).
> > 	The APA Publication Manual is the most extreme: it advises us to
> > 	always capitalize an independent clause following a colon. The advice
> > 	given above is consistent with the Gregg Reference Manual.
> >
> > Based on that, I think that a capital is the correct case here.
> 
> Does that manual have anything to say about semicolons, which is a
> different thing?

You're correct, I overlooked that.

Fixed,
Dscho

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

Hi Junio,

On Thu, 1 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> though).  The "one sequencer to rule them all" may even have to say
> >> "now give name ':1' to the result of the previous operation" in one
> >> step and in another later step have an instruction "merge ':1'".
> >> When that happens, you cannot even pre-populate the commit object
> >> when the sequencer reads the file, as the commit has not yet been
> >> created at that point.
> >
> > These considerations are pretty hypothetical. I would even place a bet
> > that we will *never* have ":1" as names, not if I have anything to say...
> > ;-)
> 
> If you can always work with pre-existing commit, then you can
> validate all object references that appear in the instructions
> upfront.

Or if *some* of the commands work with pre-existing commits, *those*
commands can be validated up-front.

Which is exactly what my code does.

> I was sort of expecting that, when you do the preserve-merges mode
> of "rebase -i", you would need to jump around, doing "we have
> reconstructed the side branch on a new 'onto', let's give the result
> this temporary name ':1', and then switch to the trunk (which would
> call for 'reset <commit>' instruction) and merge that thing (which
> would be 'merge :1' or perhaps called 'pick :1')", and at that point
> you no longer validate the object references upfront.

Except that is not how --preserve-merges works: it *still* uses the SHA-1s
as identifiers, even when the SHA-1 may have changed in the meantime.

That is part of why it was a bad design.

> If you do not have to have such a "mark this point" and a "refer to
> that point we previously marked", then I agree that you should be
> able to pre-validate and keep the result in the structure.

Even then, those markers should *still* be validated. They, too, need to
be created and later used, usage before creation would be an error.

But...

1) this is not yet a problem, so why are we discussing it here? Do we not
   have actual problems with these patches to discuss anymore?

2) the SHA-1s that *can* be validated *should* be validated, so I find the
   objection a little bogus.

Ciao,
Dscho

Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Jakub Narębski]

Hello Johannes,

W dniu 09.09.2016 o 17:12, Johannes Schindelin napisał:
> On Thu, 1 Sep 2016, Junio C Hamano wrote: 
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I was sort of expecting that, when you do the preserve-merges mode
>> of "rebase -i", you would need to jump around, doing "we have
>> reconstructed the side branch on a new 'onto', let's give the result
>> this temporary name ':1', and then switch to the trunk (which would
>> call for 'reset <commit>' instruction) and merge that thing (which
>> would be 'merge :1' or perhaps called 'pick :1')", and at that point
>> you no longer validate the object references upfront.
> 
> Except that is not how --preserve-merges works: it *still* uses the SHA-1s
> as identifiers, even when the SHA-1 may have changed in the meantime.
> 
> That is part of why it was a bad design.

When preserving merges, there are (as far as I understand it), two
problems:
 - what it means to preserve changes (which change to pick,
   that is what is the mainline changes rebase is re-applying)
 - what are parents of the merge commit (at least one parent
   would be usually rewritten)

Maybe the internal (and perhaps also user-visible) representation
of merge in instruction sheet could use the notation of filter-branch,
that is 'map(<sha-1>)'... it could also imply the mainline.

That is the instruction in the internal instruction sheet could
look like this:

  merge -m 1 map(2fd4e1c67a2d28fced849ee1bb76e7391b93eb12) da39a3ee5e6b4b0d3255bfef95601890afd80709 \t Merge 'foo' into master  


Note that it has nothing to do with this series!

Best regards,
-- 
Jakub Narębski

Re: [PATCH 22/22] sequencer: refactor write_message()

[Jakub Narębski]

Hello Johannes,

W dniu 09.09.2016 o 16:40, Johannes Schindelin napisał:
> On Fri, 2 Sep 2016, Jakub Narębski wrote:
>> W dniu 01.09.2016 o 16:20, Johannes Schindelin pisze:
>>> On Thu, 1 Sep 2016, Jakub Narębski wrote: 
>>>> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
>>
>>>>>  	if (commit_lock_file(&msg_file) < 0)
>>>>>  		return error(_("Error wrapping up %s."), filename);
>>>>
>>>> Another "while at it"... though the one that can be safely postponed
>>>> (well, the make message easier to understand part, not the quote
>>>> filename part):
>>>>
>>>>   		return error(_("Error wrapping up writing to '%s'."), filename);
>>>
>>> As I inherited this message, I'll keep it.
>>
>> Well, please then add quotes while at it, at least, for consistency
>>
>>   		return error(_("Error wrapping up '%s'."), filename);
> 
> I may do that as a final patch, once all the other concerns are addressed.
> I really do not want to change the error message during the conversion.

Is not wanting to change error messages during conversion because of
your use of Scientist tool to catch errors in conversion process?, as
you wrote in
  https://blogs.msdn.microsoft.com/visualstudioalm/2016/09/03/whats-new-in-git-for-windows-2-10/

BTW. could you tell us what were those three regression caught by the
cross-validation?

Best,
-- 
Jakub Narębski

Re: [PATCH 22/22] sequencer: refactor write_message()

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2604 bytes --]

Hi Kuba,

On Fri, 9 Sep 2016, Jakub Narębski wrote:

> W dniu 09.09.2016 o 16:40, Johannes Schindelin napisał:
> > On Fri, 2 Sep 2016, Jakub Narębski wrote:
> >> W dniu 01.09.2016 o 16:20, Johannes Schindelin pisze:
> >>> On Thu, 1 Sep 2016, Jakub Narębski wrote: 
> >>>> W dniu 29.08.2016 o 10:06, Johannes Schindelin pisze:
> >>
> >>>>>  	if (commit_lock_file(&msg_file) < 0)
> >>>>>  		return error(_("Error wrapping up %s."), filename);
> >>>>
> >>>> Another "while at it"... though the one that can be safely postponed
> >>>> (well, the make message easier to understand part, not the quote
> >>>> filename part):
> >>>>
> >>>>   		return error(_("Error wrapping up writing to '%s'."), filename);
> >>>
> >>> As I inherited this message, I'll keep it.
> >>
> >> Well, please then add quotes while at it, at least, for consistency
> >>
> >>   		return error(_("Error wrapping up '%s'."), filename);
> > 
> > I may do that as a final patch, once all the other concerns are addressed.
> > I really do not want to change the error message during the conversion.
> 
> Is not wanting to change error messages during conversion because of
> your use of Scientist tool to catch errors in conversion process?

It is more out of an aversion to mix unrelated purposes in the same patch.
You will see that I inserted an extra patch with the purpose of fixing the
style, that touches all the relevant error messages in sequencer.c.

> BTW. could you tell us what were those three regression caught by the
> cross-validation?

Sure!

The first one was that my original version of the rebase-i-extra patches
did not reorder patches correctly when there was more than one space after
the fixup!. I fixed it, and added this test:

cbcd2cb (rebase -i: we allow extra spaces after fixup!/squash!, 2016-07-07)

The second one was that `git commit --fixup` unwraps the commit subject
into one long line and rebase -i *still* manages to find the correct
commit to fix up. The test is part of the rebase-i-extra patches (and
therefore you will find this commit only in my fork):

9fc25ce (t3415: test fixup with wrapped oneline, 2016-07-24)

The third one was an obscure one: when I marked a commit as 'edit' and
there was a merge conflict cherry-picking that particular commit, rebase
--continue would squash the resolved changes *into the previous* commit,
but with the cherry-picked commit's message. It was a simple, stupid
oversight to write the "amend" file in the "edit" code path even if the
cherry-pick failed.

All fixed, of course.

Ciao,
Dscho

Git garden shears, was Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3006 bytes --]

Hi Kuba,

On Fri, 9 Sep 2016, Jakub Narębski wrote:

> Hello Johannes,
> 
> W dniu 09.09.2016 o 17:12, Johannes Schindelin napisał:
> > On Thu, 1 Sep 2016, Junio C Hamano wrote: 
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> I was sort of expecting that, when you do the preserve-merges mode
> >> of "rebase -i", you would need to jump around, doing "we have
> >> reconstructed the side branch on a new 'onto', let's give the result
> >> this temporary name ':1', and then switch to the trunk (which would
> >> call for 'reset <commit>' instruction) and merge that thing (which
> >> would be 'merge :1' or perhaps called 'pick :1')", and at that point
> >> you no longer validate the object references upfront.
> > 
> > Except that is not how --preserve-merges works: it *still* uses the SHA-1s
> > as identifiers, even when the SHA-1 may have changed in the meantime.
> > 
> > That is part of why it was a bad design.
> 
> When preserving merges, there are (as far as I understand it), two
> problems:
>  - what it means to preserve changes (which change to pick,
>    that is what is the mainline changes rebase is re-applying)
>  - what are parents of the merge commit (at least one parent
>    would be usually rewritten)
> 
> Maybe the internal (and perhaps also user-visible) representation
> of merge in instruction sheet could use the notation of filter-branch,
> that is 'map(<sha-1>)'... it could also imply the mainline.
> 
> That is the instruction in the internal instruction sheet could
> look like this:
> 
>   merge -m 1 map(2fd4e1c67a2d28fced849ee1bb76e7391b93eb12) da39a3ee5e6b4b0d3255bfef95601890afd80709 \t Merge 'foo' into master  
> 
> 
> Note that it has nothing to do with this series!

Right. But I did solve that already. In the Git garden shears [*1*]
(essentially my New And Improved attempt at recreating branch structures
while rebasing), I generate and process scripts like this:

	mark onto

	# Branch: super-cool-feature
	rewind onto
	pick 00001 feature
	pick 00002 documentation
	mark super-cool-feature

	# Branch: typo-fix
	rewind onto
	pick 0000a fix a tyop

	rewind onto
	merge -C cafebabe super-cool-feature
	merge -C babecafe typo-fix

	cleanup super-cool-feature typo-fix

Of course this will change a little, still, once I get around to implement
this on top of the rebase--helper.

For example, I am not so hot about the "merge -C ..." syntax. I'll
probably split that into a "remerge <SHA-1> <mark>" and a new "merge
<mark>" command (the latter asking interactively for the merge commit
message).

And also: the cleanup stage should not be necessary, as the "mark"
commands can accumulate the known marks into a file in the state
directory.

But you get the idea.

No :1 or some such. That's machine readable. But it's utter nonsense for
user-facing UIs.

Ciao,
Dscho

Footnote *1*:
https://github.com/git-for-windows/build-extra/blob/master/shears.sh

[PATCH v2 00/25] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

This patch series marks the  '4' in the countdown to speed up rebase -i
by implementing large parts in C. It is based on the `libify-sequencer`
patch series that I submitted last week.

The patches in this series merely prepare the sequencer code for the
next patch series that actually teaches the sequencer to run an
interactive rebase.

The reason to split these two patch series is simple: to keep them at a
sensible size.

The two patch series after that are much smaller: a two-patch "series"
that switches rebase -i to use the sequencer (except with --root or
--preserve-merges), and a couple of patches to move several pretty
expensive script processing steps to C (think: autosquash).

The end game of this patch series is a git-rebase--helper that makes
rebase -i 5x faster on Windows (according to t/perf/p3404). Travis says
that even MacOSX and Linux benefit (4x and 3x, respectively).

I have been working on this since early February, whenever time allowed,
and it is time to put it into the users' hands. To that end, I will most
likely submit the remaining three patch series in the next two days, and
integrate the whole shebang into Git for Windows 2.10.0.

Therefore I would be most grateful for every in-depth review.

Changes vs v1:

- clarified why the code refactoring into the short_commit_name()
  function is desirable.

- fixed a typo in a commit message (s/se/so/).

- removed a bogus call to read_and_refresh_cache(opts) that was added
  to sequencer_rollback() by mistake.

- clarified in the commit message why the TODO_LIST_INIT macro does not
  assign all-zeroes.

- converted IS_REBASE_I() into an inline function.

- clarified the comment about retaining author metadata when calling
  sequencer_commit() in rebase -i mode.

- simplified TODO_LIST_INIT and REPLAY_OPTS_INIT.

- added an example how to use sequencer_entrust() to the commit message
  introducing it.

- the gpg_sign option's value is now also entrusted to the sequencer
  (i.e. it is released in sequencer_remove_state()).

- renamed the "set_me_free_after_use" to "to_free". Since that is still
  not a self-explanatory name, add a comment to the function
  declaration.

- clarified in the "completely revamped todo parsing" commit message
  that the main benefit of the early parsing is for rebase -i.

- start the `enum todo_command` with TODO_PICK that is explicitly
  assigned the value 0.

- an error was marked as translatable.

- converted one forgotten git_path_todo_file() to use
  get_todo_path(opts) instead.

- fixed numbering of "malformed instruction sheet"s after removing one.

- reintroduced the overzealous assumption that todo scripts can only
  perform revert commands during `git revert` and pick commands during
  `git cherry-pick`. This overzealous assumption is *still* disabled in
  rebase -i mode, of course.

- clarified in the commit message why we call the field "arg", not
  "oneline", and fixed the description that claimed that we store the
  end offset (we store the length instead).

- clarified in the commit message of "allow editing the commit message
  on a case-by-case basis" that we are talking about the
  sequencer_commit() function, and how things were done previously.

- some grammar touch-ups.

- marked a couple of error messages for translation.

- explicitly assign the first todo_command the value 0, so that we can
  be certain that the array of command strings matches up.

- removed code to skip CR/LF at the end of line after reading with
  read_oneliner(): That function already ensures that.

- ensured that the todo_list is released even when reading/parsing fails.

- replaced an error(..., strerror()) call with an error_errno(...) one.

- clarified in the commit message why the new todo_list maintains the
  script as an array instead of a linked list.

- renamed the append_todo() function into append_new_todo(), to explain
  better what the function does and why it does not take the values as
  parameters that should populate the new todo_item.

- marked action_name() for translation.

- surrounded all file names in error messages in sequencer.c with single
  quotes, for consistency.

- removed a bogus hint for translators that asked them to translate
  commands of the git-rebase-todo file (or cherry-pick's equivalent).

- turned capitals after semicolons to lower-case.


Johannes Schindelin (25):
  sequencer: use static initializers for replay_opts
  sequencer: use memoized sequencer directory path
  sequencer: avoid unnecessary indirection
  sequencer: future-proof remove_sequencer_state()
  sequencer: allow the sequencer to take custody of malloc()ed data
  sequencer: release memory that was allocated when reading options
  sequencer: future-proof read_populate_todo()
  sequencer: completely revamp the "todo" script parsing
  sequencer: avoid completely different messages for different actions
  sequencer: get rid of the subcommand field
  sequencer: refactor the code to obtain a short commit name
  sequencer: remember the onelines when parsing the todo file
  sequencer: prepare for rebase -i's commit functionality
  sequencer: introduce a helper to read files written by scripts
  sequencer: prepare for rebase -i's GPG settings
  sequencer: allow editing the commit message on a case-by-case basis
  sequencer: support amending commits
  sequencer: support cleaning up commit messages
  sequencer: remember do_recursive_merge()'s return value
  sequencer: left-trim lines read from the script
  sequencer: refactor write_message()
  sequencer: remove overzealous assumption in rebase -i mode
  sequencer: mark action_name() for translation
  sequencer: quote filenames in error messages
  sequencer: remove bogus hint for translators

 builtin/commit.c |   2 +-
 builtin/revert.c |  42 ++--
 sequencer.c      | 632 +++++++++++++++++++++++++++++++++++++------------------
 sequencer.h      |  31 +--
 4 files changed, 468 insertions(+), 239 deletions(-)

Based-On: libify-sequencer at https://github.com/dscho/git
Fetch-Base-Via: git fetch https://github.com/dscho/git libify-sequencer
Published-As: https://github.com/dscho/git/releases/tag/prepare-sequencer-v2
Fetch-It-Via: git fetch https://github.com/dscho/git prepare-sequencer-v2

Interdiff vs v1:

 diff --git a/sequencer.c b/sequencer.c
 index 7d56676..cdff0f1 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -41,7 +41,10 @@ static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
  static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
  
  /* We will introduce the 'interactive rebase' mode later */
 -#define IS_REBASE_I() 0
 +static inline int is_rebase_i(const struct replay_opts *opts)
 +{
 +	return 0;
 +}
  
  static const char *get_dir(const struct replay_opts *opts)
  {
 @@ -145,12 +148,12 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
  	return buf.buf;
  }
  
 -void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use)
 +void *sequencer_entrust(struct replay_opts *opts, void *to_free)
  {
  	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
 -	opts->owned[opts->owned_nr++] = set_me_free_after_use;
 +	opts->owned[opts->owned_nr++] = to_free;
  
 -	return set_me_free_after_use;
 +	return to_free;
  }
  
  int sequencer_remove_state(struct replay_opts *opts)
 @@ -173,7 +176,7 @@ int sequencer_remove_state(struct replay_opts *opts)
  
  static const char *action_name(const struct replay_opts *opts)
  {
 -	return opts->action == REPLAY_REVERT ? "revert" : "cherry-pick";
 +	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
  }
  
  struct commit_message {
 @@ -248,11 +251,11 @@ static int write_with_lock_file(const char *filename,
  	if (msg_fd < 0)
  		return error_errno(_("Could not lock '%s'"), filename);
  	if (write_in_full(msg_fd, buf, len) < 0)
 -		return error_errno(_("Could not write to %s"), filename);
 +		return error_errno(_("Could not write to '%s'"), filename);
  	if (append_eol && write(msg_fd, "\n", 1) < 0)
 -		return error_errno(_("Could not write eol to %s"), filename);
 +		return error_errno(_("Could not write eol to '%s"), filename);
  	if (commit_lock_file(&msg_file) < 0)
 -		return error(_("Error wrapping up %s."), filename);
 +		return error(_("Error wrapping up '%s'."), filename);
  
  	return 0;
  }
 @@ -309,10 +312,10 @@ static struct tree *empty_tree(void)
  static int error_dirty_index(struct replay_opts *opts)
  {
  	if (read_cache_unmerged())
 -		return error_resolve_conflict(action_name(opts));
 +		return error_resolve_conflict(_(action_name(opts)));
  
  	error(_("Your local changes would be overwritten by %s."),
 -		action_name(opts));
 +		_(action_name(opts)));
  
  	if (advice_commit_before_merge)
  		advise(_("Commit your changes or stash them to proceed."));
 @@ -330,7 +333,7 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
  	if (checkout_fast_forward(from, to, 1))
  		return -1; /* the callee should have complained already */
  
 -	strbuf_addf(&sb, _("%s: fast-forward"), action_name(opts));
 +	strbuf_addf(&sb, _("%s: fast-forward"), _(action_name(opts)));
  
  	transaction = ref_transaction_begin(&err);
  	if (!transaction ||
 @@ -406,7 +409,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
  	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
  		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
  		return error(_("%s: Unable to write new index file"),
 -			action_name(opts));
 +			_(action_name(opts)));
  	rollback_lock_file(&index_lock);
  
  	if (opts->signoff)
 @@ -488,9 +491,13 @@ static char **read_author_script(void)
   * If we are cherry-pick, and if the merge did not result in
   * hand-editing, we will hit this commit and inherit the original
   * author date and name.
 + *
   * If we are revert, or if our cherry-pick results in a hand merge,
 - * we had better say that the current user is responsible for that
 - * (except, of course, while running an interactive rebase).
 + * we had better say that the current user is responsible for that.
 + *
 + * An exception is when sequencer_commit() is called during an
 + * interactive rebase: in that case, we will want to retain the
 + * author metadata.
   */
  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
  			  int allow_empty, int edit, int amend,
 @@ -501,7 +508,7 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
  	int rc;
  	const char *value;
  
 -	if (IS_REBASE_I()) {
 +	if (is_rebase_i(opts)) {
  		env = read_author_script();
  		if (!env) {
  			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 @@ -513,7 +520,7 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
  				"If they are meant to go into a new commit, "
  				"run:\n\n"
  				"  git commit %s\n\n"
 -				"In both case, once you're done, continue "
 +				"In both cases, once you're done, continue "
  				"with:\n\n"
  				"  git rebase --continue\n", gpg_opt, gpg_opt);
  		}
 @@ -613,7 +620,7 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
  }
  
  enum todo_command {
 -	TODO_PICK,
 +	TODO_PICK = 0,
  	TODO_REVERT
  };
  
 @@ -690,8 +697,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
  
  	if (parent && parse_commit(parent) < 0)
 -		/* TRANSLATORS: The first %s will be "revert" or
 -		   "cherry-pick", the second %s a SHA1 */
  		return error(_("%s: cannot parse parent commit %s"),
  			command_to_string(command),
  			oid_to_hex(&parent->object.oid));
 @@ -833,14 +838,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
  	if (read_index_preload(&the_index, NULL) < 0) {
  		rollback_lock_file(&index_lock);
  		return error(_("git %s: failed to read the index"),
 -			action_name(opts));
 +			_(action_name(opts)));
  	}
  	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, NULL, NULL, NULL);
  	if (the_index.cache_changed && index_fd >= 0) {
  		if (write_locked_index(&the_index, &index_lock, COMMIT_LOCK)) {
  			rollback_lock_file(&index_lock);
  			return error(_("git %s: failed to refresh the index"),
 -				action_name(opts));
 +				_(action_name(opts)));
  		}
  	}
  	rollback_lock_file(&index_lock);
 @@ -861,7 +866,7 @@ struct todo_list {
  	int nr, alloc, current;
  };
  
 -#define TODO_LIST_INIT { STRBUF_INIT, NULL, 0, 0, 0 }
 +#define TODO_LIST_INIT { STRBUF_INIT }
  
  static void todo_list_release(struct todo_list *todo_list)
  {
 @@ -871,7 +876,7 @@ static void todo_list_release(struct todo_list *todo_list)
  	todo_list->nr = todo_list->alloc = 0;
  }
  
 -struct todo_item *append_todo(struct todo_list *todo_list)
 +struct todo_item *append_new_todo(struct todo_list *todo_list)
  {
  	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
  	return todo_list->items + todo_list->nr++;
 @@ -925,11 +930,11 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  	for (i = 1; *p; i++) {
  		char *eol = strchrnul(p, '\n');
  
 -		item = append_todo(todo_list);
 +		item = append_new_todo(todo_list);
  		item->offset_in_buf = p - todo_list->buf.buf;
  		if (parse_insn_line(item, p, eol)) {
 -			error("Invalid line: %.*s", (int)(eol - p), p);
 -			res |= error(_("Could not parse line %d."), i);
 +			res |= error(_("Invalid line %d: %.*s"),
 +				i, (int)(eol - p), p);
  			item->command = -1;
  		}
  		p = *eol ? eol + 1 : eol;
 @@ -948,16 +953,31 @@ static int read_populate_todo(struct todo_list *todo_list,
  	strbuf_reset(&todo_list->buf);
  	fd = open(todo_file, O_RDONLY);
  	if (fd < 0)
 -		return error_errno(_("Could not open %s"), todo_file);
 +		return error_errno(_("Could not open '%s'"), todo_file);
  	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
  		close(fd);
 -		return error(_("Could not read %s."), todo_file);
 +		return error(_("Could not read '%s'."), todo_file);
  	}
  	close(fd);
  
  	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
  	if (res)
 -		return error(_("Unusable instruction sheet: %s"), todo_file);
 +		return error(_("Unusable instruction sheet: '%s'"), todo_file);
 +
 +	if (!is_rebase_i(opts)) {
 +		enum todo_command valid =
 +			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
 +		int i;
 +
 +		for (i = 0; i < todo_list->nr; i++)
 +			if (valid == todo_list->items[i].command)
 +				continue;
 +			else if (valid == TODO_PICK)
 +				return error(_("Cannot cherry-pick during a revert."));
 +			else
 +				return error(_("Cannot revert during a cherry-pick."));
 +	}
 +
  	return 0;
  }
  
 @@ -1003,22 +1023,16 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
  
  static int read_populate_opts(struct replay_opts *opts)
  {
 -	if (IS_REBASE_I()) {
 +	if (is_rebase_i(opts)) {
  		struct strbuf buf = STRBUF_INIT;
  
  		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
 -			if (buf.len && buf.buf[buf.len - 1] == '\n') {
 -				if (--buf.len &&
 -				    buf.buf[buf.len - 1] == '\r')
 -					buf.len--;
 -				buf.buf[buf.len] = '\0';
 -			}
 -
  			if (!starts_with(buf.buf, "-S"))
  				strbuf_reset(&buf);
  			else {
  				opts->gpg_sign = buf.buf + 2;
 -				strbuf_detach(&buf, NULL);
 +				sequencer_entrust(opts,
 +					strbuf_detach(&buf, NULL));
  			}
  		}
  
 @@ -1034,7 +1048,7 @@ static int read_populate_opts(struct replay_opts *opts)
  	 * are pretty certain that it is syntactically correct.
  	 */
  	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
 -		return error(_("Malformed options sheet: %s"),
 +		return error(_("Malformed options sheet: '%s'"),
  			git_path_opts_file());
  	return 0;
  }
 @@ -1044,13 +1058,14 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
  {
  	enum todo_command command = opts->action == REPLAY_PICK ?
  		TODO_PICK : TODO_REVERT;
 +	const char *command_string = todo_command_strings[command];
  	struct commit *commit;
  
  	if (prepare_revs(opts))
  		return -1;
  
  	while ((commit = get_revision(opts->revs))) {
 -		struct todo_item *item = append_todo(todo_list);
 +		struct todo_item *item = append_new_todo(todo_list);
  		const char *commit_buffer = get_commit_buffer(commit, NULL);
  		const char *subject;
  		int subject_len;
 @@ -1061,8 +1076,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
  		item->arg_len = 0;
  		item->offset_in_buf = todo_list->buf.len;
  		subject_len = find_commit_subject(commit_buffer, &subject);
 -		strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
 -			opts->action == REPLAY_PICK ?  "pick" : "revert",
 +		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
  			short_commit_name(commit), subject_len, subject);
  		unuse_commit_buffer(commit, commit_buffer);
  	}
 @@ -1077,7 +1091,7 @@ static int create_seq_dir(void)
  		return -1;
  	}
  	else if (mkdir(git_path_seq_dir(), 0777) < 0)
 -		return error_errno(_("Could not create sequencer directory %s"),
 +		return error_errno(_("Could not create sequencer directory '%s'"),
  				   git_path_seq_dir());
  	return 0;
  }
 @@ -1096,12 +1110,12 @@ static int save_head(const char *head)
  	strbuf_addf(&buf, "%s\n", head);
  	if (write_in_full(fd, buf.buf, buf.len) < 0) {
  		rollback_lock_file(&head_lock);
 -		return error_errno(_("Could not write to %s"),
 +		return error_errno(_("Could not write to '%s'"),
  				   git_path_head_file());
  	}
  	if (commit_lock_file(&head_lock) < 0) {
  		rollback_lock_file(&head_lock);
 -		return error(_("Error wrapping up %s."), git_path_head_file());
 +		return error(_("Error wrapping up '%s'."), git_path_head_file());
  	}
  	return 0;
  }
 @@ -1136,9 +1150,6 @@ int sequencer_rollback(struct replay_opts *opts)
  	unsigned char sha1[20];
  	struct strbuf buf = STRBUF_INIT;
  
 -	if (read_and_refresh_cache(opts))
 -		return -1;
 -
  	f = fopen(git_path_head_file(), "r");
  	if (!f && errno == ENOENT) {
  		/*
 @@ -1149,9 +1160,9 @@ int sequencer_rollback(struct replay_opts *opts)
  		return rollback_single_pick();
  	}
  	if (!f)
 -		return error_errno(_("cannot open %s"), git_path_head_file());
 +		return error_errno(_("cannot open '%s'"), git_path_head_file());
  	if (strbuf_getline_lf(&buf, f)) {
 -		error(_("cannot read %s: %s"), git_path_head_file(),
 +		error(_("cannot read '%s': %s"), git_path_head_file(),
  		      ferror(f) ?  strerror(errno) : _("unexpected end of file"));
  		fclose(f);
  		goto fail;
 @@ -1183,16 +1194,14 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
  
  	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
  	if (fd < 0)
 -		return error_errno(_("Could not lock '%s'"),
 -				   git_path_todo_file());
 +		return error_errno(_("Could not lock '%s'"), todo_path);
  	offset = next < todo_list->nr ?
  		todo_list->items[next].offset_in_buf : todo_list->buf.len;
  	if (write_in_full(fd, todo_list->buf.buf + offset,
  			todo_list->buf.len - offset) < 0)
 -		return error(_("Could not write to %s (%s)"),
 -			todo_path, strerror(errno));
 +		return error_errno(_("Could not write to '%s'"), todo_path);
  	if (commit_lock_file(&todo_lock) < 0)
 -		return error(_("Error wrapping up %s."), todo_path);
 +		return error(_("Error wrapping up '%s'."), todo_path);
  	return 0;
  }
  
 @@ -1279,21 +1288,25 @@ int sequencer_continue(struct replay_opts *opts)
  
  	if (!file_exists(get_todo_path(opts)))
  		return continue_single_pick();
 -	if (read_populate_opts(opts) ||
 -			read_populate_todo(&todo_list, opts))
 +	if (read_populate_opts(opts))
  		return -1;
 +	if ((res = read_populate_todo(&todo_list, opts)))
 +		goto release_todo_list;
  
  	/* Verify that the conflict has been resolved */
  	if (file_exists(git_path_cherry_pick_head()) ||
  	    file_exists(git_path_revert_head())) {
 -		int ret = continue_single_pick();
 -		if (ret)
 -			return ret;
 +		res = continue_single_pick();
 +		if (res)
 +			goto release_todo_list;
 +	}
 +	if (index_differs_from("HEAD", 0)) {
 +		res = error_dirty_index(opts);
 +		goto release_todo_list;
  	}
 -	if (index_differs_from("HEAD", 0))
 -		return error_dirty_index(opts);
  	todo_list.current++;
  	res = pick_commits(&todo_list, opts);
 +release_todo_list:
  	todo_list_release(&todo_list);
  	return res;
  }
 diff --git a/sequencer.h b/sequencer.h
 index e272549..688fff1 100644
 --- a/sequencer.h
 +++ b/sequencer.h
 @@ -40,9 +40,13 @@ struct replay_opts {
  	void **owned;
  	int owned_nr, owned_alloc;
  };
 -#define REPLAY_OPTS_INIT { -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, NULL, NULL, NULL, 0, 0, NULL, NULL, 0, 0 }
 +#define REPLAY_OPTS_INIT { -1 }
  
 -void *sequencer_entrust(struct replay_opts *opts, void *set_me_free_after_use);
 +/*
 + * Make it the duty of sequencer_remove_state() to release the memory;
 + * For ease of use, return the same pointer.
 + */
 +void *sequencer_entrust(struct replay_opts *opts, void *to_free);
  
  int sequencer_pick_revisions(struct replay_opts *opts);
  int sequencer_continue(struct replay_opts *opts);
 diff --git a/t/t3510-cherry-pick-sequence.sh b/t/t3510-cherry-pick-sequence.sh
 index 6465edf..7b7a89d 100755
 --- a/t/t3510-cherry-pick-sequence.sh
 +++ b/t/t3510-cherry-pick-sequence.sh
 @@ -459,6 +459,17 @@ test_expect_success 'malformed instruction sheet 1' '
  	test_expect_code 128 git cherry-pick --continue
  '
  
 +test_expect_success 'malformed instruction sheet 2' '
 +	pristine_detach initial &&
 +	test_expect_code 1 git cherry-pick base..anotherpick &&
 +	echo "resolved" >foo &&
 +	git add foo &&
 +	git commit &&
 +	sed "s/pick/revert/" .git/sequencer/todo >new_sheet &&
 +	cp new_sheet .git/sequencer/todo &&
 +	test_expect_code 128 git cherry-pick --continue
 +'
 +
  test_expect_success 'empty commit set' '
  	pristine_detach initial &&
  	test_expect_code 128 git cherry-pick base..base

-- 
2.10.0.windows.1.10.g803177d

base-commit: 8a04d01a511ecffba8a397d26ee7fa6ca56b6e7e

[PATCH v2 01/25] sequencer: use static initializers for replay_opts

[Johannes Schindelin]

This change is not completely faithful: instead of initializing all fields
to 0, we choose to initialize command and subcommand to -1 (instead of
defaulting to REPLAY_REVERT and REPLAY_NONE, respectively). Practically,
it makes no difference at all, but future-proofs the code to require
explicit assignments for both fields.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 6 ++----
 sequencer.h      | 1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 4e69380..7365559 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -178,10 +178,9 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	if (isatty(0))
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
@@ -195,10 +194,9 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 
 int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
 	parse_args(argc, argv, &opts);
diff --git a/sequencer.h b/sequencer.h
index 5ed5cb1..db425ad 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -47,6 +47,7 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
+#define REPLAY_OPTS_INIT { -1, -1 }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 02/25] sequencer: use memoized sequencer directory path

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/commit.c |  2 +-
 sequencer.c      | 11 ++++++-----
 sequencer.h      |  5 +----
 3 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index bb9f79b..e79af9d 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -183,7 +183,7 @@ static void determine_whence(struct wt_status *s)
 		whence = FROM_MERGE;
 	else if (file_exists(git_path_cherry_pick_head())) {
 		whence = FROM_CHERRY_PICK;
-		if (file_exists(git_path(SEQ_DIR)))
+		if (file_exists(git_path_seq_dir()))
 			sequencer_in_use = 1;
 	}
 	else
diff --git a/sequencer.c b/sequencer.c
index eec8a60..cb16cbd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,10 +21,11 @@
 const char sign_off_header[] = "Signed-off-by: ";
 static const char cherry_picked_prefix[] = "(cherry picked from commit ";
 
-static GIT_PATH_FUNC(git_path_todo_file, SEQ_TODO_FILE)
-static GIT_PATH_FUNC(git_path_opts_file, SEQ_OPTS_FILE)
-static GIT_PATH_FUNC(git_path_seq_dir, SEQ_DIR)
-static GIT_PATH_FUNC(git_path_head_file, SEQ_HEAD_FILE)
+GIT_PATH_FUNC(git_path_seq_dir, "sequencer")
+
+static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
+static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
+static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
 static int is_rfc2822_line(const char *buf, int len)
 {
@@ -112,7 +113,7 @@ static void remove_sequencer_state(void)
 {
 	struct strbuf seq_dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path(SEQ_DIR));
+	strbuf_addstr(&seq_dir, git_path_seq_dir());
 	remove_dir_recursively(&seq_dir, 0);
 	strbuf_release(&seq_dir);
 }
diff --git a/sequencer.h b/sequencer.h
index db425ad..dd4d33a 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -1,10 +1,7 @@
 #ifndef SEQUENCER_H
 #define SEQUENCER_H
 
-#define SEQ_DIR		"sequencer"
-#define SEQ_HEAD_FILE	"sequencer/head"
-#define SEQ_TODO_FILE	"sequencer/todo"
-#define SEQ_OPTS_FILE	"sequencer/opts"
+const char *git_path_seq_dir(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 03/25] sequencer: avoid unnecessary indirection

[Johannes Schindelin]

We really do not need the *pointer to a* pointer to the options in
the read_populate_opts() function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index cb16cbd..c2fbf6f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -813,7 +813,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
-static int read_populate_opts(struct replay_opts **opts)
+static int read_populate_opts(struct replay_opts *opts)
 {
 	if (!file_exists(git_path_opts_file()))
 		return 0;
@@ -823,7 +823,7 @@ static int read_populate_opts(struct replay_opts **opts)
 	 * about this case, though, because we wrote that file ourselves, so we
 	 * are pretty certain that it is syntactically correct.
 	 */
-	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
+	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
 		return error(_("Malformed options sheet: %s"),
 			git_path_opts_file());
 	return 0;
@@ -1054,7 +1054,7 @@ static int sequencer_continue(struct replay_opts *opts)
 
 	if (!file_exists(git_path_todo_file()))
 		return continue_single_pick();
-	if (read_populate_opts(&opts) ||
+	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
 		return -1;
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 04/25] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

In a couple of commits, we will teach the sequencer to handle the
nitty gritty of the interactive rebase, which keeps its state in a
different directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index c2fbf6f..8d272fb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,11 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static const char *get_dir(const struct replay_opts *opts)
+{
+	return git_path_seq_dir();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -109,13 +114,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(void)
+static void remove_sequencer_state(const struct replay_opts *opts)
 {
-	struct strbuf seq_dir = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path_seq_dir());
-	remove_dir_recursively(&seq_dir, 0);
-	strbuf_release(&seq_dir);
+	strbuf_addf(&dir, "%s", get_dir(opts));
+	remove_dir_recursively(&dir, 0);
+	strbuf_release(&dir);
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -940,7 +945,7 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	strbuf_release(&buf);
 	return 0;
 fail:
@@ -1034,7 +1039,7 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	return 0;
 }
 
@@ -1095,7 +1100,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	 * one that is being continued
 	 */
 	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state();
+		remove_sequencer_state(opts);
 		return 0;
 	}
 	if (opts->subcommand == REPLAY_ROLLBACK)
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 05/25] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Schindelin]

The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
like a one-shot command when it reads its configuration: memory is
allocated and released only when the command exits.

This is kind of okay for git-cherry-pick, which *is* a one-shot
command. All the work to make the sequencer its work horse was
done to allow using the functionality as a library function, though,
including proper clean-up after use.

This patch introduces an API to pass the responsibility of releasing
certain memory to the sequencer. Example:

	const char *label =
		sequencer_entrust(opts, xstrfmt("From: %s", email));

The entrusted memory will remain valid until sequencer_remove_state() is
called, or the program exits, whichever comes first.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 +++++++++++++
 sequencer.h | 10 ++++++++++
 2 files changed, 23 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 8d272fb..8d56a05 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -114,9 +114,22 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
+void *sequencer_entrust(struct replay_opts *opts, void *to_free)
+{
+	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
+	opts->owned[opts->owned_nr++] = to_free;
+
+	return to_free;
+}
+
 static void remove_sequencer_state(const struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
+	int i;
+
+	for (i = 0; i < opts->owned_nr; i++)
+		free(opts->owned[i]);
+	free(opts->owned);
 
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
diff --git a/sequencer.h b/sequencer.h
index dd4d33a..04892a9 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -43,9 +43,19 @@ struct replay_opts {
 
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
+
+	/* malloc()ed data entrusted to the sequencer */
+	void **owned;
+	int owned_nr, owned_alloc;
 };
 #define REPLAY_OPTS_INIT { -1, -1 }
 
+/*
+ * Make it the duty of sequencer_remove_state() to release the memory;
+ * For ease of use, return the same pointer.
+ */
+void *sequencer_entrust(struct replay_opts *opts, void *to_free);
+
 int sequencer_pick_revisions(struct replay_opts *opts);
 
 extern const char sign_off_header[];
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 06/25] sequencer: release memory that was allocated when reading options

[Johannes Schindelin]

The sequencer reads options from disk and stores them in its struct
for use during sequencer's operations.

With this patch, the memory is released afterwards, plugging a
memory leak.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 8d56a05..3ca231f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -131,6 +131,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
 		free(opts->owned[i]);
 	free(opts->owned);
 
+	free(opts->xopts);
+
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
 	strbuf_release(&dir);
@@ -815,13 +817,18 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
 	else if (!strcmp(key, "options.mainline"))
 		opts->mainline = git_config_int(key, value);
-	else if (!strcmp(key, "options.strategy"))
+	else if (!strcmp(key, "options.strategy")) {
 		git_config_string(&opts->strategy, key, value);
-	else if (!strcmp(key, "options.gpg-sign"))
+		sequencer_entrust(opts, (char *) opts->strategy);
+	}
+	else if (!strcmp(key, "options.gpg-sign")) {
 		git_config_string(&opts->gpg_sign, key, value);
+		sequencer_entrust(opts, (char *) opts->gpg_sign);
+	}
 	else if (!strcmp(key, "options.strategy-option")) {
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
-		opts->xopts[opts->xopts_nr++] = xstrdup(value);
+		opts->xopts[opts->xopts_nr++] =
+			sequencer_entrust(opts, xstrdup(value));
 	} else
 		return error(_("Invalid key: %s"), key);
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 07/25] sequencer: future-proof read_populate_todo()

[Johannes Schindelin]

Over the next commits, we will work on improving the sequencer to the
point where it can process the edit script of an interactive rebase. To
that end, we will need to teach the sequencer to read interactive
rebase's todo file. In preparation, we consolidate all places where
that todo file is needed to call a function that we will later extend.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3ca231f..da6de22 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -32,6 +32,11 @@ static const char *get_dir(const struct replay_opts *opts)
 	return git_path_seq_dir();
 }
 
+static const char *get_todo_path(const struct replay_opts *opts)
+{
+	return git_path_todo_file();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -776,25 +781,24 @@ static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
 static int read_populate_todo(struct commit_list **todo_list,
 			struct replay_opts *opts)
 {
+	const char *todo_file = get_todo_path(opts);
 	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
-	fd = open(git_path_todo_file(), O_RDONLY);
+	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"),
-				   git_path_todo_file());
+		return error_errno(_("Could not open %s"), todo_file);
 	if (strbuf_read(&buf, fd, 0) < 0) {
 		close(fd);
 		strbuf_release(&buf);
-		return error(_("Could not read %s."), git_path_todo_file());
+		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(buf.buf, todo_list, opts);
 	strbuf_release(&buf);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"),
-			git_path_todo_file());
+		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
@@ -1077,7 +1081,7 @@ static int sequencer_continue(struct replay_opts *opts)
 {
 	struct commit_list *todo_list = NULL;
 
-	if (!file_exists(git_path_todo_file()))
+	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 08/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

When we came up with the "sequencer" idea, we really wanted to have
kind of a plumbing equivalent of the interactive rebase. Hence the
choice of words: the "todo" script, a "pick", etc.

However, when it came time to implement the entire shebang, somehow this
idea got lost and the sequencer was used as working horse for
cherry-pick and revert instead. So as not to interfere with the
interactive rebase, it even uses a separate directory to store its
state.

Furthermore, it also is stupidly strict about the "todo" script it
accepts: while it parses commands in a way that was *designed* to be
similar to the interactive rebase, it then goes on to *error out* if the
commands disagree with the overall action (cherry-pick or revert).

Finally, the sequencer code chose to deviate from the interactive rebase
code insofar that it *reformats* the "todo" script instead of just
writing the part of the parsed script that were not yet processed. This
is not only unnecessary churn, but might well lose information that is
valuable to the user (i.e. comments after the commands).

Let's just bite the bullet and rewrite the entire parser; the code now
becomes not only more elegant: it allows us to go on and teach the
sequencer how to parse *true* "todo" scripts as used by the interactive
rebase itself. In a way, the sequencer is about to grow up to do its
older brother's job. Better.

In particular, we choose to maintain the list of commands in an array
instead of a linked list: this is flexible enough to allow us later on to
even implement rebase -i's reordering of fixup!/squash! commits very
easily (and with a very nice speed bonus, at least on Windows).

While at it, do not stop at the first problem, but list *all* of the
problems. This will help the user when the sequencer will do `rebase
-i`'s work by allowing to address all issues in one go rather than going
back and forth until the todo list is valid.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 275 +++++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 159 insertions(+), 116 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index da6de22..b971ad0 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -473,7 +473,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
+enum todo_command {
+	TODO_PICK = 0,
+	TODO_REVERT
+};
+
+static const char *todo_command_strings[] = {
+	"pick",
+	"revert"
+};
+
+static const char *command_to_string(const enum todo_command command)
+{
+	if (command < ARRAY_SIZE(todo_command_strings))
+		return todo_command_strings[command];
+	die("Unknown command: %d", command);
+}
+
+
+static int do_pick_commit(enum todo_command command, struct commit *commit,
+		struct replay_opts *opts)
 {
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
@@ -535,7 +554,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		/* TRANSLATORS: The first %s will be "revert" or
 		   "cherry-pick", the second %s a SHA1 */
 		return error(_("%s: cannot parse parent commit %s"),
-			action_name(opts), oid_to_hex(&parent->object.oid));
+			command_to_string(command),
+			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
 		return error(_("Cannot get commit message for %s"),
@@ -548,7 +568,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * reverse of it if we are revert.
 	 */
 
-	if (opts->action == REPLAY_REVERT) {
+	if (command == TODO_REVERT) {
 		base = commit;
 		base_label = msg.label;
 		next = parent;
@@ -589,7 +609,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
+	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
@@ -615,17 +635,17 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * However, if the merge did not even start, then we don't want to
 	 * write it at all.
 	 */
-	if (opts->action == REPLAY_PICK && !opts->no_commit && (res == 0 || res == 1) &&
+	if (command == TODO_PICK && !opts->no_commit && (res == 0 || res == 1) &&
 	    update_ref(NULL, "CHERRY_PICK_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
-	if (opts->action == REPLAY_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
+	if (command == TODO_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
 	    update_ref(NULL, "REVERT_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
 
 	if (res) {
-		error(opts->action == REPLAY_REVERT
+		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
 		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
@@ -687,116 +707,121 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
-static int format_todo(struct strbuf *buf, struct commit_list *todo_list,
-		struct replay_opts *opts)
+struct todo_item {
+	enum todo_command command;
+	struct commit *commit;
+	size_t offset_in_buf;
+};
+
+struct todo_list {
+	struct strbuf buf;
+	struct todo_item *items;
+	int nr, alloc, current;
+};
+
+#define TODO_LIST_INIT { STRBUF_INIT }
+
+static void todo_list_release(struct todo_list *todo_list)
 {
-	struct commit_list *cur = NULL;
-	const char *sha1_abbrev = NULL;
-	const char *action_str = opts->action == REPLAY_REVERT ? "revert" : "pick";
-	const char *subject;
-	int subject_len;
+	strbuf_release(&todo_list->buf);
+	free(todo_list->items);
+	todo_list->items = NULL;
+	todo_list->nr = todo_list->alloc = 0;
+}
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		const char *commit_buffer = get_commit_buffer(cur->item, NULL);
-		sha1_abbrev = find_unique_abbrev(cur->item->object.oid.hash, DEFAULT_ABBREV);
-		subject_len = find_commit_subject(commit_buffer, &subject);
-		strbuf_addf(buf, "%s %s %.*s\n", action_str, sha1_abbrev,
-			subject_len, subject);
-		unuse_commit_buffer(cur->item, commit_buffer);
-	}
-	return 0;
+struct todo_item *append_new_todo(struct todo_list *todo_list)
+{
+	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
+	return todo_list->items + todo_list->nr++;
 }
 
-static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
+static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 {
 	unsigned char commit_sha1[20];
-	enum replay_action action;
 	char *end_of_object_name;
-	int saved, status, padding;
-
-	if (starts_with(bol, "pick")) {
-		action = REPLAY_PICK;
-		bol += strlen("pick");
-	} else if (starts_with(bol, "revert")) {
-		action = REPLAY_REVERT;
-		bol += strlen("revert");
-	} else
-		return NULL;
+	int i, saved, status, padding;
+
+	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
+		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_strings))
+		return -1;
 
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
-		return NULL;
+		return -1;
 	bol += padding;
 
-	end_of_object_name = bol + strcspn(bol, " \t\n");
+	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
-	/*
-	 * Verify that the action matches up with the one in
-	 * opts; we don't support arbitrary instructions
-	 */
-	if (action != opts->action) {
-		if (action == REPLAY_REVERT)
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot revert during another revert.")
-			    : _("Cannot revert during a cherry-pick."));
-		else
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot cherry-pick during a revert.")
-			    : _("Cannot cherry-pick during another cherry-pick."));
-		return NULL;
-	}
-
 	if (status < 0)
-		return NULL;
+		return -1;
 
-	return lookup_commit_reference(commit_sha1);
+	item->commit = lookup_commit_reference(commit_sha1);
+	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
-			struct replay_opts *opts)
+static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
-	struct commit_list **next = todo_list;
-	struct commit *commit;
+	struct todo_item *item;
 	char *p = buf;
-	int i;
+	int i, res = 0;
 
 	for (i = 1; *p; i++) {
 		char *eol = strchrnul(p, '\n');
-		commit = parse_insn_line(p, eol, opts);
-		if (!commit)
-			return error(_("Could not parse line %d."), i);
-		next = commit_list_append(commit, next);
+
+		item = append_new_todo(todo_list);
+		item->offset_in_buf = p - todo_list->buf.buf;
+		if (parse_insn_line(item, p, eol)) {
+			res |= error(_("Invalid line %d: %.*s"),
+				i, (int)(eol - p), p);
+			item->command = -1;
+		}
 		p = *eol ? eol + 1 : eol;
 	}
-	if (!*todo_list)
+	if (!todo_list->nr)
 		return error(_("No commits parsed."));
-	return 0;
+	return res;
 }
 
-static int read_populate_todo(struct commit_list **todo_list,
+static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
 	const char *todo_file = get_todo_path(opts);
-	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
+	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
 		return error_errno(_("Could not open %s"), todo_file);
-	if (strbuf_read(&buf, fd, 0) < 0) {
+	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		strbuf_release(&buf);
 		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
-	res = parse_insn_buffer(buf.buf, todo_list, opts);
-	strbuf_release(&buf);
+	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
+	if (!res) {
+		enum todo_command valid =
+			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
+		int i;
+
+		for (i = 0; i < todo_list->nr; i++)
+			if (valid == todo_list->items[i].command)
+				continue;
+			else if (valid == TODO_PICK)
+				return error(_("Cannot cherry-pick during a revert."));
+			else
+				return error(_("Cannot revert during a cherry-pick."));
+	}
+
 	if (res)
 		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
@@ -858,18 +883,33 @@ static int read_populate_opts(struct replay_opts *opts)
 	return 0;
 }
 
-static int walk_revs_populate_todo(struct commit_list **todo_list,
+static int walk_revs_populate_todo(struct todo_list *todo_list,
 				struct replay_opts *opts)
 {
+	enum todo_command command = opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT;
+	const char *command_string = todo_command_strings[command];
 	struct commit *commit;
-	struct commit_list **next;
 
 	if (prepare_revs(opts))
 		return -1;
 
-	next = todo_list;
-	while ((commit = get_revision(opts->revs)))
-		next = commit_list_append(commit, next);
+	while ((commit = get_revision(opts->revs))) {
+		struct todo_item *item = append_new_todo(todo_list);
+		const char *commit_buffer = get_commit_buffer(commit, NULL);
+		const char *subject;
+		int subject_len;
+
+		item->command = command;
+		item->commit = commit;
+		item->offset_in_buf = todo_list->buf.len;
+		subject_len = find_commit_subject(commit_buffer, &subject);
+		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV),
+			subject_len, subject);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
 	return 0;
 }
 
@@ -977,30 +1017,22 @@ static int sequencer_rollback(struct replay_opts *opts)
 	return -1;
 }
 
-static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
+static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	static struct lock_file todo_lock;
-	struct strbuf buf = STRBUF_INIT;
-	int fd;
+	const char *todo_path = get_todo_path(opts);
+	int next = todo_list->current, offset, fd;
 
-	fd = hold_lock_file_for_update(&todo_lock, git_path_todo_file(), 0);
+	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
-		return error_errno(_("Could not lock '%s'"),
-				   git_path_todo_file());
-	if (format_todo(&buf, todo_list, opts) < 0) {
-		strbuf_release(&buf);
-		return error(_("Could not format %s."), git_path_todo_file());
-	}
-	if (write_in_full(fd, buf.buf, buf.len) < 0) {
-		strbuf_release(&buf);
-		return error_errno(_("Could not write to %s"),
-				   git_path_todo_file());
-	}
-	if (commit_lock_file(&todo_lock) < 0) {
-		strbuf_release(&buf);
-		return error(_("Error wrapping up %s."), git_path_todo_file());
-	}
-	strbuf_release(&buf);
+		return error_errno(_("Could not lock '%s'"), todo_path);
+	offset = next < todo_list->nr ?
+		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	if (write_in_full(fd, todo_list->buf.buf + offset,
+			todo_list->buf.len - offset) < 0)
+		return error_errno(_("Could not write to '%s'"), todo_path);
+	if (commit_lock_file(&todo_lock) < 0)
+		return error(_("Error wrapping up %s."), todo_path);
 	return 0;
 }
 
@@ -1039,9 +1071,8 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
-static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
+static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	struct commit_list *cur;
 	int res;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
@@ -1051,10 +1082,12 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		if (save_todo(cur, opts))
+	while (todo_list->current < todo_list->nr) {
+		struct todo_item *item = todo_list->items + todo_list->current;
+		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(cur->item, opts);
+		res = do_pick_commit(item->command, item->commit, opts);
+		todo_list->current++;
 		if (res)
 			return res;
 	}
@@ -1079,38 +1112,46 @@ static int continue_single_pick(void)
 
 static int sequencer_continue(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
 
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
-	if (read_populate_opts(opts) ||
-			read_populate_todo(&todo_list, opts))
+	if (read_populate_opts(opts))
 		return -1;
+	if ((res = read_populate_todo(&todo_list, opts)))
+		goto release_todo_list;
 
 	/* Verify that the conflict has been resolved */
 	if (file_exists(git_path_cherry_pick_head()) ||
 	    file_exists(git_path_revert_head())) {
-		int ret = continue_single_pick();
-		if (ret)
-			return ret;
+		res = continue_single_pick();
+		if (res)
+			goto release_todo_list;
 	}
-	if (index_differs_from("HEAD", 0))
-		return error_dirty_index(opts);
-	todo_list = todo_list->next;
-	return pick_commits(todo_list, opts);
+	if (index_differs_from("HEAD", 0)) {
+		res = error_dirty_index(opts);
+		goto release_todo_list;
+	}
+	todo_list.current++;
+	res = pick_commits(&todo_list, opts);
+release_todo_list:
+	todo_list_release(&todo_list);
+	return res;
 }
 
 static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
-	return do_pick_commit(cmit, opts);
+	return do_pick_commit(opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT, cmit, opts);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
 	unsigned char sha1[20];
-	int i;
+	int i, res;
 
 	if (opts->subcommand == REPLAY_NONE)
 		assert(opts->revs);
@@ -1185,7 +1226,9 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 		return -1;
 	if (save_opts(opts))
 		return -1;
-	return pick_commits(todo_list, opts);
+	res = pick_commits(&todo_list, opts);
+	todo_list_release(&todo_list);
+	return res;
 }
 
 void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 09/25] sequencer: avoid completely different messages for different actions

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index b971ad0..7ba932b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -232,11 +232,8 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(action_name(opts));
 
-	/* Different translation strings for cherry-pick and revert */
-	if (opts->action == REPLAY_PICK)
-		error(_("Your local changes would be overwritten by cherry-pick."));
-	else
-		error(_("Your local changes would be overwritten by revert."));
+	error(_("Your local changes would be overwritten by %s."),
+		action_name(opts));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 10/25] sequencer: get rid of the subcommand field

[Johannes Schindelin]

The subcommands are used exactly once, at the very beginning of
sequencer_pick_revisions(), to determine what to do. This is an
unnecessary level of indirection: we can simply call the correct
function to begin with. So let's do that.

While at it, ensure that the subcommands return an error code so that
they do not have to die() all over the place (bad practice for library
functions...).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 36 ++++++++++++++++--------------------
 sequencer.c      | 35 +++++++++++------------------------
 sequencer.h      | 13 ++++---------
 3 files changed, 31 insertions(+), 53 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 7365559..c9ae4dc 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -71,7 +71,7 @@ static void verify_opt_compatible(const char *me, const char *base_opt, ...)
 		die(_("%s: %s cannot be used with %s"), me, this_opt, base_opt);
 }
 
-static void parse_args(int argc, const char **argv, struct replay_opts *opts)
+static int run_sequencer(int argc, const char **argv, struct replay_opts *opts)
 {
 	const char * const * usage_str = revert_or_cherry_pick_usage(opts);
 	const char *me = action_name(opts);
@@ -115,25 +115,15 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 	if (opts->keep_redundant_commits)
 		opts->allow_empty = 1;
 
-	/* Set the subcommand */
-	if (cmd == 'q')
-		opts->subcommand = REPLAY_REMOVE_STATE;
-	else if (cmd == 'c')
-		opts->subcommand = REPLAY_CONTINUE;
-	else if (cmd == 'a')
-		opts->subcommand = REPLAY_ROLLBACK;
-	else
-		opts->subcommand = REPLAY_NONE;
-
 	/* Check for incompatible command line arguments */
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		char *this_operation;
-		if (opts->subcommand == REPLAY_REMOVE_STATE)
+		if (cmd == 'q')
 			this_operation = "--quit";
-		else if (opts->subcommand == REPLAY_CONTINUE)
+		else if (cmd == 'c')
 			this_operation = "--continue";
 		else {
-			assert(opts->subcommand == REPLAY_ROLLBACK);
+			assert(cmd == 'a');
 			this_operation = "--abort";
 		}
 
@@ -156,7 +146,7 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 				"--edit", opts->edit,
 				NULL);
 
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		opts->revs = NULL;
 	} else {
 		struct setup_revision_opt s_r_opt;
@@ -174,6 +164,14 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 	if (argc > 1)
 		usage_with_options(usage_str, options);
+
+	if (cmd == 'q')
+		return sequencer_remove_state(opts);
+	if (cmd == 'c')
+		return sequencer_continue(opts);
+	if (cmd == 'a')
+		return sequencer_rollback(opts);
+	return sequencer_pick_revisions(opts);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
@@ -185,8 +183,7 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("revert failed"));
 	return res;
@@ -199,8 +196,7 @@ int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("cherry-pick failed"));
 	return res;
diff --git a/sequencer.c b/sequencer.c
index 7ba932b..053e78c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -127,7 +127,7 @@ void *sequencer_entrust(struct replay_opts *opts, void *to_free)
 	return to_free;
 }
 
-static void remove_sequencer_state(const struct replay_opts *opts)
+int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
 	int i;
@@ -141,6 +141,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
 	strbuf_release(&dir);
+
+	return 0;
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -971,7 +973,7 @@ static int rollback_single_pick(void)
 	return reset_for_rollback(head_sha1);
 }
 
-static int sequencer_rollback(struct replay_opts *opts)
+int sequencer_rollback(struct replay_opts *opts)
 {
 	FILE *f;
 	unsigned char sha1[20];
@@ -1006,9 +1008,8 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state(opts);
 	strbuf_release(&buf);
-	return 0;
+	return sequencer_remove_state(opts);
 fail:
 	strbuf_release(&buf);
 	return -1;
@@ -1093,8 +1094,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state(opts);
-	return 0;
+	return sequencer_remove_state(opts);
 }
 
 static int continue_single_pick(void)
@@ -1107,11 +1107,14 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
-static int sequencer_continue(struct replay_opts *opts)
+int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
 	int res;
 
+	if (read_and_refresh_cache(opts))
+		return -1;
+
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
@@ -1150,26 +1153,10 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	unsigned char sha1[20];
 	int i, res;
 
-	if (opts->subcommand == REPLAY_NONE)
-		assert(opts->revs);
-
+	assert(opts->revs);
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	/*
-	 * Decide what to do depending on the arguments; a fresh
-	 * cherry-pick should be handled differently from an existing
-	 * one that is being continued
-	 */
-	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state(opts);
-		return 0;
-	}
-	if (opts->subcommand == REPLAY_ROLLBACK)
-		return sequencer_rollback(opts);
-	if (opts->subcommand == REPLAY_CONTINUE)
-		return sequencer_continue(opts);
-
 	for (i = 0; i < opts->revs->pending.nr; i++) {
 		unsigned char sha1[20];
 		const char *name = opts->revs->pending.objects[i].name;
diff --git a/sequencer.h b/sequencer.h
index 04892a9..0b3950d 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -10,16 +10,8 @@ enum replay_action {
 	REPLAY_PICK
 };
 
-enum replay_subcommand {
-	REPLAY_NONE,
-	REPLAY_REMOVE_STATE,
-	REPLAY_CONTINUE,
-	REPLAY_ROLLBACK
-};
-
 struct replay_opts {
 	enum replay_action action;
-	enum replay_subcommand subcommand;
 
 	/* Boolean options */
 	int edit;
@@ -48,7 +40,7 @@ struct replay_opts {
 	void **owned;
 	int owned_nr, owned_alloc;
 };
-#define REPLAY_OPTS_INIT { -1, -1 }
+#define REPLAY_OPTS_INIT { -1 }
 
 /*
  * Make it the duty of sequencer_remove_state() to release the memory;
@@ -57,6 +49,9 @@ struct replay_opts {
 void *sequencer_entrust(struct replay_opts *opts, void *to_free);
 
 int sequencer_pick_revisions(struct replay_opts *opts);
+int sequencer_continue(struct replay_opts *opts);
+int sequencer_rollback(struct replay_opts *opts);
+int sequencer_remove_state(struct replay_opts *opts);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 11/25] sequencer: refactor the code to obtain a short commit name

[Johannes Schindelin]

Not only does this DRY up the code (providing a better documentation what
the code is about, as well as allowing to change the behavior in a single
place), it also makes it substantially shorter to use the same
functionality in functions to be introduced when we teach the sequencer to
process interactive-rebase's git-rebase-todo file.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 053e78c..0c8dec4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -157,13 +157,18 @@ struct commit_message {
 	const char *message;
 };
 
+static const char *short_commit_name(struct commit *commit)
+{
+	return find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+}
+
 static int get_message(struct commit *commit, struct commit_message *out)
 {
 	const char *abbrev, *subject;
 	int subject_len;
 
 	out->message = logmsg_reencode(commit, NULL, get_commit_output_encoding());
-	abbrev = find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+	abbrev = short_commit_name(commit);
 
 	subject_len = find_commit_subject(out->message, &subject);
 
@@ -647,8 +652,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
-		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
-		      msg.subject);
+		      short_commit_name(commit), msg.subject);
 		print_advice(res == 1, opts);
 		rerere(opts->allow_rerere_auto);
 		goto leave;
@@ -904,9 +908,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
-			find_unique_abbrev(commit->object.oid.hash,
-				DEFAULT_ABBREV),
-			subject_len, subject);
+			short_commit_name(commit), subject_len, subject);
 		unuse_commit_buffer(commit, commit_buffer);
 	}
 	return 0;
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 12/25] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

The `git-rebase-todo` file contains a list of commands. Most of those
commands have the form

	<verb> <sha1> <oneline>

The <oneline> is displayed primarily for the user's convenience, as
rebase -i really interprets only the <verb> <sha1> part. However, there
are *some* places in interactive rebase where the <oneline> is used to
display messages, e.g. for reporting at which commit we stopped.

So let's just remember it when parsing the todo file; we keep a copy of
the entire todo file anyway (to write out the new `done` and
`git-rebase-todo` file just before processing each command), so all we
need to do is remember the begin offsets and lengths.

As we will have to parse and remember the command-line for `exec` commands
later, we do not call the field "oneline" but rather "arg" (and will reuse
that for exec's command-line).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 0c8dec4..ca1961c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -713,6 +713,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	const char *arg;
+	int arg_len;
 	size_t offset_in_buf;
 };
 
@@ -764,6 +766,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
+	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_len = (int)(eol - item->arg);
+
 	if (status < 0)
 		return -1;
 
@@ -905,6 +910,8 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
+		item->arg = NULL;
+		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 13/25] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

In interactive rebases, we commit a little bit differently than the
sequencer did so far: we heed the "author-script", the "message" and
the "amend" files in the .git/rebase-merge/ subdirectory.

Likewise, we may want to edit the commit message *even* when providing
a file containing the suggested commit message. Therefore we change the
code to not even provide a default message when we do not want any, and
to call the editor explicitly.

As interactive rebase's GPG settings are configured differently from
how cherry-pick (and therefore sequencer) handles them, we will leave
support for that to the next commit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 sequencer.h |  3 ++
 2 files changed, 89 insertions(+), 11 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ca1961c..6c35fe8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,19 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+/*
+ * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
+ * GIT_AUTHOR_DATE that will be used for the commit that is currently
+ * being rebased.
+ */
+static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+
+/* We will introduce the 'interactive rebase' mode later */
+static inline int is_rebase_i(const struct replay_opts *opts)
+{
+	return 0;
+}
+
 static const char *get_dir(const struct replay_opts *opts)
 {
 	return git_path_seq_dir();
@@ -377,20 +390,76 @@ static int is_index_unchanged(void)
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
 
+static char **read_author_script(void)
+{
+	struct strbuf script = STRBUF_INIT;
+	int i, count = 0;
+	char *p, *p2, **env;
+	size_t env_size;
+
+	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
+		return NULL;
+
+	for (p = script.buf; *p; p++)
+		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
+			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
+		else if (*p == '\'')
+			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
+		else if (*p == '\n') {
+			*p = '\0';
+			count++;
+		}
+
+	env_size = (count + 1) * sizeof(*env);
+	strbuf_grow(&script, env_size);
+	memmove(script.buf + env_size, script.buf, script.len);
+	p = script.buf + env_size;
+	env = (char **)strbuf_detach(&script, NULL);
+
+	for (i = 0; i < count; i++) {
+		env[i] = p;
+		p += strlen(p) + 1;
+	}
+	env[count] = NULL;
+
+	return env;
+}
+
 /*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
  * author date and name.
+ *
  * If we are revert, or if our cherry-pick results in a hand merge,
  * we had better say that the current user is responsible for that.
+ *
+ * An exception is when sequencer_commit() is called during an
+ * interactive rebase: in that case, we will want to retain the
+ * author metadata.
  */
-static int run_git_commit(const char *defmsg, struct replay_opts *opts,
+int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty)
 {
+	char **env = NULL;
 	struct argv_array array;
 	int rc;
 	const char *value;
 
+	if (is_rebase_i(opts)) {
+		env = read_author_script();
+		if (!env)
+			return error("You have staged changes in your working "
+				"tree. If these changes are meant to be\n"
+				"squashed into the previous commit, run:\n\n"
+				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"If they are meant to go into a new commit, "
+				"run:\n\n"
+				"  git commit $gpg_sign_opt_quoted\n\n"
+				"In both cases, once you're done, continue "
+				"with:\n\n"
+				"  git rebase --continue\n");
+	}
+
 	argv_array_init(&array);
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
@@ -399,14 +468,13 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
 		argv_array_push(&array, "-s");
-	if (!opts->edit) {
-		argv_array_push(&array, "-F");
-		argv_array_push(&array, defmsg);
-		if (!opts->signoff &&
-		    !opts->record_origin &&
-		    git_config_get_value("commit.cleanup", &value))
-			argv_array_push(&array, "--cleanup=verbatim");
-	}
+	if (defmsg)
+		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (opts->edit)
+		argv_array_push(&array, "-e");
+	else if (!opts->signoff && !opts->record_origin &&
+		 git_config_get_value("commit.cleanup", &value))
+		argv_array_push(&array, "--cleanup=verbatim");
 
 	if (allow_empty)
 		argv_array_push(&array, "--allow-empty");
@@ -414,8 +482,11 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt(array.argv, RUN_GIT_CMD);
+	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+			(const char *const *)env);
 	argv_array_clear(&array);
+	free(env);
+
 	return rc;
 }
 
@@ -664,7 +735,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(git_path_merge_msg(), opts, allow);
+		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
+			opts, allow);
 
 leave:
 	free_message(commit, &msg);
@@ -877,6 +949,9 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return 0;
+
 	if (!file_exists(git_path_opts_file()))
 		return 0;
 	/*
diff --git a/sequencer.h b/sequencer.h
index 0b3950d..16deb6c 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -53,6 +53,9 @@ int sequencer_continue(struct replay_opts *opts);
 int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
+int sequencer_commit(const char *defmsg, struct replay_opts *opts,
+			  int allow_empty);
+
 extern const char sign_off_header[];
 
 void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 14/25] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

As we are slowly teaching the sequencer to perform the hard work for
the interactive rebase, we need to read files that were written by
shell scripts.

These files typically contain a single line and are invariably ended
by a line feed (and possibly a carriage return before that). Let's use
a helper to read such files and to remove the line ending.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 6c35fe8..086cd0b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -242,6 +242,37 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
 	return 0;
 }
 
+/*
+ * Reads a file that was presumably written by a shell script, i.e.
+ * with an end-of-line marker that needs to be stripped.
+ *
+ * Returns 1 if the file was read, 0 if it could not be read or does not exist.
+ */
+static int read_oneliner(struct strbuf *buf,
+	const char *path, int skip_if_empty)
+{
+	int orig_len = buf->len;
+
+	if (!file_exists(path))
+		return 0;
+
+	if (strbuf_read_file(buf, path, 0) < 0) {
+		warning_errno("could not read '%s'", path);
+		return 0;
+	}
+
+	if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
+		if (--buf->len > orig_len && buf->buf[buf->len - 1] == '\r')
+			--buf->len;
+		buf->buf[buf->len] = '\0';
+	}
+
+	if (skip_if_empty && buf->len == orig_len)
+		return 0;
+
+	return 1;
+}
+
 static struct tree *empty_tree(void)
 {
 	return lookup_tree(EMPTY_TREE_SHA1_BIN);
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 15/25] sequencer: prepare for rebase -i's GPG settings

[Johannes Schindelin]

The rebase command sports a `--gpg-sign` option that is heeded by the
interactive rebase.

This patch teaches the sequencer that trick, as part of the bigger
effort to make the sequencer the work horse of the interactive rebase.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 42 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 086cd0b..bf02565 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -15,6 +15,7 @@
 #include "merge-recursive.h"
 #include "refs.h"
 #include "argv-array.h"
+#include "quote.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -33,6 +34,11 @@ static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
  * being rebased.
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+/*
+ * The following files are written by git-rebase just after parsing the
+ * command-line (and are only consumed, not modified, by the sequencer).
+ */
+static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -132,6 +138,16 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
+static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
+{
+	static struct strbuf buf = STRBUF_INIT;
+
+	strbuf_reset(&buf);
+	if (opts->gpg_sign)
+		sq_quotef(&buf, "-S%s", opts->gpg_sign);
+	return buf.buf;
+}
+
 void *sequencer_entrust(struct replay_opts *opts, void *to_free)
 {
 	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
@@ -478,17 +494,20 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 
 	if (is_rebase_i(opts)) {
 		env = read_author_script();
-		if (!env)
+		if (!env) {
+			const char *gpg_opt = gpg_sign_opt_quoted(opts);
+
 			return error("You have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"  git commit --amend %s\n\n"
 				"If they are meant to go into a new commit, "
 				"run:\n\n"
-				"  git commit $gpg_sign_opt_quoted\n\n"
+				"  git commit %s\n\n"
 				"In both cases, once you're done, continue "
 				"with:\n\n"
-				"  git rebase --continue\n");
+				"  git rebase --continue\n", gpg_opt, gpg_opt);
+		}
 	}
 
 	argv_array_init(&array);
@@ -980,8 +999,21 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
-	if (is_rebase_i(opts))
+	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
+		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
+			if (!starts_with(buf.buf, "-S"))
+				strbuf_reset(&buf);
+			else {
+				opts->gpg_sign = buf.buf + 2;
+				sequencer_entrust(opts,
+					strbuf_detach(&buf, NULL));
+			}
+		}
+
 		return 0;
+	}
 
 	if (!file_exists(git_path_opts_file()))
 		return 0;
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 16/25] sequencer: allow editing the commit message on a case-by-case basis

[Johannes Schindelin]

In the upcoming commits, we will implement more and more of rebase
-i's functionality. One particular feature of the commands to come is
that some of them allow editing the commit message while others don't,
i.e. we cannot define in the replay_opts whether the commit message
should be edited or not.

Let's add a new parameter to the sequencer_commit() function. Previously,
it was the duty of the caller to ensure that the opts->edit setting
indicates whether to let the user edit the commit message or not,
indicating that it is an "all or nothing" setting, i.e. that the sequencer
wants to let the user edit *all* commit message, or none at all. In the
upcoming rebase -i mode, it will depend on the particular command that is
currently executed, though.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++---
 sequencer.h | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index bf02565..6e9732c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -485,7 +485,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty)
+			  int allow_empty, int edit)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -520,7 +520,7 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
-	if (opts->edit)
+	if (edit)
 		argv_array_push(&array, "-e");
 	else if (!opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
@@ -786,7 +786,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
-			opts, allow);
+			opts, allow, opts->edit);
 
 leave:
 	free_message(commit, &msg);
diff --git a/sequencer.h b/sequencer.h
index 16deb6c..7f5222f 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -54,7 +54,7 @@ int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty);
+			  int allow_empty, int edit);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 17/25] sequencer: support amending commits

[Johannes Schindelin]

This teaches the sequencer_commit() function to take an argument that
will allow us to implement "todo" commands that need to amend the commit
messages ("fixup", "squash" and "reword").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++--
 sequencer.h | 2 +-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 6e9732c..60b522e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -485,7 +485,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit)
+			  int allow_empty, int edit, int amend)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -514,6 +514,8 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
 
+	if (amend)
+		argv_array_push(&array, "--amend");
 	if (opts->gpg_sign)
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
@@ -786,7 +788,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
-			opts, allow, opts->edit);
+			opts, allow, opts->edit, 0);
 
 leave:
 	free_message(commit, &msg);
diff --git a/sequencer.h b/sequencer.h
index 7f5222f..c45f5c4 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -54,7 +54,7 @@ int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit);
+			  int allow_empty, int edit, int amend);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 18/25] sequencer: support cleaning up commit messages

[Johannes Schindelin]

The sequencer_commit() function already knows how to amend commits, and
with this new option, it can also clean up commit messages (i.e. strip
out commented lines). This is needed to implement rebase -i's 'fixup'
and 'squash' commands as sequencer commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 sequencer.h |  3 ++-
 2 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 60b522e..75772b8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -485,7 +485,8 @@ static char **read_author_script(void)
  * author metadata.
  */
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit, int amend)
+			  int allow_empty, int edit, int amend,
+			  int cleanup_commit_message)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -522,9 +523,12 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (cleanup_commit_message)
+		argv_array_push(&array, "--cleanup=strip");
 	if (edit)
 		argv_array_push(&array, "-e");
-	else if (!opts->signoff && !opts->record_origin &&
+	else if (!cleanup_commit_message &&
+		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
 		argv_array_push(&array, "--cleanup=verbatim");
 
@@ -788,7 +792,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
-			opts, allow, opts->edit, 0);
+			opts, allow, opts->edit, 0, 0);
 
 leave:
 	free_message(commit, &msg);
diff --git a/sequencer.h b/sequencer.h
index c45f5c4..688fff1 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -54,7 +54,8 @@ int sequencer_rollback(struct replay_opts *opts);
 int sequencer_remove_state(struct replay_opts *opts);
 
 int sequencer_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit, int amend);
+			  int allow_empty, int edit, int amend,
+			  int cleanup_commit_message);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 19/25] sequencer: remember do_recursive_merge()'s return value

[Johannes Schindelin]

The return value of do_recursive_merge() may be positive (indicating merge
conflicts), so let's OR later error conditions so as not to overwrite them
with 0.

This is not yet a problem, but preparing for the patches to come: we will
teach the sequencer to do rebase -i's job.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 75772b8..7953a05 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -630,7 +630,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	const char *base_label, *next_label;
 	struct commit_message msg = { NULL, NULL, NULL, NULL };
 	struct strbuf msgbuf = STRBUF_INIT;
-	int res, unborn = 0, allow;
+	int res = 0, unborn = 0, allow;
 
 	if (opts->no_commit) {
 		/*
@@ -741,7 +741,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 
 	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
-		res = do_recursive_merge(base, next, base_label, next_label,
+		res |= do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
 			return res;
@@ -750,7 +750,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct commit_list *common = NULL;
 		struct commit_list *remotes = NULL;
 
-		res = write_message(&msgbuf, git_path_merge_msg());
+		res |= write_message(&msgbuf, git_path_merge_msg());
 
 		commit_list_insert(base, &common);
 		commit_list_insert(next, &remotes);
@@ -787,11 +787,12 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 
 	allow = allow_empty(opts, commit);
 	if (allow < 0) {
-		res = allow;
+		res |= allow;
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
+		res |= sequencer_commit(opts->edit ?
+				NULL : git_path_merge_msg(),
 			opts, allow, opts->edit, 0, 0);
 
 leave:
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 20/25] sequencer: left-trim lines read from the script

[Johannes Schindelin]

Interactive rebase's scripts may be indented; we need to handle this
case, too, now that we prepare the sequencer to process interactive
rebases.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 7953a05..5e5d113 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -875,6 +875,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	/* left-trim */
+	bol += strspn(bol, " \t");
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 21/25] sequencer: refactor write_message()

[Johannes Schindelin]

The write_message() function safely writes an strbuf to a file.
Sometimes it is inconvenient to require an strbuf, though: the text to
be written may not be stored in a strbuf, or the strbuf should not be
released after writing.

Let's refactor "safely writing string to a file" into
write_with_lock_file(), and make write_message() use it. The new
function makes it easy to create new convenience function
write_file_gently(); as some of the upcoming callers of this new
function would want to append a newline character, add a flag for it in
write_file_gently(), and thus in write_with_lock_file().

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5e5d113..aa949d4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -242,22 +242,37 @@ static void print_advice(int show_hint, struct replay_opts *opts)
 	}
 }
 
-static int write_message(struct strbuf *msgbuf, const char *filename)
+static int write_with_lock_file(const char *filename,
+				const void *buf, size_t len, int append_eol)
 {
 	static struct lock_file msg_file;
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
 		return error_errno(_("Could not lock '%s'"), filename);
-	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
-		return error_errno(_("Could not write to %s"), filename);
-	strbuf_release(msgbuf);
+	if (write_in_full(msg_fd, buf, len) < 0)
+		return error_errno(_("Could not write to '%s'"), filename);
+	if (append_eol && write(msg_fd, "\n", 1) < 0)
+		return error_errno(_("Could not write eol to '%s"), filename);
 	if (commit_lock_file(&msg_file) < 0)
 		return error(_("Error wrapping up %s."), filename);
 
 	return 0;
 }
 
+static int write_message(struct strbuf *msgbuf, const char *filename)
+{
+	int res = write_with_lock_file(filename, msgbuf->buf, msgbuf->len, 0);
+	strbuf_release(msgbuf);
+	return res;
+}
+
+static int write_file_gently(const char *filename,
+			     const char *text, int append_eol)
+{
+	return write_with_lock_file(filename, text, strlen(text), append_eol);
+}
+
 /*
  * Reads a file that was presumably written by a shell script, i.e.
  * with an end-of-line marker that needs to be stripped.
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 22/25] sequencer: remove overzealous assumption in rebase -i mode

[Johannes Schindelin]

The sequencer was introduced to make the cherry-pick and revert
functionality available as library function, with the original idea
being to extend the sequencer to also implement the rebase -i
functionality.

The test to ensure that all of the commands in the script are identical
to the overall operation does not mesh well with that.

Therefore let's disable the test in rebase -i mode.

While at it, error out early if the "instruction sheet" (i.e. the todo
script) could not be parsed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index aa949d4..5144245 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -963,7 +963,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (!res) {
+	if (res)
+		return error(_("Unusable instruction sheet: %s"), todo_file);
+
+	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
 		int i;
@@ -977,8 +980,6 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("Cannot revert during a cherry-pick."));
 	}
 
-	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 23/25] sequencer: mark action_name() for translation

[Johannes Schindelin]

The definition of this function goes back all the way to 043a449
(sequencer: factor code out of revert builtin, 2012-01-11), long before a
serious effort was made to translate all the error messages.

It is slightly out of the context of the current patch series (whose
purpose it is to re-implement the performance critical parts of the
interactive rebase in C) to make the error messages in the sequencer
translatable, but what the heck. We'll just do it while we're looking at
this part of the code.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5144245..1e7f29e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -176,7 +176,7 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? "revert" : "cherry-pick";
+	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
 }
 
 struct commit_message {
@@ -312,10 +312,10 @@ static struct tree *empty_tree(void)
 static int error_dirty_index(struct replay_opts *opts)
 {
 	if (read_cache_unmerged())
-		return error_resolve_conflict(action_name(opts));
+		return error_resolve_conflict(_(action_name(opts)));
 
 	error(_("Your local changes would be overwritten by %s."),
-		action_name(opts));
+		_(action_name(opts)));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
@@ -333,7 +333,7 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	if (checkout_fast_forward(from, to, 1))
 		return -1; /* the callee should have complained already */
 
-	strbuf_addf(&sb, _("%s: fast-forward"), action_name(opts));
+	strbuf_addf(&sb, _("%s: fast-forward"), _(action_name(opts)));
 
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
@@ -409,7 +409,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
 		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
 		return error(_("%s: Unable to write new index file"),
-			action_name(opts));
+			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
 
 	if (opts->signoff)
@@ -840,14 +840,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	if (read_index_preload(&the_index, NULL) < 0) {
 		rollback_lock_file(&index_lock);
 		return error(_("git %s: failed to read the index"),
-			action_name(opts));
+			_(action_name(opts)));
 	}
 	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, NULL, NULL, NULL);
 	if (the_index.cache_changed && index_fd >= 0) {
 		if (write_locked_index(&the_index, &index_lock, COMMIT_LOCK)) {
 			rollback_lock_file(&index_lock);
 			return error(_("git %s: failed to refresh the index"),
-				action_name(opts));
+				_(action_name(opts)));
 		}
 	}
 	rollback_lock_file(&index_lock);
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 24/25] sequencer: quote filenames in error messages

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3681 bytes --]

This makes the code consistent by fixing quite a couple of error messages.

Suggested by Jakub Narębski.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 1e7f29e..465e018 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -255,7 +255,7 @@ static int write_with_lock_file(const char *filename,
 	if (append_eol && write(msg_fd, "\n", 1) < 0)
 		return error_errno(_("Could not write eol to '%s"), filename);
 	if (commit_lock_file(&msg_file) < 0)
-		return error(_("Error wrapping up %s."), filename);
+		return error(_("Error wrapping up '%s'."), filename);
 
 	return 0;
 }
@@ -955,16 +955,16 @@ static int read_populate_todo(struct todo_list *todo_list,
 	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"), todo_file);
+		return error_errno(_("Could not open '%s'"), todo_file);
 	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		return error(_("Could not read %s."), todo_file);
+		return error(_("Could not read '%s'."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
+		return error(_("Unusable instruction sheet: '%s'"), todo_file);
 
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
@@ -1050,7 +1050,7 @@ static int read_populate_opts(struct replay_opts *opts)
 	 * are pretty certain that it is syntactically correct.
 	 */
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
-		return error(_("Malformed options sheet: %s"),
+		return error(_("Malformed options sheet: '%s'"),
 			git_path_opts_file());
 	return 0;
 }
@@ -1093,7 +1093,7 @@ static int create_seq_dir(void)
 		return -1;
 	}
 	else if (mkdir(git_path_seq_dir(), 0777) < 0)
-		return error_errno(_("Could not create sequencer directory %s"),
+		return error_errno(_("Could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
 }
@@ -1112,12 +1112,12 @@ static int save_head(const char *head)
 	strbuf_addf(&buf, "%s\n", head);
 	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not write to %s"),
+		return error_errno(_("Could not write to '%s'"),
 				   git_path_head_file());
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-		return error(_("Error wrapping up %s."), git_path_head_file());
+		return error(_("Error wrapping up '%s'."), git_path_head_file());
 	}
 	return 0;
 }
@@ -1162,9 +1162,9 @@ int sequencer_rollback(struct replay_opts *opts)
 		return rollback_single_pick();
 	}
 	if (!f)
-		return error_errno(_("cannot open %s"), git_path_head_file());
+		return error_errno(_("cannot open '%s'"), git_path_head_file());
 	if (strbuf_getline_lf(&buf, f)) {
-		error(_("cannot read %s: %s"), git_path_head_file(),
+		error(_("cannot read '%s': %s"), git_path_head_file(),
 		      ferror(f) ?  strerror(errno) : _("unexpected end of file"));
 		fclose(f);
 		goto fail;
@@ -1203,7 +1203,7 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 			todo_list->buf.len - offset) < 0)
 		return error_errno(_("Could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
-		return error(_("Error wrapping up %s."), todo_path);
+		return error(_("Error wrapping up '%s'."), todo_path);
 	return 0;
 }
 
-- 
2.10.0.windows.1.10.g803177d

[PATCH v2 25/25] sequencer: remove bogus hint for translators

[Johannes Schindelin]

When translating error messages, we need to be careful *not* to translate
the todo commands such as "pick", "reword", etc because they are commands,
and Git would not understand translated versions of those commands.

Therefore, translating the commands in the error messages would simply be
misleading.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 465e018..cdff0f1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -697,8 +697,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
 
 	if (parent && parse_commit(parent) < 0)
-		/* TRANSLATORS: The first %s will be "revert" or
-		   "cherry-pick", the second %s a SHA1 */
 		return error(_("%s: cannot parse parent commit %s"),
 			command_to_string(command),
 			oid_to_hex(&parent->object.oid));
-- 
2.10.0.windows.1.10.g803177d

Re: [PATCH v2 25/25] sequencer: remove bogus hint for translators

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> When translating error messages, we need to be careful *not* to translate
> the todo commands such as "pick", "reword", etc because they are commands,
> and Git would not understand translated versions of those commands.
>
> Therefore, translating the commands in the error messages would simply be
> misleading.

This comes from b9c993e0 ("i18n: git-revert literal "me" messages",
2011-02-22) where it said

Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

    i18n: git-revert literal "me" messages
    
    Translate messages that use the `me' variable. These are all error
    messages referencing the command name, so the name shouldn't be
    translated.
    
    Reported-by: Jonathan Nieder <jrnieder@gmail.com>
    Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    Signed-off-by: Junio C Hamano <gitster@pobox.com>

This looks like a positive attempt to remind translators that their
translation must flow with these literal command names that will not
get translated in place, not a bogus hint at all, at least to me.

>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 465e018..cdff0f1 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -697,8 +697,6 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
>  
>  	if (parent && parse_commit(parent) < 0)
> -		/* TRANSLATORS: The first %s will be "revert" or
> -		   "cherry-pick", the second %s a SHA1 */
>  		return error(_("%s: cannot parse parent commit %s"),
>  			command_to_string(command),
>  			oid_to_hex(&parent->object.oid));

Re: [PATCH v2 24/25] sequencer: quote filenames in error messages

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This makes the code consistent by fixing quite a couple of error messages.

Looks OK.  While at it, we may want another one to downcase the
first word, perhaps?

These may not be messages added by your series and can be left
outside this series, but I have to point out that

    if (commit_lock_file(&msg_file) < 0)
        return error(_("Error wrapping up '%s'."), filename);

results in "error: Error wrapping up", which sounds quite funny.

"failed to finalize" or something would flow a bit better, I'd say.

> diff --git a/sequencer.c b/sequencer.c
> index 1e7f29e..465e018 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -255,7 +255,7 @@ static int write_with_lock_file(const char *filename,
>  	if (append_eol && write(msg_fd, "\n", 1) < 0)
>  		return error_errno(_("Could not write eol to '%s"), filename);
>  	if (commit_lock_file(&msg_file) < 0)
> -		return error(_("Error wrapping up %s."), filename);
> +		return error(_("Error wrapping up '%s'."), filename);
>  
>  	return 0;
>  }

Re: [PATCH v2 22/25] sequencer: remove overzealous assumption in rebase -i mode

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> While at it, error out early if the "instruction sheet" (i.e. the todo
> script) could not be parsed.

Sounds good.

Re: [PATCH v2 21/25] sequencer: refactor write_message()

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The write_message() function safely writes an strbuf to a file.
> Sometimes it is inconvenient to require an strbuf, though: the text to
> be written may not be stored in a strbuf, or the strbuf should not be
> released after writing.
>
> Let's refactor "safely writing string to a file" into
> write_with_lock_file(), and make write_message() use it. The new
> function makes it easy to create new convenience function
> write_file_gently(); as some of the upcoming callers of this new
> function would want to append a newline character, add a flag for it in
> write_file_gently(), and thus in write_with_lock_file().

Makes sense.  As an abstraction, "give me strbuf for my sole use,
and I'll trash its contents when I am done" feels like a horrible
helper interface; perhaps that was overly aggressively refactored by
noticing that then-current callers all released the strbuf, but
still feels wrong.

And this makes the underlying helper a lot more useful.

>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 23 +++++++++++++++++++----
>  1 file changed, 19 insertions(+), 4 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 5e5d113..aa949d4 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -242,22 +242,37 @@ static void print_advice(int show_hint, struct replay_opts *opts)
>  	}
>  }
>  
> -static int write_message(struct strbuf *msgbuf, const char *filename)
> +static int write_with_lock_file(const char *filename,
> +				const void *buf, size_t len, int append_eol)
>  {
>  	static struct lock_file msg_file;
>  
>  	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
>  	if (msg_fd < 0)
>  		return error_errno(_("Could not lock '%s'"), filename);
> -	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
> -		return error_errno(_("Could not write to %s"), filename);
> -	strbuf_release(msgbuf);
> +	if (write_in_full(msg_fd, buf, len) < 0)
> +		return error_errno(_("Could not write to '%s'"), filename);
> +	if (append_eol && write(msg_fd, "\n", 1) < 0)
> +		return error_errno(_("Could not write eol to '%s"), filename);
>  	if (commit_lock_file(&msg_file) < 0)
>  		return error(_("Error wrapping up %s."), filename);
>  
>  	return 0;
>  }
>  
> +static int write_message(struct strbuf *msgbuf, const char *filename)
> +{
> +	int res = write_with_lock_file(filename, msgbuf->buf, msgbuf->len, 0);
> +	strbuf_release(msgbuf);
> +	return res;
> +}
> +
> +static int write_file_gently(const char *filename,
> +			     const char *text, int append_eol)
> +{
> +	return write_with_lock_file(filename, text, strlen(text), append_eol);
> +}
> +
>  /*
>   * Reads a file that was presumably written by a shell script, i.e.
>   * with an end-of-line marker that needs to be stripped.

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Interactive rebase's scripts may be indented; we need to handle this
> case, too, now that we prepare the sequencer to process interactive
> rebases.

Hmph, have we ever given the sequencer instructions indented to the
user to edit?  I do not offhand see why we want to be lenient here,
especially only to the left.

>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/sequencer.c b/sequencer.c
> index 7953a05..5e5d113 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -875,6 +875,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  	char *end_of_object_name;
>  	int i, saved, status, padding;
>  
> +	/* left-trim */
> +	bol += strspn(bol, " \t");
> +
>  	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
>  		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
>  			item->command = i;

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Johannes Schindelin]

Hi Junio,

On Sun, 11 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > Interactive rebase's scripts may be indented; we need to handle this
> > case, too, now that we prepare the sequencer to process interactive
> > rebases.
> 
> Hmph, have we ever given the sequencer instructions indented to the
> user to edit?

No.

We also never provide the user with an indented todo script in the
sequencer. Yet we still parse it. Because Postel's Law.

> I do not offhand see why we want to be lenient here,
> especially only to the left.

Postel's Law.

We do not care about the right: that's only comments (onelines, for the
users' pleasure).

Ciao,
Dscho

Re: [PATCH v2 21/25] sequencer: refactor write_message()

[Johannes Sixt]

Am 11.09.2016 um 12:55 schrieb Johannes Schindelin:
> -static int write_message(struct strbuf *msgbuf, const char *filename)
> +static int write_with_lock_file(const char *filename,
> +				const void *buf, size_t len, int append_eol)
>  {
>  	static struct lock_file msg_file;
>
>  	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
>  	if (msg_fd < 0)
>  		return error_errno(_("Could not lock '%s'"), filename);
> -	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
> -		return error_errno(_("Could not write to %s"), filename);
> -	strbuf_release(msgbuf);
> +	if (write_in_full(msg_fd, buf, len) < 0)
> +		return error_errno(_("Could not write to '%s'"), filename);
> +	if (append_eol && write(msg_fd, "\n", 1) < 0)
> +		return error_errno(_("Could not write eol to '%s"), filename);
>  	if (commit_lock_file(&msg_file) < 0)
>  		return error(_("Error wrapping up %s."), filename);
>
>  	return 0;
>  }

The two error paths in the added lines should both

		rollback_lock_file(&msg_file);

, I think. But I do notice that this is not exactly new, so...

-- Hannes

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I do not offhand see why we want to be lenient here,
>> especially only to the left.
>
> Postel's Law.

How would that compare/relate to yagni, though?

Re: [PATCH v2 05/25] sequencer: allow the sequencer to take custody of malloc()ed data

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
> like a one-shot command when it reads its configuration: memory is
> allocated and released only when the command exits.
>
> This is kind of okay for git-cherry-pick, which *is* a one-shot
> command. All the work to make the sequencer its work horse was
> done to allow using the functionality as a library function, though,
> including proper clean-up after use.
>
> This patch introduces an API to pass the responsibility of releasing
> certain memory to the sequencer. Example:
>
> 	const char *label =
> 		sequencer_entrust(opts, xstrfmt("From: %s", email));

I thought we (not just me) were already pretty clear during the last
round of review that we will not want this entrust() thing.

Re: [PATCH v2 01/25] sequencer: use static initializers for replay_opts

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This change is not completely faithful: instead of initializing all fields
> to 0, we choose to initialize command and subcommand to -1 (instead of
> defaulting to REPLAY_REVERT and REPLAY_NONE, respectively). Practically,
> it makes no difference at all, but future-proofs the code to require
> explicit assignments for both fields.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

OK.

Re: [PATCH v2 02/25] sequencer: use memoized sequencer directory path

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  builtin/commit.c |  2 +-
>  sequencer.c      | 11 ++++++-----
>  sequencer.h      |  5 +----
>  3 files changed, 8 insertions(+), 10 deletions(-)

OK.  It's nice to see code reduction.

> -#define SEQ_DIR		"sequencer"
> -#define SEQ_HEAD_FILE	"sequencer/head"
> -#define SEQ_TODO_FILE	"sequencer/todo"
> -#define SEQ_OPTS_FILE	"sequencer/opts"
> +const char *git_path_seq_dir(void);

Re: [PATCH v2 03/25] sequencer: avoid unnecessary indirection

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> -static int read_populate_opts(struct replay_opts **opts)
> +static int read_populate_opts(struct replay_opts *opts)
>  {
>  	if (!file_exists(git_path_opts_file()))
>  		return 0;
> @@ -823,7 +823,7 @@ static int read_populate_opts(struct replay_opts **opts)
>  	 * about this case, though, because we wrote that file ourselves, so we
>  	 * are pretty certain that it is syntactically correct.
>  	 */
> -	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
> +	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
>  		return error(_("Malformed options sheet: %s"),
>  			git_path_opts_file());
>  	return 0;
> @@ -1054,7 +1054,7 @@ static int sequencer_continue(struct replay_opts *opts)
>  
>  	if (!file_exists(git_path_todo_file()))
>  		return continue_single_pick();
> -	if (read_populate_opts(&opts) ||
> +	if (read_populate_opts(opts) ||
>  			read_populate_todo(&todo_list, opts))
>  		return -1;

Good!

Re: [PATCH v2 04/25] sequencer: future-proof remove_sequencer_state()

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +static const char *get_dir(const struct replay_opts *opts)
> +{
> +	return git_path_seq_dir();
> +}

Presumably this is what "In a couple of commits" meant, i.e. opts
will be used soonish.

> -static void remove_sequencer_state(void)
> +static void remove_sequencer_state(const struct replay_opts *opts)
>  {
> -	struct strbuf seq_dir = STRBUF_INIT;
> +	struct strbuf dir = STRBUF_INIT;
>  
> -	strbuf_addstr(&seq_dir, git_path_seq_dir());
> -	remove_dir_recursively(&seq_dir, 0);
> -	strbuf_release(&seq_dir);
> +	strbuf_addf(&dir, "%s", get_dir(opts));
> +	remove_dir_recursively(&dir, 0);
> +	strbuf_release(&dir);
>  }

As long as "who called the sequencer" is the only thing that
determines where the state is kept, this should work fine, I would
think.  I wondered that it would introduce a chicken-and-egg problem
if we had to support "git sequencer --clear-state" command ;-) but
that is not the case.

Good.

Re: [PATCH v2 19/25] sequencer: remember do_recursive_merge()'s return value

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The return value of do_recursive_merge() may be positive (indicating merge
> conflicts), so let's OR later error conditions so as not to overwrite them
> with 0.

Are the untold assumptions as follows?

 - The caller wants to act on positive (not quite an error), zero
   (success) or negative (error);

 - do_recursive_merge() can return positive (success with
   reservation), zero or negative, and the call to it would return
   immediately if it got negative;

 - all other functions that come later may return either zero or negative, and 
   never positive;

 - Hence the caller can be assured that "res" being positive can
   only mean do_recursive_merge() succeeded with reservation and
   everything else succeeded.

This can be extended if the only thing the caller cares about is
positive/zero/negative and it does not care what exact positive
value it gets--in such a case, we can loosen the condition on the
return values from other functions whose return values are OR'ed
together; they may also return positive to signal the same "not
quite an error", i.e. updating the latter two points to

 - all other functions that come later can return positive (success
   with reservation), zero or negative.

 - Hence the caller can be assured that "res" being positive can
   mean nobody failed with negative return, but it is not an
   unconditional success, which is signalled by value "res" being
   0.

I cannot quite tell which is the case, especially what is planned in
the future.  The proposed log message is a good place to explain the
future course this code will take.

> This is not yet a problem, but preparing for the patches to come: we will
> teach the sequencer to do rebase -i's job.

Thanks.

Re: [PATCH v2 18/25] sequencer: support cleaning up commit messages

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> @@ -788,7 +792,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	}
>  	if (!opts->no_commit)
>  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> -			opts, allow, opts->edit, 0);
> +			opts, allow, opts->edit, 0, 0);
>  
>  leave:
>  	free_message(commit, &msg);

So the existing caller did not ask to cleanup


>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit, int amend)
> +			  int allow_empty, int edit, int amend,
> +			  int cleanup_commit_message)
>  {
>  	char **env = NULL;
>  	struct argv_array array;
> @@ -522,9 +523,12 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
>  		argv_array_push(&array, "-s");
>  	if (defmsg)
>  		argv_array_pushl(&array, "-F", defmsg, NULL);
> +	if (cleanup_commit_message)
> +		argv_array_push(&array, "--cleanup=strip");
>  	if (edit)
>  		argv_array_push(&array, "-e");
> -	else if (!opts->signoff && !opts->record_origin &&
> +	else if (!cleanup_commit_message &&
> +		 !opts->signoff && !opts->record_origin &&
>  		 git_config_get_value("commit.cleanup", &value))
>  		argv_array_push(&array, "--cleanup=verbatim");

Which is consistent with this change.  This is a no-op enhancement
for existing callers and makes new callers to ask for cleaning up.

We will see if a hardcoded "--cleanup=strip" is sufficient when we
see new callers (we _know_ it would be sufficient for the first new
caller by definition ;-).

Makes sense.  Thanks.

Re: [PATCH v2 17/25] sequencer: support amending commits

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This teaches the sequencer_commit() function to take an argument that
> will allow us to implement "todo" commands that need to amend the commit
> messages ("fixup", "squash" and "reword").
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 6 ++++--
>  sequencer.h | 2 +-
>  2 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 6e9732c..60b522e 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -485,7 +485,7 @@ static char **read_author_script(void)
>   * author metadata.
>   */
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit)
> +			  int allow_empty, int edit, int amend)
>  {
>  	char **env = NULL;
>  	struct argv_array array;
> @@ -514,6 +514,8 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
>  	argv_array_push(&array, "commit");
>  	argv_array_push(&array, "-n");
>  
> +	if (amend)
> +		argv_array_push(&array, "--amend");
>  	if (opts->gpg_sign)
>  		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
>  	if (opts->signoff)
> @@ -786,7 +788,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	}
>  	if (!opts->no_commit)
>  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> -			opts, allow, opts->edit);
> +			opts, allow, opts->edit, 0);
>  
>  leave:
>  	free_message(commit, &msg);

Hmm, this is more about a comment on 18/25, but I suspect that
"amend" or any opportunity given to the user to futz with the
contents in the editor invites a wish for the result to be treated
with stripspace.

No existing callers use "amend" to call this function, so there is
no change in behaviour, but at the same time, we do not have enough
information to see if 'amend' should by default toggle cleanup.


> diff --git a/sequencer.h b/sequencer.h
> index 7f5222f..c45f5c4 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -54,7 +54,7 @@ int sequencer_rollback(struct replay_opts *opts);
>  int sequencer_remove_state(struct replay_opts *opts);
>  
>  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit);
> +			  int allow_empty, int edit, int amend);
>  
>  extern const char sign_off_header[];

Re: [PATCH v2 10/25] sequencer: get rid of the subcommand field

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The subcommands are used exactly once, at the very beginning of
> sequencer_pick_revisions(), to determine what to do. This is an
> unnecessary level of indirection: we can simply call the correct
> function to begin with. So let's do that.

Makes sense.  And the diffstat is also pleasant to the eyes.

>  builtin/revert.c | 36 ++++++++++++++++--------------------
>  sequencer.c      | 35 +++++++++++------------------------
>  sequencer.h      | 13 ++++---------
>  3 files changed, 31 insertions(+), 53 deletions(-)

Re: [PATCH v2 21/25] sequencer: refactor write_message()

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Am 11.09.2016 um 12:55 schrieb Johannes Schindelin:
>> -static int write_message(struct strbuf *msgbuf, const char *filename)
>> +static int write_with_lock_file(const char *filename,
>> +				const void *buf, size_t len, int append_eol)
>>  {
>>  	static struct lock_file msg_file;
>>
>>  	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
>>  	if (msg_fd < 0)
>>  		return error_errno(_("Could not lock '%s'"), filename);
>> -	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
>> -		return error_errno(_("Could not write to %s"), filename);
>> -	strbuf_release(msgbuf);
>> +	if (write_in_full(msg_fd, buf, len) < 0)
>> +		return error_errno(_("Could not write to '%s'"), filename);
>> +	if (append_eol && write(msg_fd, "\n", 1) < 0)
>> +		return error_errno(_("Could not write eol to '%s"), filename);
>>  	if (commit_lock_file(&msg_file) < 0)
>>  		return error(_("Error wrapping up %s."), filename);
>>
>>  	return 0;
>>  }
>
> The two error paths in the added lines should both
>
> 		rollback_lock_file(&msg_file);
>
> , I think. But I do notice that this is not exactly new, so...

It may not be new for this step, but overall the series is aiming to
libify the stuff, so we should fix fd and lockfile leaks like this
as we notice them.

Thanks.

Re: Git garden shears, was Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Jakub Narębski]

Hello Dscho,

W dniu 11.09.2016 o 10:33, Johannes Schindelin napisał: 
> On Fri, 9 Sep 2016, Jakub Narębski wrote:
[...]

>> When preserving merges, there are (as far as I understand it), two
>> problems:
>>  - what it means to preserve changes (which change to pick,
>>    that is what is the mainline changes rebase is re-applying)
>>  - what are parents of the merge commit (at least one parent
>>    would be usually rewritten)
>>
>> Maybe the internal (and perhaps also user-visible) representation
>> of merge in instruction sheet could use the notation of filter-branch,
>> that is 'map(<sha-1>)'... it could also imply the mainline.
>>
>> That is the instruction in the internal instruction sheet could
>> look like this:
>>
>>   merge -m 1 map(2fd4e1c6...) da39a3ee... \t Merge 'foo' into master  
>>
>>
>> Note that it has nothing to do with this series!
> 
> Right. But I did solve that already. In the Git garden shears [*1*]
> (essentially my New And Improved attempt at recreating branch structures
> while rebasing), I generate and process scripts like this:
> 
> 	mark onto
> 
> 	# Branch: super-cool-feature
> 	rewind onto
> 	pick 00001 feature
> 	pick 00002 documentation
> 	mark super-cool-feature
> 
> 	# Branch: typo-fix
> 	rewind onto
> 	pick 0000a fix a tyop

There probably should be there

 	mark typo-fix

> 
> 	rewind onto
> 	merge -C cafebabe super-cool-feature
> 	merge -C babecafe typo-fix
> 
> 	cleanup super-cool-feature typo-fix
> 
> Of course this will change a little, still, once I get around to implement
> this on top of the rebase--helper.

Do I understand it correctly that it is user-visible instruction sheet, and
not the internal instruction sheet for sequencer?  This looks very nice
and is well readable.

I guess that it needs to be pre-populated by Git based on topology of the
branch being rebased.

As I see, there are three basic topologies of non-linear branch to be
rebased; all else is combination of thereof, or derivative:

1. Merge commit without branching point, that is we need to go
   from the following situation

   *---*---*---#---o---o---o        <-- old base
               \\
                \\=a===b===M===c    <-- branch being rebased
                          /
         ...---x---x---x-/          <-- side branch

  to the following:

   *---*---*---#---o---o---o        
                            \
                             \-a'--b'--M'--c' 
                                      /
         ...---x---x---x-------------/          

I think this case is the only one supported by `--preserve-merges`,
but I may be mistaken - I never had the need to use this feature IRL.

2. Branching point without accompanying merge commit, or in other words
   rebasing many branches tied together; a shrub if you will.  That is,
   we want to go from the following situation:

   *---*---*---#---o---o---o           <-- old base
               \
                \--a---b---c           <-- branch being rebased
                        \
                         \-1           <-- dependent branch

   to the following one:

   *---*---*---#---o---o---o
                            \
                             \--a'--b'--c'
                                     \
                                      \-1'

I don't think Git supports something like that out of the box, but it
is not hard to create something like that "by hand". It is not much
of a problem... unless you forget to rebase the second dependent branch.

3. Branching point with merge point, that is subbranch created and
   merged - an "eye" (it is not a loop in DAG):

   *---*---*---#---o---o---o             <-- old base
               \
                \--a---b---c---M---d     <-- branch being rebased
                    \         /
                     \-1---2-/         [ <-- possibly a branch ]

   All edges are directed edges, with arrows pointing from right to
   left; that is  *---*  is really  *<---*

   The expected result is:

   *---*---*---#---o---o---o
                            \
                             \--a'--b'--c'--M'--d'
                                 \         /
                                  \-1'--2'/

I guess that is the main purpose of your git-garden-shears script,
isn't it?

> 
> For example, I am not so hot about the "merge -C ..." syntax. I'll
> probably split that into a "remerge <SHA-1> <mark>" and a new "merge
> <mark>" command (the latter asking interactively for the merge commit
> message).

There is also an additional complication in that merge commit message
may be *partially* automatically generated. First there is the subject
generated by 'git merge' ("Merge branch 'foo'") or 'git pull <URL>'.
It might have been translated, or extended.  Second there is a place
for branch cover letter. Third, subject to merge.log / merge.summary
there is a shortlog.

From those shortlog should be surely updated to correspond to the
post-rebase state.  The first line could be used to pre-populate
mark lines, but during merge it should be, I think, updated to the
new name of internal branch if it was changed.


As to 'merge -C <sha1> <marker>' vs 'remerge <sha1> <marker>',
I don't have specified opinion.  It would be nice to have one
character shortcuts for insn sheet instructions, to which
'm -C <sha1> <marker>' is more amendable... 

> 
> And also: the cleanup stage should not be necessary, as the "mark"
> commands can accumulate the known marks into a file in the state
> directory.
> 
> But you get the idea.

Right. No need to make user do something that computer can easily
and without errors do.

> 
> No :1 or some such. That's machine readable. But it's utter nonsense for
> user-facing UIs.

Of course. It's all right for machine-facing instructions, like the
'todo' file for the sequencer, or for git-fast-import stream...

> 
> Ciao,
> Dscho
> 
> Footnote *1*:
> https://github.com/git-for-windows/build-extra/blob/master/shears.sh
 
P.S. I wonder if git-imerge[2] requires for integrated branches to have
both linear history for it to work.

[2]: https://github.com/mhagger/git-imerge
     http://softwareswirl.blogspot.com/2013/05/git-imerge-practical-introduction.html
     https://www.youtube.com/watch?v=FMZ2_-Ny_zc

Best,
-- 
Jakub Narębski

Re: Git garden shears, was Re: [PATCH 13/22] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 8819 bytes --]

Hi Kuba,

On Wed, 21 Sep 2016, Jakub Narębski wrote:

> W dniu 11.09.2016 o 10:33, Johannes Schindelin napisał: 
> > On Fri, 9 Sep 2016, Jakub Narębski wrote:
> [...]
> 
> >> When preserving merges, there are (as far as I understand it), two
> >> problems:
> >>  - what it means to preserve changes (which change to pick,
> >>    that is what is the mainline changes rebase is re-applying)
> >>  - what are parents of the merge commit (at least one parent
> >>    would be usually rewritten)
> >>
> >> Maybe the internal (and perhaps also user-visible) representation
> >> of merge in instruction sheet could use the notation of filter-branch,
> >> that is 'map(<sha-1>)'... it could also imply the mainline.
> >>
> >> That is the instruction in the internal instruction sheet could
> >> look like this:
> >>
> >>   merge -m 1 map(2fd4e1c6...) da39a3ee... \t Merge 'foo' into master  
> >>
> >>
> >> Note that it has nothing to do with this series!
> > 
> > Right. But I did solve that already. In the Git garden shears [*1*]
> > (essentially my New And Improved attempt at recreating branch structures
> > while rebasing), I generate and process scripts like this:
> > 
> > 	mark onto
> > 
> > 	# Branch: super-cool-feature
> > 	rewind onto
> > 	pick 00001 feature
> > 	pick 00002 documentation
> > 	mark super-cool-feature
> > 
> > 	# Branch: typo-fix
> > 	rewind onto
> > 	pick 0000a fix a tyop
> 
> There probably should be there
> 
>  	mark typo-fix

Correct. Sorry for the omission.

> > 	rewind onto
> > 	merge -C cafebabe super-cool-feature
> > 	merge -C babecafe typo-fix
> > 
> > 	cleanup super-cool-feature typo-fix
> > 
> > Of course this will change a little, still, once I get around to implement
> > this on top of the rebase--helper.
> 
> Do I understand it correctly that it is user-visible instruction sheet, and
> not the internal instruction sheet for sequencer?  This looks very nice
> and is well readable.

It is intended as that.

Currently I need a little trickery to make this work, as rebase -i does
not understand rewind nor merge. The trick is to re-use the shears.sh
script as editor that then populates the edit script, calls the real
editor, and then installs a temporary alias that gets called for all
custom commands via exec (turning e.g. "rewind abc" into "exec git .r
rewind abc").

The temporary alias point back to the shears script, too, of course.

In the end, I hope to teach the sequencer a variant of this dialect, as
well as the trick to generate such edit scripts.

> I guess that it needs to be pre-populated by Git based on topology of the
> branch being rebased.

Yes. The shears.sh script is in charge of that, and it has to perform a
couple of Git calls to do so.

> As I see, there are three basic topologies of non-linear branch to be
> rebased; all else is combination of thereof, or derivative:
> 
> 1. Merge commit without branching point, that is we need to go
>    from the following situation
> 
>    *---*---*---#---o---o---o        <-- old base
>                \\
>                 \\=a===b===M===c    <-- branch being rebased
>                           /
>          ...---x---x---x-/          <-- side branch
> 
>   to the following:
> 
>    *---*---*---#---o---o---o        
>                             \
>                              \-a'--b'--M'--c' 
>                                       /
>          ...---x---x---x-------------/          

In other words: rebasing a merge commit merging non-rebased commits.

This is not yet supported by the shears script, as it would require logic
that is not only slow in shell script, but also convoluted. IOW this
feature waits for the sequencer to know how to run regular rebase -i
already.

> I think this case is the only one supported by `--preserve-merges`,
> but I may be mistaken - I never had the need to use this feature IRL.

No, -p would handle merges of non-rebased commits as well as merges of
to-be-rebased commits.

> 2. Branching point without accompanying merge commit, or in other words
>    rebasing many branches tied together; a shrub if you will.  That is,
>    we want to go from the following situation:
> 
>    *---*---*---#---o---o---o           <-- old base
>                \
>                 \--a---b---c           <-- branch being rebased
>                         \
>                          \-1           <-- dependent branch
> 
>    to the following one:
> 
>    *---*---*---#---o---o---o
>                             \
>                              \--a'--b'--c'
>                                      \
>                                       \-1'

This is outside the scope of rebase -i (and of the shears), as you are
talking about *parallel* rebases. I don't do that, nor does rebase -i,
rebase -p nor the shears.

> I don't think Git supports something like that out of the box, but it
> is not hard to create something like that "by hand". It is not much
> of a problem... unless you forget to rebase the second dependent branch.

The safer thing to do, of course, is to merge all those tips back into the
branch you then rebase in one go.

> 3. Branching point with merge point, that is subbranch created and
>    merged - an "eye" (it is not a loop in DAG):
> 
>    *---*---*---#---o---o---o             <-- old base
>                \
>                 \--a---b---c---M---d     <-- branch being rebased
>                     \         /
>                      \-1---2-/         [ <-- possibly a branch ]
> 
>    All edges are directed edges, with arrows pointing from right to
>    left; that is  *---*  is really  *<---*
> 
>    The expected result is:
> 
>    *---*---*---#---o---o---o
>                             \
>                              \--a'--b'--c'--M'--d'
>                                  \         /
>                                   \-1'--2'/
> 
> I guess that is the main purpose of your git-garden-shears script,
> isn't it?

Yes.

> > For example, I am not so hot about the "merge -C ..." syntax. I'll
> > probably split that into a "remerge <SHA-1> <mark>" and a new "merge
> > <mark>" command (the latter asking interactively for the merge commit
> > message).
> 
> There is also an additional complication in that merge commit message
> may be *partially* automatically generated. First there is the subject
> generated by 'git merge' ("Merge branch 'foo'") or 'git pull <URL>'.
> It might have been translated, or extended.  Second there is a place
> for branch cover letter. Third, subject to merge.log / merge.summary
> there is a shortlog.

True. The current shears will simply use the original commit message
verbatim. And I think it would be wise to keep this behaviour until the
sequencer knows how to recreate merges.

Once that happens, I cordially invite you to implement any merge commit
message munging mode your heart desires, on top of the straight port of
the shears.

> From those shortlog should be surely updated to correspond to the
> post-rebase state.  The first line could be used to pre-populate
> mark lines, but during merge it should be, I think, updated to the
> new name of internal branch if it was changed.

Surely. Or not so surely. Too unsurely, for sure, to be discussed without
a dedicated patch series submitted by a dedicated person requiring that
feature.

> As to 'merge -C <sha1> <marker>' vs 'remerge <sha1> <marker>',
> I don't have specified opinion.  It would be nice to have one
> character shortcuts for insn sheet instructions, to which
> 'm -C <sha1> <marker>' is more amendable... 

I am pretty certain that you will find yourself disagreeing with that
statement once you used the shears for a while.

I use it for over a year now, and I *never* had to type any "merge -C"
command (i.e. the command I would call "remerge"). The reason is that you
simply move those generated lines, and that there is no scenario I
encountered where you will want to insert a merge manually: you simply
merge all you want *before* or *after* rebasing.

As to "merge <marker>": those are quite common, in particular when
splitting patch series into parts.

> > No :1 or some such. That's machine readable. But it's utter nonsense for
> > user-facing UIs.
> 
> Of course. It's all right for machine-facing instructions, like the
> 'todo' file for the sequencer, or for git-fast-import stream...

The 'todo' file for the sequencer *should* be human-friendly. There is no
reason not to make it so, in particular when you can simply reuse the refs
machinery for marks, and when it facilitates debugging (and I can testify
to that: it does, oh yeah, it does...).

Ciao,
Dscho

Re: [PATCH v2 05/25] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Schindelin]

Hi Junio,

On Mon, 12 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
> > like a one-shot command when it reads its configuration: memory is
> > allocated and released only when the command exits.
> >
> > This is kind of okay for git-cherry-pick, which *is* a one-shot
> > command. All the work to make the sequencer its work horse was
> > done to allow using the functionality as a library function, though,
> > including proper clean-up after use.
> >
> > This patch introduces an API to pass the responsibility of releasing
> > certain memory to the sequencer. Example:
> >
> > 	const char *label =
> > 		sequencer_entrust(opts, xstrfmt("From: %s", email));
> 
> I thought we (not just me) were already pretty clear during the last
> round of review that we will not want this entrust() thing.

That does not match my understanding.

The problem is that we are building functionality for libgit.a, not merely
for a builtin that we know will simply exit() and take all allocated
memory with it.

The additional problem is that the sequencer was *already* meant for
libgit.a, yet simply strdup()s data left and right and assigns it to const
fields, purposefully wasting memory.

Sure, I can leave those memory leaks in, but then I also have to introduce
new ones via the rebase -i support.

If you prefer to accept such sloppy work, I will change it of course,
feeling dirty that it has my name on it.

Ciao,
Dscho

Re: [PATCH v2 04/25] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

Hi Junio,

On Mon, 12 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +static const char *get_dir(const struct replay_opts *opts)
> > +{
> > +	return git_path_seq_dir();
> > +}
> 
> Presumably this is what "In a couple of commits" meant, i.e. opts
> will be used soonish.

Exactly. The sequencer code was taught to use a state directory different
from rebase -i's (even if it quite clearly imitated rebase -i's approach),
to allow for rebase -i to run at the same time as the sequencer (by
calling it via cherry-pick).

So we need to be able to use different seq_dir locations, depending on the
mode we are running in.

I briefly considered consolidating them and using .git/rebase-merge/ as
state directory also for cherry-pick/revert, but that would cause
problems: I am surely not the only user who cherry-picks commits manually
while running interactive rebases.

Ciao,
Dscho

Re: [PATCH v2 19/25] sequencer: remember do_recursive_merge()'s return value

[Johannes Schindelin]

Hi Junio,

On Mon, 12 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The return value of do_recursive_merge() may be positive (indicating merge
> > conflicts), so let's OR later error conditions so as not to overwrite them
> > with 0.
> 
> Are the untold assumptions as follows?
> 
>  - The caller wants to act on positive (not quite an error), zero
>  (success) or negative (error);
> 
>  - do_recursive_merge() can return positive (success with reservation),
>  zero or negative, and the call to it would return immediately if it got
>  negative;
> 
>  - all other functions that come later may return either zero or
>  negative, and never positive;
> 
>  - Hence the caller can be assured that "res" being positive can only
>  mean do_recursive_merge() succeeded with reservation and everything
>  else succeeded.

More or less. The idea is that there are problems with the merge, and
there are fatal errors.

Alex and I chose to stay close to the original Python implementation when
we decided that merge_trees() should return 1 for success and 0 for
failure. In hindsight, I regret that, as it disagrees with the C
convention to return 0 for success. However, it would have been yet
another mistake to return -1 in case of merge conflicts: the function did
not have a problem (such as out-of-memory or some such).

I can only guess that the do_recursive_merge() function tried to undo the
damage by reversing the logic: 0 for success, 1 for unclean merge. It is
at least more in line with the C convention.

So much so, in fact, that we can still use negative values to indicate
fatal errors that should be handled accordingly.

> This can be extended if the only thing the caller cares about is
> positive/zero/negative and it does not care what exact positive
> value it gets--in such a case, we can loosen the condition on the
> return values from other functions whose return values are OR'ed
> together; they may also return positive to signal the same "not
> quite an error", i.e. updating the latter two points to
> 
>  - all other functions that come later can return positive (success
>    with reservation), zero or negative.
> 
>  - Hence the caller can be assured that "res" being positive can
>    mean nobody failed with negative return, but it is not an
>    unconditional success, which is signalled by value "res" being
>    0.
> 
> I cannot quite tell which is the case, especially what is planned in
> the future.  The proposed log message is a good place to explain the
> future course this code will take.

Okay, I will try to come up with a better commit message.

Actually, come to think of it, I will change the patch, as it is too
confusing. What I want is to preserve a positive return value in case of
merge conflicts, and that is exactly what I should do instead of playing
games with the Boolean OR operator.

Ciao,
Dscho

Re: [PATCH v2 17/25] sequencer: support amending commits

[Johannes Schindelin]

Hi Junio,

On Mon, 12 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > This teaches the sequencer_commit() function to take an argument that
> > will allow us to implement "todo" commands that need to amend the commit
> > messages ("fixup", "squash" and "reword").
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 6 ++++--
> >  sequencer.h | 2 +-
> >  2 files changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/sequencer.c b/sequencer.c
> > index 6e9732c..60b522e 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -485,7 +485,7 @@ static char **read_author_script(void)
> >   * author metadata.
> >   */
> >  int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> > -			  int allow_empty, int edit)
> > +			  int allow_empty, int edit, int amend)
> >  {
> >  	char **env = NULL;
> >  	struct argv_array array;
> > @@ -514,6 +514,8 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
> >  	argv_array_push(&array, "commit");
> >  	argv_array_push(&array, "-n");
> >  
> > +	if (amend)
> > +		argv_array_push(&array, "--amend");
> >  	if (opts->gpg_sign)
> >  		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
> >  	if (opts->signoff)
> > @@ -786,7 +788,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  	}
> >  	if (!opts->no_commit)
> >  		res = sequencer_commit(opts->edit ? NULL : git_path_merge_msg(),
> > -			opts, allow, opts->edit);
> > +			opts, allow, opts->edit, 0);
> >  
> >  leave:
> >  	free_message(commit, &msg);
> 
> Hmm, this is more about a comment on 18/25, but I suspect that
> "amend" or any opportunity given to the user to futz with the
> contents in the editor invites a wish for the result to be treated
> with stripspace.

The point of separating the cleanup_commit_message from the edit flag is
to allow final fixup commands to stripspace, even without letting the user
edit the message.

So while you are correct that "edit != 0" should imply
"cleanup_commit_message != 0", I would rather keep that explicit.

> No existing callers use "amend" to call this function, so there is
> no change in behaviour, but at the same time, we do not have enough
> information to see if 'amend' should by default toggle cleanup.

It should not. Non-final fixup/squash commands *need* to keep the
comment lines.

Keeping things explicit makes it easier to understand the flow, methinks.

Ciao,
Dscho

Re: [PATCH v2 21/25] sequencer: refactor write_message()

[Johannes Schindelin]

Hi Junio & Hannes,

On Thu, 15 Sep 2016, Junio C Hamano wrote:

> Johannes Sixt <j6t@kdbg.org> writes:
> 
> > Am 11.09.2016 um 12:55 schrieb Johannes Schindelin:
> >> -static int write_message(struct strbuf *msgbuf, const char *filename)
> >> +static int write_with_lock_file(const char *filename,
> >> +				const void *buf, size_t len, int append_eol)
> >>  {
> >>  	static struct lock_file msg_file;
> >>
> >>  	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
> >>  	if (msg_fd < 0)
> >>  		return error_errno(_("Could not lock '%s'"), filename);
> >> -	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
> >> -		return error_errno(_("Could not write to %s"), filename);
> >> -	strbuf_release(msgbuf);
> >> +	if (write_in_full(msg_fd, buf, len) < 0)
> >> +		return error_errno(_("Could not write to '%s'"), filename);
> >> +	if (append_eol && write(msg_fd, "\n", 1) < 0)
> >> +		return error_errno(_("Could not write eol to '%s"), filename);
> >>  	if (commit_lock_file(&msg_file) < 0)
> >>  		return error(_("Error wrapping up %s."), filename);
> >>
> >>  	return 0;
> >>  }
> >
> > The two error paths in the added lines should both
> >
> > 		rollback_lock_file(&msg_file);
> >
> > , I think. But I do notice that this is not exactly new, so...
> 
> It may not be new for this step, but overall the series is aiming to
> libify the stuff, so we should fix fd and lockfile leaks like this
> as we notice them.

Makes sense, even for the final commit_lock_file().

Ciao,
Dscho

Re: [PATCH v2 04/25] sequencer: future-proof remove_sequencer_state()

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> I briefly considered consolidating them and using .git/rebase-merge/ as
> state directory also for cherry-pick/revert, but that would cause
> problems: I am surely not the only user who cherry-picks commits manually
> while running interactive rebases.

Good thinking.

Re: [PATCH v2 19/25] sequencer: remember do_recursive_merge()'s return value

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Actually, come to think of it, I will change the patch, as it is too
> confusing. What I want is to preserve a positive return value in case of
> merge conflicts, and that is exactly what I should do instead of playing
> games with the Boolean OR operator.

That would be good; that was exactly the confusion I felt that led
to my comments.

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Johannes Schindelin]

Hi Junio,

On Mon, 12 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> I do not offhand see why we want to be lenient here,
> >> especially only to the left.
> >
> > Postel's Law.
> 
> How would that compare/relate to yagni, though?

I did need it, though. Blame it on being overworked or simply tired: I
ended up inserting a spurious space before a "fixup" command. This command
was handled as intended by the scripted rebase -i, and with the patch in
question the rebase--helper agreed, too.

Note: we have no problem to the right because we do handle variable-length
whitespace between command and SHA-1, and we do not care one iota about
the exact form of the oneline (hence right-stripping is not necessary).

Ciao,
Dscho

Re: [PATCH v2 24/25] sequencer: quote filenames in error messages

[Johannes Schindelin]

Hi Junio,

On Sun, 11 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > This makes the code consistent by fixing quite a couple of error messages.
> 
> Looks OK.  While at it, we may want another one to downcase the
> first word, perhaps?
> 
> These may not be messages added by your series and can be left
> outside this series, but I have to point out that
> 
>     if (commit_lock_file(&msg_file) < 0)
>         return error(_("Error wrapping up '%s'."), filename);
> 
> results in "error: Error wrapping up", which sounds quite funny.
> 
> "failed to finalize" or something would flow a bit better, I'd say.

Fair enough. I added a patch to make it so.

Thanks,
Dscho

Re: [PATCH v2 25/25] sequencer: remove bogus hint for translators

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1534 bytes --]

Hi Junio,

On Sun, 11 Sep 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > When translating error messages, we need to be careful *not* to translate
> > the todo commands such as "pick", "reword", etc because they are commands,
> > and Git would not understand translated versions of those commands.
> >
> > Therefore, translating the commands in the error messages would simply be
> > misleading.
> 
> This comes from b9c993e0 ("i18n: git-revert literal "me" messages",
> 2011-02-22) where it said
> 
> Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> 
>     i18n: git-revert literal "me" messages
>     
>     Translate messages that use the `me' variable. These are all error
>     messages referencing the command name, so the name shouldn't be
>     translated.
>     
>     Reported-by: Jonathan Nieder <jrnieder@gmail.com>
>     Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>     Signed-off-by: Junio C Hamano <gitster@pobox.com>
> 
> This looks like a positive attempt to remind translators that their
> translation must flow with these literal command names that will not
> get translated in place, not a bogus hint at all, at least to me.

Alright, I misread it to mean: "You need to translate these terms".

I fixed it by skipping this patch, and amending the "revamp the todo
parsing" patch: the first %s now actually refers to the much more helpful
todo command rather than the overall Git command.

Thanks,
Dscho

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Johannes Sixt]

Am 06.10.2016 um 15:08 schrieb Johannes Schindelin:
> Hi Junio,
> 
> On Mon, 12 Sep 2016, Junio C Hamano wrote:
> 
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>
>>>> I do not offhand see why we want to be lenient here,
>>>> especially only to the left.
>>>
>>> Postel's Law.
>>
>> How would that compare/relate to yagni, though?
> 
> I did need it, though. Blame it on being overworked or simply tired: I
> ended up inserting a spurious space before a "fixup" command. This command
> was handled as intended by the scripted rebase -i, and with the patch in
> question the rebase--helper agreed, too.
> 
> Note: we have no problem to the right because we do handle variable-length
> whitespace between command and SHA-1, and we do not care one iota about
> the exact form of the oneline (hence right-stripping is not necessary).

The last sentence is not entirely correct. See the patch below. You
may pick up the idea in one form or another, or just queue the patch
near the end of your series.

Let me take the opportunity to say

   Thank You Very Much!

for the new implementation of rebase -i. I'm using your branch
rebase-i-extra, and it is such a pleasure to work with on Windows
compared to the old fully scripted version.

---- 8< ----
[PATCH] sequencer: strip CR from the end of exec insns

It is not unheard of that editors on Windows write CRLF even if the file
originally had only LF. This is particularly awkward for exec lines of a
rebase -i todo sheet. Take for example the insn "exec echo": The shell
script parser (either of the sequencer or of the shell that is invoked,
I do not know) splits at the LF and leaves the CR attached to "echo",
which leads to the unknown command "echo\r".

Work it around by stripping CR before the command specified with exec is
passed to the shell.

This happens to fix t9903.14 and .15 in my environment: the insn sheet
constructed here contains CRLF thanks to MSYS1's bash cleverness.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 sequencer.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index daf8f13..6961820 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -2005,8 +2005,11 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 		}
 		else if (item->command == TODO_EXEC) {
 			char *end_of_arg = (char *)(item->arg + item->arg_len);
-			int saved = *end_of_arg;
+			int saved;
 
+			while (end_of_arg != item->arg && end_of_arg[-1] == '\r')
+				end_of_arg--;
+			saved = *end_of_arg;
 			*end_of_arg = '\0';
 			res = do_exec(item->arg);
 			*end_of_arg = saved;
-- 
2.10.0.343.g37bc62b

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Let me take the opportunity to say
>
>    Thank You Very Much!
>
> for the new implementation of rebase -i. I'm using your branch
> rebase-i-extra, and it is such a pleasure to work with on Windows
> compared to the old fully scripted version.

Thanks for testing.  

Having more guinea pigs ^W ^W testers before the series is reviewed
here would be a nice boost and would hopefully encourage more
reviewers to help the series into a good shape to be upstreamed ;-)

> ---- 8< ----
> [PATCH] sequencer: strip CR from the end of exec insns
>
> It is not unheard of that editors on Windows write CRLF even if the file
> originally had only LF. This is particularly awkward for exec lines of a
> rebase -i todo sheet. Take for example the insn "exec echo": The shell
> script parser (either of the sequencer or of the shell that is invoked,
> I do not know) splits at the LF and leaves the CR attached to "echo",
> which leads to the unknown command "echo\r".

Interesting find.  So it's not just ltrim is being lenient only to
end-user typo, but not doing rtrim can actively hurt ;-)

> Work it around by stripping CR before the command specified with exec is
> passed to the shell.

Makes sense.

Re: [PATCH v2 05/25] sequencer: allow the sequencer to take custody of malloc()ed data

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> If you prefer to accept such sloppy work, I will change it of course,
> feeling dirty that it has my name on it.

I do want neither leaks nor sloppyness, and I thought that was
understood by everybody, hence I thought the last round made it
clear that _entrust() must not exist.

Let's try again.

We manage lifetime of a field in a structure in one of three ways in
our codebase [*1*].

 * A field can always point at a borrowed piece of memory that the
   destruction of the containing structure or re-assignment to the
   field do not have to free the current value of the field.  This
   is a minority, simply because it is rare for a field to be always
   able to borrow from others.  The destruction of the containing
   structure or re-assignment to the field needs to do nothing
   special.

 * A field can own the piece of memory that it points at.  The
   destruction of the containing structure or re-assignment to the
   field needs to free the current value of the field [*2*].  A
   field that can be assigned a value from the configuration (which
   is typically xstrdup()'ed) is an example of such a field, and if
   a command line option also can assign to the field, we'd need to
   xstrdup() it, even though we know we could borrow from argv[],
   because cleaning-up needs to be able to free(3) it.

 * A field can sometimes own and sometimes borrow the memory, and it
   is accompanied by another field to tell which case it is, so that
   cleaning-up can tell when it needs to be free(3)d.  This is a
   minority case, and we generally avoid it especially in modern
   code for small allocation, as it makes the lifetime rule more
   complex than it is worth.

The _entrust() thing may have been an excellent fourth option if it
were cost-free.  xstrdup() that the second approach necessitates for
literal strings (like part of argv[]) would become unnecessary and
we can treat as if all the fields in a structure were all borrowing
the memory from elsewhere (in fact, _entrust() creates the missing
owners of pieces of memory that need to be freed later); essentially
it turns a "mixed ownership" case into the first approach.

But it is not cost-free.  The mechanism needs to allocate memory to
become the missing owner and keep track of which pieces of memory
need to be freed later, and the resulting code does not become any
easier to follow.  The programmer still needs to know which ones to
_entrust() just like the programmer in the model #2 needs to make
sure xstrdup() is done for appropriate pieces of memory--the only
difference is that an _entrust() programmer needs to mark the pieces
of memory to be freed, while a #2 programmer needs to xstrdup() the
pieces of memory that are being "borrowed".

So let's not add the fourth way to our codebase.  "mixed ownership"
case should be turned into "field owns the memory", i.e. approach #2.


[Footnotes]

*1* It is normal for different fields in the same structure follow
    different lifetime rules.

*2* Unless leaks are allowed, that is.  I think we have instances
    where "git cmd --opt=A --opt=B" allocates and stores a new piece
    of memory that is computed based on A and store it to a field,
    and then overwrites the field with another allocation of a value
    based on B without freeing old value, saying "the caller can
    avoid passing the same thing twice, and such a leak is miniscule
    anyway".  That is not nice, and we've been reducing them over
    time.

Re: [PATCH v2 05/25] sequencer: allow the sequencer to take custody of malloc()ed data

[Jakub Narębski]

W dniu 06.10.2016 o 21:23, Junio C Hamano pisze:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
>> If you prefer to accept such sloppy work, I will change it of course,
>> feeling dirty that it has my name on it.
> 
> I do want neither leaks nor sloppyness, and I thought that was
> understood by everybody, hence I thought the last round made it
> clear that _entrust() must not exist.

Also, the *_entrust() mechanism in v1 was quite sequencer-specific,
rather than be a generic mechanism.

> 
> Let's try again.
> 
> We manage lifetime of a field in a structure in one of three ways in
> our codebase [*1*].
> 
>  * A field can always point at a borrowed piece of memory that the
>    destruction of the containing structure or re-assignment to the
>    field do not have to free the current value of the field.  This
>    is a minority, simply because it is rare for a field to be always
>    able to borrow from others.  The destruction of the containing
>    structure or re-assignment to the field needs to do nothing
>    special.
> 
>  * A field can own the piece of memory that it points at.  The
>    destruction of the containing structure or re-assignment to the
>    field needs to free the current value of the field [*2*].  A
>    field that can be assigned a value from the configuration (which
>    is typically xstrdup()'ed) is an example of such a field, and if
>    a command line option also can assign to the field, we'd need to
>    xstrdup() it, even though we know we could borrow from argv[],
>    because cleaning-up needs to be able to free(3) it.
> 
>  * A field can sometimes own and sometimes borrow the memory, and it
>    is accompanied by another field to tell which case it is, so that
>    cleaning-up can tell when it needs to be free(3)d.  This is a
>    minority case, and we generally avoid it especially in modern
>    code for small allocation, as it makes the lifetime rule more
>    complex than it is worth.

Great writeup!

> The _entrust() thing may have been an excellent fourth option if it
> were cost-free.  xstrdup() that the second approach necessitates for
> literal strings (like part of argv[]) would become unnecessary and
> we can treat as if all the fields in a structure were all borrowing
> the memory from elsewhere (in fact, _entrust() creates the missing
> owners of pieces of memory that need to be freed later); essentially
> it turns a "mixed ownership" case into the first approach.
> 
> But it is not cost-free.  The mechanism needs to allocate memory to
> become the missing owner and keep track of which pieces of memory
> need to be freed later, and the resulting code does not become any
> easier to follow.  The programmer still needs to know which ones to
> _entrust() just like the programmer in the model #2 needs to make
> sure xstrdup() is done for appropriate pieces of memory--the only
> difference is that an _entrust() programmer needs to mark the pieces
> of memory to be freed, while a #2 programmer needs to xstrdup() the
> pieces of memory that are being "borrowed".
> 
> So let's not add the fourth way to our codebase.  "mixed ownership"
> case should be turned into "field owns the memory", i.e. approach #2.

On the other hand the _entrust() mechanism might be a good solution
if the amount of memory was large, for example order of magnitude more
than what would be needed to keep ownership info *and* borrowing would
not be possible for some reason.

But the _entrust() mechanism reminds me on one hand side of memory
debuggers like dmalloc, Electric Fence (eFence), or valgrind; combined
a bit with garbage collection.  Namely, when we are in a library call,
record all allocations (perhaps by redirecting alloc functions), then
free those resources at the exit of library call.
 
> 
> [Footnotes]
> 
> *1* It is normal for different fields in the same structure follow
>     different lifetime rules.
> 
> *2* Unless leaks are allowed, that is.  I think we have instances
>     where "git cmd --opt=A --opt=B" allocates and stores a new piece
>     of memory that is computed based on A and store it to a field,
>     and then overwrites the field with another allocation of a value
>     based on B without freeing old value, saying "the caller can
>     avoid passing the same thing twice, and such a leak is miniscule
>     anyway".  That is not nice, and we've been reducing them over
>     time.
> 

Re: [PATCH v2 05/25] sequencer: allow the sequencer to take custody of malloc()ed data

[Junio C Hamano]

Jakub Narębski <jnareb@gmail.com> writes:

>> We manage lifetime of a field in a structure in one of three ways in
>> our codebase [*1*].
>> 
>>  ...
>>  * A field can sometimes own and sometimes borrow the memory, and it
>>    is accompanied by another field to tell which case it is, so that
>>    cleaning-up can tell when it needs to be free(3)d.  This is a
>>    minority case, and we generally avoid it especially in modern
>>    code for small allocation, as it makes the lifetime rule more
>>    complex than it is worth.
> ...
> On the other hand the _entrust() mechanism might be a good solution
> if the amount of memory was large, for example order of magnitude more
> than what would be needed to keep ownership info *and* borrowing would
> not be possible for some reason.

We have approach #3 exactly for that usage pattern.

Re: [PATCH v2 05/25] sequencer: allow the sequencer to take custody of malloc()ed data

[Johannes Schindelin]

Hi Junio,

On Thu, 6 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > If you prefer to accept such sloppy work, I will change it of course,
> > feeling dirty that it has my name on it.
> 
> I do want neither leaks nor sloppyness, and I thought that was
> understood by everybody, hence I thought the last round made it
> clear that _entrust() must not exist.

Well, you leave me with but one choice. With shame in my face I will send
out the next re-roll with this construct:

@@ -796,24 +1002,52 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
                opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
        else if (!strcmp(key, "options.mainline"))
                opts->mainline = git_config_int(key, value);
-       else if (!strcmp(key, "options.strategy"))
-               git_config_string(&opts->strategy, key, value);
-       else if (!strcmp(key, "options.gpg-sign"))
-               git_config_string(&opts->gpg_sign, key, value);
+       else if (!strcmp(key, "options.strategy")) {
+               if (!value)
+                       config_error_nonbool(key);
+               else {
+                       free(opts->strategy);
+                       opts->strategy = xstrdup(value);
+               }
+       }
+       else if (!strcmp(key, "options.gpg-sign")) {
+               if (!value)
+                       config_error_nonbool(key);
+               else {
+                       free(opts->gpg_sign);
+                       opts->gpg_sign = xstrdup(value);
+               }
+       }
...
@@ -37,18 +26,32 @@ struct replay_opts {

        int mainline;

-       const char *gpg_sign;
+       char *gpg_sign;

        /* Merge strategy */
-       const char *strategy;
-       const char **xopts;
+       char *strategy;
+       char **xopts;

(this is not the complete diff, of course, but you get the idea.)

Ciao,
Dscho

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Johannes Schindelin]

Hi Hannes,

On Thu, 6 Oct 2016, Johannes Sixt wrote:

> [PATCH] sequencer: strip CR from the end of exec insns
> 
> It is not unheard of that editors on Windows write CRLF even if the file
> originally had only LF. This is particularly awkward for exec lines of a
> rebase -i todo sheet. Take for example the insn "exec echo": The shell
> script parser (either of the sequencer or of the shell that is invoked,
> I do not know) splits at the LF and leaves the CR attached to "echo",
> which leads to the unknown command "echo\r".
> 
> Work it around by stripping CR before the command specified with exec is
> passed to the shell.
> 
> This happens to fix t9903.14 and .15 in my environment: the insn sheet
> constructed here contains CRLF thanks to MSYS1's bash cleverness.

Good point. I decided to do it at a different level, though:
parse_insn_line() should already receive the line without trailing
end-of-line markers (this was already the case for LF-only todo scripts).

I reused your commit message and touched it up a bit, hope you don't mind!

Ciao,
Dscho

Re: [PATCH v2 20/25] sequencer: left-trim lines read from the script

[Johannes Sixt]

Am 09.10.2016 um 10:57 schrieb Johannes Schindelin:
> Good point. I decided to do it at a different level, though:
> parse_insn_line() should already receive the line without trailing
> end-of-line markers (this was already the case for LF-only todo scripts).
>
> I reused your commit message and touched it up a bit, hope you don't mind!

I don't mind at all.

Thanks,
-- Hannes

[PATCH v3 00/25] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

This patch series marks the  '4' in the countdown to speed up rebase -i
by implementing large parts in C. It is based on the `libify-sequencer`
patch series that I submitted last week.

The patches in this series merely prepare the sequencer code for the
next patch series that actually teaches the sequencer to run an
interactive rebase.

The reason to split these two patch series is simple: to keep them at a
sensible size.

The two patch series after that are much smaller: a two-patch "series"
that switches rebase -i to use the sequencer (except with --root or
--preserve-merges), and a couple of patches to move several pretty
expensive script processing steps to C (think: autosquash).

The end game of this patch series is a git-rebase--helper that makes
rebase -i 5x faster on Windows (according to t/perf/p3404). Travis says
that even MacOSX and Linux benefit (4x and 3x, respectively).

I have been working on this since early February, whenever time allowed,
and it is time to put it into the users' hands. To that end, I will most
likely submit the remaining three patch series in the next two days, and
integrate the whole shebang into Git for Windows 2.10.0.

Therefore I would be most grateful for every in-depth review.

Changes vs v2:

- dramatically simplified the logic to retain do_recursive_merge()'s
  return value, even when positive (indicating merge conflicts).

- ensured that write_message() rolls back locked files in case of
  errors.

- added a patch to downcase all first letters of sequencer's error
  messages, as suggested by Junio.

- replaced 25/25 ("sequencer: remove bogus hint for translators") by a
  fix to 8/25 ("completely revamp todo parsing"): the first %s is no
  longer "revert" or "cherry-pick", but a "todo" command.

- backed out the sequencer_entrust() function and uglified the code to
  always duplicate the option values for the sake of being able to
  properly releasing them afterwards.

- CR/LFs are now handled like LFs in todo scripts: they are stripped
  (thanks to Hannes Sixt for pointing that out).

- unexported sequencer_commit() (it can safely remain a static
  function). While at it, renamed it back to run_git_commit() (the
  unspecific name does not hurt if it remains private to sequencer.c).

- clarified in a code comment what read_author_script() does.

- marked for translation the last remaining error message in sequencer.c
  that was not yet marked for translation.


Johannes Schindelin (25):
  sequencer: use static initializers for replay_opts
  sequencer: use memoized sequencer directory path
  sequencer: avoid unnecessary indirection
  sequencer: future-proof remove_sequencer_state()
  sequencer: eventually release memory allocated for the option values
  sequencer: future-proof read_populate_todo()
  sequencer: completely revamp the "todo" script parsing
  sequencer: strip CR from the todo script
  sequencer: avoid completely different messages for different actions
  sequencer: get rid of the subcommand field
  sequencer: refactor the code to obtain a short commit name
  sequencer: remember the onelines when parsing the todo file
  sequencer: prepare for rebase -i's commit functionality
  sequencer: introduce a helper to read files written by scripts
  sequencer: allow editing the commit message on a case-by-case basis
  sequencer: support amending commits
  sequencer: support cleaning up commit messages
  sequencer: do not try to commit when there were merge conflicts
  sequencer: left-trim lines read from the script
  sequencer: refactor write_message()
  sequencer: remove overzealous assumption in rebase -i mode
  sequencer: mark action_name() for translation
  sequencer: quote filenames in error messages
  sequencer: start error messages consistently with lower case
  sequencer: mark all error messages for translation

 builtin/commit.c              |   2 +-
 builtin/revert.c              |  48 +--
 sequencer.c                   | 690 ++++++++++++++++++++++++++++--------------
 sequencer.h                   |  23 +-
 t/t3501-revert-cherry-pick.sh |   2 +-
 5 files changed, 503 insertions(+), 262 deletions(-)

Published-As: https://github.com/dscho/git/releases/tag/prepare-sequencer-v3
Fetch-It-Via: git fetch https://github.com/dscho/git prepare-sequencer-v3

Interdiff vs v2:

 diff --git a/builtin/revert.c b/builtin/revert.c
 index c9ae4dc..0a7b5f4 100644
 --- a/builtin/revert.c
 +++ b/builtin/revert.c
 @@ -165,6 +165,12 @@ static int run_sequencer(int argc, const char **argv, struct replay_opts *opts)
  	if (argc > 1)
  		usage_with_options(usage_str, options);
  
 +	/* These option values will be free()d */
 +	if (opts->gpg_sign)
 +		opts->gpg_sign = xstrdup(opts->gpg_sign);
 +	if (opts->strategy)
 +		opts->strategy = xstrdup(opts->strategy);
 +
  	if (cmd == 'q')
  		return sequencer_remove_state(opts);
  	if (cmd == 'c')
 diff --git a/sequencer.c b/sequencer.c
 index cdff0f1..86d86ce 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -148,23 +148,15 @@ static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
  	return buf.buf;
  }
  
 -void *sequencer_entrust(struct replay_opts *opts, void *to_free)
 -{
 -	ALLOC_GROW(opts->owned, opts->owned_nr + 1, opts->owned_alloc);
 -	opts->owned[opts->owned_nr++] = to_free;
 -
 -	return to_free;
 -}
 -
  int sequencer_remove_state(struct replay_opts *opts)
  {
  	struct strbuf dir = STRBUF_INIT;
  	int i;
  
 -	for (i = 0; i < opts->owned_nr; i++)
 -		free(opts->owned[i]);
 -	free(opts->owned);
 -
 +	free(opts->gpg_sign);
 +	free(opts->strategy);
 +	for (i = 0; i < opts->xopts_nr; i++)
 +		free(opts->xopts[i]);
  	free(opts->xopts);
  
  	strbuf_addf(&dir, "%s", get_dir(opts));
 @@ -249,13 +241,19 @@ static int write_with_lock_file(const char *filename,
  
  	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
  	if (msg_fd < 0)
 -		return error_errno(_("Could not lock '%s'"), filename);
 -	if (write_in_full(msg_fd, buf, len) < 0)
 -		return error_errno(_("Could not write to '%s'"), filename);
 -	if (append_eol && write(msg_fd, "\n", 1) < 0)
 -		return error_errno(_("Could not write eol to '%s"), filename);
 -	if (commit_lock_file(&msg_file) < 0)
 -		return error(_("Error wrapping up '%s'."), filename);
 +		return error_errno(_("could not lock '%s'"), filename);
 +	if (write_in_full(msg_fd, buf, len) < 0) {
 +		rollback_lock_file(&msg_file);
 +		return error_errno(_("could not write to '%s'"), filename);
 +	}
 +	if (append_eol && write(msg_fd, "\n", 1) < 0) {
 +		rollback_lock_file(&msg_file);
 +		return error_errno(_("could not write eol to '%s"), filename);
 +	}
 +	if (commit_lock_file(&msg_file) < 0) {
 +		rollback_lock_file(&msg_file);
 +		return error(_("failed to finalize '%s'."), filename);
 +	}
  
  	return 0;
  }
 @@ -288,7 +286,7 @@ static int read_oneliner(struct strbuf *buf,
  		return 0;
  
  	if (strbuf_read_file(buf, path, 0) < 0) {
 -		warning_errno("could not read '%s'", path);
 +		warning_errno(_("could not read '%s'"), path);
  		return 0;
  	}
  
 @@ -314,11 +312,11 @@ static int error_dirty_index(struct replay_opts *opts)
  	if (read_cache_unmerged())
  		return error_resolve_conflict(_(action_name(opts)));
  
 -	error(_("Your local changes would be overwritten by %s."),
 +	error(_("your local changes would be overwritten by %s."),
  		_(action_name(opts)));
  
  	if (advice_commit_before_merge)
 -		advise(_("Commit your changes or stash them to proceed."));
 +		advise(_("commit your changes or stash them to proceed."));
  	return -1;
  }
  
 @@ -379,7 +377,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
  	struct merge_options o;
  	struct tree *result, *next_tree, *base_tree, *head_tree;
  	int clean;
 -	const char **xopt;
 +	char **xopt;
  	static struct lock_file index_lock;
  
  	hold_locked_index(&index_lock, 1);
 @@ -427,7 +425,7 @@ static int is_index_unchanged(void)
  	struct commit *head_commit;
  
  	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
 -		return error(_("Could not resolve HEAD commit\n"));
 +		return error(_("could not resolve HEAD commit\n"));
  
  	head_commit = lookup_commit(head_sha1);
  
 @@ -447,11 +445,15 @@ static int is_index_unchanged(void)
  
  	if (!cache_tree_fully_valid(active_cache_tree))
  		if (cache_tree_update(&the_index, 0))
 -			return error(_("Unable to update cache tree\n"));
 +			return error(_("unable to update cache tree\n"));
  
  	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
  }
  
 +/*
 + * Read the author-script file into an environment block, ready for use in
 + * run_command(), that can be free()d afterwards.
 + */
  static char **read_author_script(void)
  {
  	struct strbuf script = STRBUF_INIT;
 @@ -495,11 +497,11 @@ static char **read_author_script(void)
   * If we are revert, or if our cherry-pick results in a hand merge,
   * we had better say that the current user is responsible for that.
   *
 - * An exception is when sequencer_commit() is called during an
 + * An exception is when run_git_commit() is called during an
   * interactive rebase: in that case, we will want to retain the
   * author metadata.
   */
 -int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 +static int run_git_commit(const char *defmsg, struct replay_opts *opts,
  			  int allow_empty, int edit, int amend,
  			  int cleanup_commit_message)
  {
 @@ -513,16 +515,19 @@ int sequencer_commit(const char *defmsg, struct replay_opts *opts,
  		if (!env) {
  			const char *gpg_opt = gpg_sign_opt_quoted(opts);
  
 -			return error("You have staged changes in your working "
 -				"tree. If these changes are meant to be\n"
 -				"squashed into the previous commit, run:\n\n"
 -				"  git commit --amend %s\n\n"
 -				"If they are meant to go into a new commit, "
 -				"run:\n\n"
 -				"  git commit %s\n\n"
 -				"In both cases, once you're done, continue "
 -				"with:\n\n"
 -				"  git rebase --continue\n", gpg_opt, gpg_opt);
 +			return error(_("you have staged changes in your "
 +				       "working tree. If these changes are "
 +				       "meant to be\n"
 +				       "squashed into the previous commit, "
 +				       "run:\n\n"
 +				       "  git commit --amend %s\n\n"
 +				       "If they are meant to go into a new "
 +				       "commit, run:\n\n"
 +				       "  git commit %s\n\n"
 +				       "In both cases, once you're done, "
 +				       "continue with:\n\n"
 +				       "  git rebase --continue\n"),
 +				     gpg_opt, gpg_opt);
  		}
  	}
  
 @@ -566,12 +571,12 @@ static int is_original_commit_empty(struct commit *commit)
  	const unsigned char *ptree_sha1;
  
  	if (parse_commit(commit))
 -		return error(_("Could not parse commit %s\n"),
 +		return error(_("could not parse commit %s\n"),
  			     oid_to_hex(&commit->object.oid));
  	if (commit->parents) {
  		struct commit *parent = commit->parents->item;
  		if (parse_commit(parent))
 -			return error(_("Could not parse parent commit %s\n"),
 +			return error(_("could not parse parent commit %s\n"),
  				oid_to_hex(&parent->object.oid));
  		ptree_sha1 = parent->tree->object.oid.hash;
  	} else {
 @@ -645,7 +650,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	const char *base_label, *next_label;
  	struct commit_message msg = { NULL, NULL, NULL, NULL };
  	struct strbuf msgbuf = STRBUF_INIT;
 -	int res = 0, unborn = 0, allow;
 +	int res, unborn = 0, allow;
  
  	if (opts->no_commit) {
  		/*
 @@ -655,7 +660,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		 * to work on.
  		 */
  		if (write_cache_as_tree(head, 0, NULL))
 -			return error(_("Your index file is unmerged."));
 +			return error(_("your index file is unmerged."));
  	} else {
  		unborn = get_sha1("HEAD", head);
  		if (unborn)
 @@ -674,7 +679,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		struct commit_list *p;
  
  		if (!opts->mainline)
 -			return error(_("Commit %s is a merge but no -m option was given."),
 +			return error(_("commit %s is a merge but no -m option was given."),
  				oid_to_hex(&commit->object.oid));
  
  		for (cnt = 1, p = commit->parents;
 @@ -682,11 +687,11 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		     cnt++)
  			p = p->next;
  		if (cnt != opts->mainline || !p)
 -			return error(_("Commit %s does not have parent %d"),
 +			return error(_("commit %s does not have parent %d"),
  				oid_to_hex(&commit->object.oid), opts->mainline);
  		parent = p->item;
  	} else if (0 < opts->mainline)
 -		return error(_("Mainline was specified but commit %s is not a merge."),
 +		return error(_("mainline was specified but commit %s is not a merge."),
  			oid_to_hex(&commit->object.oid));
  	else
  		parent = commit->parents->item;
 @@ -697,12 +702,16 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
  
  	if (parent && parse_commit(parent) < 0)
 +		/*
 +		 * TRANSLATORS: The first %s will be a "todo" command like
 +		 * "revert" or "pick", the second %s a SHA1.
 +		 */
  		return error(_("%s: cannot parse parent commit %s"),
  			command_to_string(command),
  			oid_to_hex(&parent->object.oid));
  
  	if (get_message(commit, &msg) != 0)
 -		return error(_("Cannot get commit message for %s"),
 +		return error(_("cannot get commit message for %s"),
  			oid_to_hex(&commit->object.oid));
  
  	/*
 @@ -754,7 +763,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  	}
  
  	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 -		res |= do_recursive_merge(base, next, base_label, next_label,
 +		res = do_recursive_merge(base, next, base_label, next_label,
  					 head, &msgbuf, opts);
  		if (res < 0)
  			return res;
 @@ -763,11 +772,12 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		struct commit_list *common = NULL;
  		struct commit_list *remotes = NULL;
  
 -		res |= write_message(&msgbuf, git_path_merge_msg());
 +		res = write_message(&msgbuf, git_path_merge_msg());
  
  		commit_list_insert(base, &common);
  		commit_list_insert(next, &remotes);
 -		res |= try_merge_command(opts->strategy, opts->xopts_nr, opts->xopts,
 +		res |= try_merge_command(opts->strategy,
 +					 opts->xopts_nr, (const char **)opts->xopts,
  					common, sha1_to_hex(head), remotes);
  		free_commit_list(common);
  		free_commit_list(remotes);
 @@ -800,13 +810,12 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  
  	allow = allow_empty(opts, commit);
  	if (allow < 0) {
 -		res |= allow;
 +		res = allow;
  		goto leave;
  	}
 -	if (!opts->no_commit)
 -		res |= sequencer_commit(opts->edit ?
 -				NULL : git_path_merge_msg(),
 -			opts, allow, opts->edit, 0, 0);
 +	if (!res && !opts->no_commit)
 +		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
 +				     opts, allow, opts->edit, 0, 0);
  
  leave:
  	free_message(commit, &msg);
 @@ -924,23 +933,27 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
  static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  {
  	struct todo_item *item;
 -	char *p = buf;
 +	char *p = buf, *next_p;
  	int i, res = 0;
  
 -	for (i = 1; *p; i++) {
 +	for (i = 1; *p; i++, p = next_p) {
  		char *eol = strchrnul(p, '\n');
  
 +		next_p = *eol ? eol + 1 /* skip LF */ : eol;
 +
 +		if (p != eol && eol[-1] == '\r')
 +			eol--; /* skip Carriage Return */
 +
  		item = append_new_todo(todo_list);
  		item->offset_in_buf = p - todo_list->buf.buf;
  		if (parse_insn_line(item, p, eol)) {
 -			res |= error(_("Invalid line %d: %.*s"),
 +			res = error(_("invalid line %d: %.*s"),
  				i, (int)(eol - p), p);
  			item->command = -1;
  		}
 -		p = *eol ? eol + 1 : eol;
  	}
  	if (!todo_list->nr)
 -		return error(_("No commits parsed."));
 +		return error(_("no commits parsed."));
  	return res;
  }
  
 @@ -953,16 +966,16 @@ static int read_populate_todo(struct todo_list *todo_list,
  	strbuf_reset(&todo_list->buf);
  	fd = open(todo_file, O_RDONLY);
  	if (fd < 0)
 -		return error_errno(_("Could not open '%s'"), todo_file);
 +		return error_errno(_("could not open '%s'"), todo_file);
  	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
  		close(fd);
 -		return error(_("Could not read '%s'."), todo_file);
 +		return error(_("could not read '%s'."), todo_file);
  	}
  	close(fd);
  
  	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
  	if (res)
 -		return error(_("Unusable instruction sheet: '%s'"), todo_file);
 +		return error(_("unusable instruction sheet: '%s'"), todo_file);
  
  	if (!is_rebase_i(opts)) {
  		enum todo_command valid =
 @@ -973,9 +986,9 @@ static int read_populate_todo(struct todo_list *todo_list,
  			if (valid == todo_list->items[i].command)
  				continue;
  			else if (valid == TODO_PICK)
 -				return error(_("Cannot cherry-pick during a revert."));
 +				return error(_("cannot cherry-pick during a revert."));
  			else
 -				return error(_("Cannot revert during a cherry-pick."));
 +				return error(_("cannot revert during a cherry-pick."));
  	}
  
  	return 0;
 @@ -1001,22 +1014,29 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
  	else if (!strcmp(key, "options.mainline"))
  		opts->mainline = git_config_int(key, value);
  	else if (!strcmp(key, "options.strategy")) {
 -		git_config_string(&opts->strategy, key, value);
 -		sequencer_entrust(opts, (char *) opts->strategy);
 +		if (!value)
 +			config_error_nonbool(key);
 +		else {
 +			free(opts->strategy);
 +			opts->strategy = xstrdup(value);
 +		}
  	}
  	else if (!strcmp(key, "options.gpg-sign")) {
 -		git_config_string(&opts->gpg_sign, key, value);
 -		sequencer_entrust(opts, (char *) opts->gpg_sign);
 +		if (!value)
 +			config_error_nonbool(key);
 +		else {
 +			free(opts->gpg_sign);
 +			opts->gpg_sign = xstrdup(value);
 +		}
  	}
  	else if (!strcmp(key, "options.strategy-option")) {
  		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
 -		opts->xopts[opts->xopts_nr++] =
 -			sequencer_entrust(opts, xstrdup(value));
 +		opts->xopts[opts->xopts_nr++] = xstrdup(value);
  	} else
 -		return error(_("Invalid key: %s"), key);
 +		return error(_("invalid key: %s"), key);
  
  	if (!error_flag)
 -		return error(_("Invalid value for %s: %s"), key, value);
 +		return error(_("invalid value for %s: %s"), key, value);
  
  	return 0;
  }
 @@ -1030,11 +1050,11 @@ static int read_populate_opts(struct replay_opts *opts)
  			if (!starts_with(buf.buf, "-S"))
  				strbuf_reset(&buf);
  			else {
 -				opts->gpg_sign = buf.buf + 2;
 -				sequencer_entrust(opts,
 -					strbuf_detach(&buf, NULL));
 +				free(opts->gpg_sign);
 +				opts->gpg_sign = xstrdup(buf.buf + 2);
  			}
  		}
 +		strbuf_release(&buf);
  
  		return 0;
  	}
 @@ -1048,7 +1068,7 @@ static int read_populate_opts(struct replay_opts *opts)
  	 * are pretty certain that it is syntactically correct.
  	 */
  	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
 -		return error(_("Malformed options sheet: '%s'"),
 +		return error(_("malformed options sheet: '%s'"),
  			git_path_opts_file());
  	return 0;
  }
 @@ -1091,7 +1111,7 @@ static int create_seq_dir(void)
  		return -1;
  	}
  	else if (mkdir(git_path_seq_dir(), 0777) < 0)
 -		return error_errno(_("Could not create sequencer directory '%s'"),
 +		return error_errno(_("could not create sequencer directory '%s'"),
  				   git_path_seq_dir());
  	return 0;
  }
 @@ -1105,17 +1125,17 @@ static int save_head(const char *head)
  	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
  	if (fd < 0) {
  		rollback_lock_file(&head_lock);
 -		return error_errno(_("Could not lock HEAD"));
 +		return error_errno(_("could not lock HEAD"));
  	}
  	strbuf_addf(&buf, "%s\n", head);
  	if (write_in_full(fd, buf.buf, buf.len) < 0) {
  		rollback_lock_file(&head_lock);
 -		return error_errno(_("Could not write to '%s'"),
 +		return error_errno(_("could not write to '%s'"),
  				   git_path_head_file());
  	}
  	if (commit_lock_file(&head_lock) < 0) {
  		rollback_lock_file(&head_lock);
 -		return error(_("Error wrapping up '%s'."), git_path_head_file());
 +		return error(_("failed to finalize '%s'."), git_path_head_file());
  	}
  	return 0;
  }
 @@ -1194,14 +1214,14 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
  
  	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
  	if (fd < 0)
 -		return error_errno(_("Could not lock '%s'"), todo_path);
 +		return error_errno(_("could not lock '%s'"), todo_path);
  	offset = next < todo_list->nr ?
  		todo_list->items[next].offset_in_buf : todo_list->buf.len;
  	if (write_in_full(fd, todo_list->buf.buf + offset,
  			todo_list->buf.len - offset) < 0)
 -		return error_errno(_("Could not write to '%s'"), todo_path);
 +		return error_errno(_("could not write to '%s'"), todo_path);
  	if (commit_lock_file(&todo_lock) < 0)
 -		return error(_("Error wrapping up '%s'."), todo_path);
 +		return error(_("failed to finalize '%s'."), todo_path);
  	return 0;
  }
  
 @@ -1376,7 +1396,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
  			create_seq_dir() < 0)
  		return -1;
  	if (get_sha1("HEAD", sha1) && (opts->action == REPLAY_REVERT))
 -		return error(_("Can't revert as initial commit"));
 +		return error(_("can't revert as initial commit"));
  	if (save_head(sha1_to_hex(sha1)))
  		return -1;
  	if (save_opts(opts))
 diff --git a/sequencer.h b/sequencer.h
 index 688fff1..7a513c5 100644
 --- a/sequencer.h
 +++ b/sequencer.h
 @@ -26,37 +26,23 @@ struct replay_opts {
  
  	int mainline;
  
 -	const char *gpg_sign;
 +	char *gpg_sign;
  
  	/* Merge strategy */
 -	const char *strategy;
 -	const char **xopts;
 +	char *strategy;
 +	char **xopts;
  	size_t xopts_nr, xopts_alloc;
  
  	/* Only used by REPLAY_NONE */
  	struct rev_info *revs;
 -
 -	/* malloc()ed data entrusted to the sequencer */
 -	void **owned;
 -	int owned_nr, owned_alloc;
  };
  #define REPLAY_OPTS_INIT { -1 }
  
 -/*
 - * Make it the duty of sequencer_remove_state() to release the memory;
 - * For ease of use, return the same pointer.
 - */
 -void *sequencer_entrust(struct replay_opts *opts, void *to_free);
 -
  int sequencer_pick_revisions(struct replay_opts *opts);
  int sequencer_continue(struct replay_opts *opts);
  int sequencer_rollback(struct replay_opts *opts);
  int sequencer_remove_state(struct replay_opts *opts);
  
 -int sequencer_commit(const char *defmsg, struct replay_opts *opts,
 -			  int allow_empty, int edit, int amend,
 -			  int cleanup_commit_message);
 -
  extern const char sign_off_header[];
  
  void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag);
 diff --git a/t/t3501-revert-cherry-pick.sh b/t/t3501-revert-cherry-pick.sh
 index 51f3bbb..394f000 100755
 --- a/t/t3501-revert-cherry-pick.sh
 +++ b/t/t3501-revert-cherry-pick.sh
 @@ -96,7 +96,7 @@ test_expect_success 'revert forbidden on dirty working tree' '
  	echo content >extra_file &&
  	git add extra_file &&
  	test_must_fail git revert HEAD 2>errors &&
 -	test_i18ngrep "Your local changes would be overwritten by " errors
 +	test_i18ngrep "your local changes would be overwritten by " errors
  
  '
  

-- 
2.10.0.windows.1.325.ge6089c1

base-commit: a23ca1b8dc42ffd4de2ef30d67ce1e21ded29886

[PATCH v3 01/25] sequencer: use static initializers for replay_opts

[Johannes Schindelin]

This change is not completely faithful: instead of initializing all fields
to 0, we choose to initialize command and subcommand to -1 (instead of
defaulting to REPLAY_REVERT and REPLAY_NONE, respectively). Practically,
it makes no difference at all, but future-proofs the code to require
explicit assignments for both fields.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 6 ++----
 sequencer.h      | 1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 4e69380..7365559 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -178,10 +178,9 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	if (isatty(0))
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
@@ -195,10 +194,9 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 
 int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
 	parse_args(argc, argv, &opts);
diff --git a/sequencer.h b/sequencer.h
index 5ed5cb1..db425ad 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -47,6 +47,7 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
+#define REPLAY_OPTS_INIT { -1, -1 }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 02/25] sequencer: use memoized sequencer directory path

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/commit.c |  2 +-
 sequencer.c      | 11 ++++++-----
 sequencer.h      |  5 +----
 3 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 1cba3b7..9fddb19 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -183,7 +183,7 @@ static void determine_whence(struct wt_status *s)
 		whence = FROM_MERGE;
 	else if (file_exists(git_path_cherry_pick_head())) {
 		whence = FROM_CHERRY_PICK;
-		if (file_exists(git_path(SEQ_DIR)))
+		if (file_exists(git_path_seq_dir()))
 			sequencer_in_use = 1;
 	}
 	else
diff --git a/sequencer.c b/sequencer.c
index eec8a60..cb16cbd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,10 +21,11 @@
 const char sign_off_header[] = "Signed-off-by: ";
 static const char cherry_picked_prefix[] = "(cherry picked from commit ";
 
-static GIT_PATH_FUNC(git_path_todo_file, SEQ_TODO_FILE)
-static GIT_PATH_FUNC(git_path_opts_file, SEQ_OPTS_FILE)
-static GIT_PATH_FUNC(git_path_seq_dir, SEQ_DIR)
-static GIT_PATH_FUNC(git_path_head_file, SEQ_HEAD_FILE)
+GIT_PATH_FUNC(git_path_seq_dir, "sequencer")
+
+static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
+static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
+static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
 static int is_rfc2822_line(const char *buf, int len)
 {
@@ -112,7 +113,7 @@ static void remove_sequencer_state(void)
 {
 	struct strbuf seq_dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path(SEQ_DIR));
+	strbuf_addstr(&seq_dir, git_path_seq_dir());
 	remove_dir_recursively(&seq_dir, 0);
 	strbuf_release(&seq_dir);
 }
diff --git a/sequencer.h b/sequencer.h
index db425ad..dd4d33a 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -1,10 +1,7 @@
 #ifndef SEQUENCER_H
 #define SEQUENCER_H
 
-#define SEQ_DIR		"sequencer"
-#define SEQ_HEAD_FILE	"sequencer/head"
-#define SEQ_TODO_FILE	"sequencer/todo"
-#define SEQ_OPTS_FILE	"sequencer/opts"
+const char *git_path_seq_dir(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 03/25] sequencer: avoid unnecessary indirection

[Johannes Schindelin]

We really do not need the *pointer to a* pointer to the options in
the read_populate_opts() function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index cb16cbd..c2fbf6f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -813,7 +813,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
-static int read_populate_opts(struct replay_opts **opts)
+static int read_populate_opts(struct replay_opts *opts)
 {
 	if (!file_exists(git_path_opts_file()))
 		return 0;
@@ -823,7 +823,7 @@ static int read_populate_opts(struct replay_opts **opts)
 	 * about this case, though, because we wrote that file ourselves, so we
 	 * are pretty certain that it is syntactically correct.
 	 */
-	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
+	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
 		return error(_("Malformed options sheet: %s"),
 			git_path_opts_file());
 	return 0;
@@ -1054,7 +1054,7 @@ static int sequencer_continue(struct replay_opts *opts)
 
 	if (!file_exists(git_path_todo_file()))
 		return continue_single_pick();
-	if (read_populate_opts(&opts) ||
+	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
 		return -1;
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 04/25] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

In a couple of commits, we will teach the sequencer to handle the
nitty gritty of the interactive rebase, which keeps its state in a
different directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index c2fbf6f..8d272fb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,11 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static const char *get_dir(const struct replay_opts *opts)
+{
+	return git_path_seq_dir();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -109,13 +114,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(void)
+static void remove_sequencer_state(const struct replay_opts *opts)
 {
-	struct strbuf seq_dir = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path_seq_dir());
-	remove_dir_recursively(&seq_dir, 0);
-	strbuf_release(&seq_dir);
+	strbuf_addf(&dir, "%s", get_dir(opts));
+	remove_dir_recursively(&dir, 0);
+	strbuf_release(&dir);
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -940,7 +945,7 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	strbuf_release(&buf);
 	return 0;
 fail:
@@ -1034,7 +1039,7 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	return 0;
 }
 
@@ -1095,7 +1100,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	 * one that is being continued
 	 */
 	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state();
+		remove_sequencer_state(opts);
 		return 0;
 	}
 	if (opts->subcommand == REPLAY_ROLLBACK)
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 05/25] sequencer: eventually release memory allocated for the option values

[Johannes Schindelin]

The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
like a one-shot command when it reads its configuration: memory is
allocated and released only when the command exits.

This is kind of okay for git-cherry-pick, which *is* a one-shot
command. All the work to make the sequencer its work horse was
done to allow using the functionality as a library function, though,
including proper clean-up after use.

To remedy that, we now take custody of the option values in question,
requiring those values to be malloc()ed or strdup()ed (an alternative
approach, to add a list of pointers to malloc()ed data and to ask the
sequencer to release all of them in the end, was proposed earlier but
rejected during review).

Sadly, the current approach makes the code uglier, as we now have to
take care to strdup() the values passed via the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c |  6 ++++++
 sequencer.c      | 32 ++++++++++++++++++++++++++------
 sequencer.h      |  6 +++---
 3 files changed, 35 insertions(+), 9 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 7365559..fce9c75 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -174,6 +174,12 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 	if (argc > 1)
 		usage_with_options(usage_str, options);
+
+	/* These option values will be free()d */
+	if (opts->gpg_sign)
+		opts->gpg_sign = xstrdup(opts->gpg_sign);
+	if (opts->strategy)
+		opts->strategy = xstrdup(opts->strategy);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
diff --git a/sequencer.c b/sequencer.c
index 8d272fb..22c31c8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -117,6 +117,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 static void remove_sequencer_state(const struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
+	int i;
+
+	free(opts->gpg_sign);
+	free(opts->strategy);
+	for (i = 0; i < opts->xopts_nr; i++)
+		free(opts->xopts[i]);
+	free(opts->xopts);
 
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
@@ -280,7 +287,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	struct merge_options o;
 	struct tree *result, *next_tree, *base_tree, *head_tree;
 	int clean;
-	const char **xopt;
+	char **xopt;
 	static struct lock_file index_lock;
 
 	hold_locked_index(&index_lock, 1);
@@ -583,7 +590,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 
 		commit_list_insert(base, &common);
 		commit_list_insert(next, &remotes);
-		res |= try_merge_command(opts->strategy, opts->xopts_nr, opts->xopts,
+		res |= try_merge_command(opts->strategy,
+					 opts->xopts_nr, (const char **)opts->xopts,
 					common, sha1_to_hex(head), remotes);
 		free_commit_list(common);
 		free_commit_list(remotes);
@@ -802,10 +810,22 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
 	else if (!strcmp(key, "options.mainline"))
 		opts->mainline = git_config_int(key, value);
-	else if (!strcmp(key, "options.strategy"))
-		git_config_string(&opts->strategy, key, value);
-	else if (!strcmp(key, "options.gpg-sign"))
-		git_config_string(&opts->gpg_sign, key, value);
+	else if (!strcmp(key, "options.strategy")) {
+		if (!value)
+			config_error_nonbool(key);
+		else {
+			free(opts->strategy);
+			opts->strategy = xstrdup(value);
+		}
+	}
+	else if (!strcmp(key, "options.gpg-sign")) {
+		if (!value)
+			config_error_nonbool(key);
+		else {
+			free(opts->gpg_sign);
+			opts->gpg_sign = xstrdup(value);
+		}
+	}
 	else if (!strcmp(key, "options.strategy-option")) {
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
 		opts->xopts[opts->xopts_nr++] = xstrdup(value);
diff --git a/sequencer.h b/sequencer.h
index dd4d33a..8453669 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -34,11 +34,11 @@ struct replay_opts {
 
 	int mainline;
 
-	const char *gpg_sign;
+	char *gpg_sign;
 
 	/* Merge strategy */
-	const char *strategy;
-	const char **xopts;
+	char *strategy;
+	char **xopts;
 	size_t xopts_nr, xopts_alloc;
 
 	/* Only used by REPLAY_NONE */
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 06/25] sequencer: future-proof read_populate_todo()

[Johannes Schindelin]

Over the next commits, we will work on improving the sequencer to the
point where it can process the todo script of an interactive rebase. To
that end, we will need to teach the sequencer to read interactive
rebase's todo file. In preparation, we consolidate all places where
that todo file is needed to call a function that we will later extend.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 22c31c8..4e00c5e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -32,6 +32,11 @@ static const char *get_dir(const struct replay_opts *opts)
 	return git_path_seq_dir();
 }
 
+static const char *get_todo_path(const struct replay_opts *opts)
+{
+	return git_path_todo_file();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -769,25 +774,24 @@ static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
 static int read_populate_todo(struct commit_list **todo_list,
 			struct replay_opts *opts)
 {
+	const char *todo_file = get_todo_path(opts);
 	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
-	fd = open(git_path_todo_file(), O_RDONLY);
+	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"),
-				   git_path_todo_file());
+		return error_errno(_("Could not open %s"), todo_file);
 	if (strbuf_read(&buf, fd, 0) < 0) {
 		close(fd);
 		strbuf_release(&buf);
-		return error(_("Could not read %s."), git_path_todo_file());
+		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(buf.buf, todo_list, opts);
 	strbuf_release(&buf);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"),
-			git_path_todo_file());
+		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
@@ -1077,7 +1081,7 @@ static int sequencer_continue(struct replay_opts *opts)
 {
 	struct commit_list *todo_list = NULL;
 
-	if (!file_exists(git_path_todo_file()))
+	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

When we came up with the "sequencer" idea, we really wanted to have
kind of a plumbing equivalent of the interactive rebase. Hence the
choice of words: the "todo" script, a "pick", etc.

However, when it came time to implement the entire shebang, somehow this
idea got lost and the sequencer was used as working horse for
cherry-pick and revert instead. So as not to interfere with the
interactive rebase, it even uses a separate directory to store its
state.

Furthermore, it also is stupidly strict about the "todo" script it
accepts: while it parses commands in a way that was *designed* to be
similar to the interactive rebase, it then goes on to *error out* if the
commands disagree with the overall action (cherry-pick or revert).

Finally, the sequencer code chose to deviate from the interactive rebase
code insofar that when it comes to writing the file with the remaining
commands, it *reformats* the "todo" script instead of just writing the
part of the parsed script that were not yet processed. This is not only
unnecessary churn, but might well lose information that is valuable to
the user (i.e. comments after the commands).

Let's just bite the bullet and rewrite the entire parser; the code now
becomes not only more elegant: it allows us to go on and teach the
sequencer how to parse *true* "todo" scripts as used by the interactive
rebase itself. In a way, the sequencer is about to grow up to do its
older brother's job. Better.

In particular, we choose to maintain the list of commands in an array
instead of a linked list: this is flexible enough to allow us later on to
even implement rebase -i's reordering of fixup!/squash! commits very
easily (and with a very nice speed bonus, at least on Windows).

While at it, do not stop at the first problem, but list *all* of the
problems. This will help the user when the sequencer will do `rebase
-i`'s work by allowing to address all issues in one go rather than going
back and forth until the todo list is valid.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 288 +++++++++++++++++++++++++++++++++++-------------------------
 1 file changed, 167 insertions(+), 121 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4e00c5e..678fdf3 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -465,7 +465,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
+enum todo_command {
+	TODO_PICK = 0,
+	TODO_REVERT
+};
+
+static const char *todo_command_strings[] = {
+	"pick",
+	"revert"
+};
+
+static const char *command_to_string(const enum todo_command command)
+{
+	if (command < ARRAY_SIZE(todo_command_strings))
+		return todo_command_strings[command];
+	die("Unknown command: %d", command);
+}
+
+
+static int do_pick_commit(enum todo_command command, struct commit *commit,
+		struct replay_opts *opts)
 {
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
@@ -524,10 +543,13 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
 
 	if (parent && parse_commit(parent) < 0)
-		/* TRANSLATORS: The first %s will be "revert" or
-		   "cherry-pick", the second %s a SHA1 */
+		/*
+		 * TRANSLATORS: The first %s will be a "todo" command like
+		 * "revert" or "pick", the second %s a SHA1.
+		 */
 		return error(_("%s: cannot parse parent commit %s"),
-			action_name(opts), oid_to_hex(&parent->object.oid));
+			command_to_string(command),
+			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
 		return error(_("Cannot get commit message for %s"),
@@ -540,7 +562,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * reverse of it if we are revert.
 	 */
 
-	if (opts->action == REPLAY_REVERT) {
+	if (command == TODO_REVERT) {
 		base = commit;
 		base_label = msg.label;
 		next = parent;
@@ -581,7 +603,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
+	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
@@ -608,17 +630,17 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * However, if the merge did not even start, then we don't want to
 	 * write it at all.
 	 */
-	if (opts->action == REPLAY_PICK && !opts->no_commit && (res == 0 || res == 1) &&
+	if (command == TODO_PICK && !opts->no_commit && (res == 0 || res == 1) &&
 	    update_ref(NULL, "CHERRY_PICK_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
-	if (opts->action == REPLAY_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
+	if (command == TODO_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
 	    update_ref(NULL, "REVERT_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
 
 	if (res) {
-		error(opts->action == REPLAY_REVERT
+		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
 		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
@@ -680,116 +702,122 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
-static int format_todo(struct strbuf *buf, struct commit_list *todo_list,
-		struct replay_opts *opts)
+struct todo_item {
+	enum todo_command command;
+	struct commit *commit;
+	size_t offset_in_buf;
+};
+
+struct todo_list {
+	struct strbuf buf;
+	struct todo_item *items;
+	int nr, alloc, current;
+};
+
+#define TODO_LIST_INIT { STRBUF_INIT }
+
+static void todo_list_release(struct todo_list *todo_list)
 {
-	struct commit_list *cur = NULL;
-	const char *sha1_abbrev = NULL;
-	const char *action_str = opts->action == REPLAY_REVERT ? "revert" : "pick";
-	const char *subject;
-	int subject_len;
+	strbuf_release(&todo_list->buf);
+	free(todo_list->items);
+	todo_list->items = NULL;
+	todo_list->nr = todo_list->alloc = 0;
+}
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		const char *commit_buffer = get_commit_buffer(cur->item, NULL);
-		sha1_abbrev = find_unique_abbrev(cur->item->object.oid.hash, DEFAULT_ABBREV);
-		subject_len = find_commit_subject(commit_buffer, &subject);
-		strbuf_addf(buf, "%s %s %.*s\n", action_str, sha1_abbrev,
-			subject_len, subject);
-		unuse_commit_buffer(cur->item, commit_buffer);
-	}
-	return 0;
+struct todo_item *append_new_todo(struct todo_list *todo_list)
+{
+	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
+	return todo_list->items + todo_list->nr++;
 }
 
-static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
+static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 {
 	unsigned char commit_sha1[20];
-	enum replay_action action;
 	char *end_of_object_name;
-	int saved, status, padding;
-
-	if (starts_with(bol, "pick")) {
-		action = REPLAY_PICK;
-		bol += strlen("pick");
-	} else if (starts_with(bol, "revert")) {
-		action = REPLAY_REVERT;
-		bol += strlen("revert");
-	} else
-		return NULL;
+	int i, saved, status, padding;
+
+	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
+		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_strings))
+		return -1;
 
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
-		return NULL;
+		return -1;
 	bol += padding;
 
-	end_of_object_name = bol + strcspn(bol, " \t\n");
+	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
-	/*
-	 * Verify that the action matches up with the one in
-	 * opts; we don't support arbitrary instructions
-	 */
-	if (action != opts->action) {
-		if (action == REPLAY_REVERT)
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot revert during another revert.")
-			    : _("Cannot revert during a cherry-pick."));
-		else
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot cherry-pick during a revert.")
-			    : _("Cannot cherry-pick during another cherry-pick."));
-		return NULL;
-	}
-
 	if (status < 0)
-		return NULL;
+		return -1;
 
-	return lookup_commit_reference(commit_sha1);
+	item->commit = lookup_commit_reference(commit_sha1);
+	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
-			struct replay_opts *opts)
+static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
-	struct commit_list **next = todo_list;
-	struct commit *commit;
-	char *p = buf;
-	int i;
+	struct todo_item *item;
+	char *p = buf, *next_p;
+	int i, res = 0;
 
-	for (i = 1; *p; i++) {
+	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
-		commit = parse_insn_line(p, eol, opts);
-		if (!commit)
-			return error(_("Could not parse line %d."), i);
-		next = commit_list_append(commit, next);
-		p = *eol ? eol + 1 : eol;
+
+		next_p = *eol ? eol + 1 /* skip LF */ : eol;
+
+		item = append_new_todo(todo_list);
+		item->offset_in_buf = p - todo_list->buf.buf;
+		if (parse_insn_line(item, p, eol)) {
+			res = error(_("Invalid line %d: %.*s"),
+				i, (int)(eol - p), p);
+			item->command = -1;
+		}
 	}
-	if (!*todo_list)
+	if (!todo_list->nr)
 		return error(_("No commits parsed."));
-	return 0;
+	return res;
 }
 
-static int read_populate_todo(struct commit_list **todo_list,
+static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
 	const char *todo_file = get_todo_path(opts);
-	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
+	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
 		return error_errno(_("Could not open %s"), todo_file);
-	if (strbuf_read(&buf, fd, 0) < 0) {
+	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		strbuf_release(&buf);
 		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
-	res = parse_insn_buffer(buf.buf, todo_list, opts);
-	strbuf_release(&buf);
+	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
+	if (!res) {
+		enum todo_command valid =
+			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
+		int i;
+
+		for (i = 0; i < todo_list->nr; i++)
+			if (valid == todo_list->items[i].command)
+				continue;
+			else if (valid == TODO_PICK)
+				return error(_("Cannot cherry-pick during a revert."));
+			else
+				return error(_("Cannot revert during a cherry-pick."));
+	}
+
 	if (res)
 		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
@@ -858,18 +886,33 @@ static int read_populate_opts(struct replay_opts *opts)
 	return 0;
 }
 
-static int walk_revs_populate_todo(struct commit_list **todo_list,
+static int walk_revs_populate_todo(struct todo_list *todo_list,
 				struct replay_opts *opts)
 {
+	enum todo_command command = opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT;
+	const char *command_string = todo_command_strings[command];
 	struct commit *commit;
-	struct commit_list **next;
 
 	if (prepare_revs(opts))
 		return -1;
 
-	next = todo_list;
-	while ((commit = get_revision(opts->revs)))
-		next = commit_list_append(commit, next);
+	while ((commit = get_revision(opts->revs))) {
+		struct todo_item *item = append_new_todo(todo_list);
+		const char *commit_buffer = get_commit_buffer(commit, NULL);
+		const char *subject;
+		int subject_len;
+
+		item->command = command;
+		item->commit = commit;
+		item->offset_in_buf = todo_list->buf.len;
+		subject_len = find_commit_subject(commit_buffer, &subject);
+		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
+			find_unique_abbrev(commit->object.oid.hash,
+				DEFAULT_ABBREV),
+			subject_len, subject);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
 	return 0;
 }
 
@@ -977,30 +1020,22 @@ static int sequencer_rollback(struct replay_opts *opts)
 	return -1;
 }
 
-static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
+static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	static struct lock_file todo_lock;
-	struct strbuf buf = STRBUF_INIT;
-	int fd;
+	const char *todo_path = get_todo_path(opts);
+	int next = todo_list->current, offset, fd;
 
-	fd = hold_lock_file_for_update(&todo_lock, git_path_todo_file(), 0);
+	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
-		return error_errno(_("Could not lock '%s'"),
-				   git_path_todo_file());
-	if (format_todo(&buf, todo_list, opts) < 0) {
-		strbuf_release(&buf);
-		return error(_("Could not format %s."), git_path_todo_file());
-	}
-	if (write_in_full(fd, buf.buf, buf.len) < 0) {
-		strbuf_release(&buf);
-		return error_errno(_("Could not write to %s"),
-				   git_path_todo_file());
-	}
-	if (commit_lock_file(&todo_lock) < 0) {
-		strbuf_release(&buf);
-		return error(_("Error wrapping up %s."), git_path_todo_file());
-	}
-	strbuf_release(&buf);
+		return error_errno(_("Could not lock '%s'"), todo_path);
+	offset = next < todo_list->nr ?
+		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	if (write_in_full(fd, todo_list->buf.buf + offset,
+			todo_list->buf.len - offset) < 0)
+		return error_errno(_("Could not write to '%s'"), todo_path);
+	if (commit_lock_file(&todo_lock) < 0)
+		return error(_("Error wrapping up %s."), todo_path);
 	return 0;
 }
 
@@ -1039,9 +1074,8 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
-static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
+static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	struct commit_list *cur;
 	int res;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
@@ -1051,10 +1085,12 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		if (save_todo(cur, opts))
+	while (todo_list->current < todo_list->nr) {
+		struct todo_item *item = todo_list->items + todo_list->current;
+		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(cur->item, opts);
+		res = do_pick_commit(item->command, item->commit, opts);
+		todo_list->current++;
 		if (res)
 			return res;
 	}
@@ -1079,38 +1115,46 @@ static int continue_single_pick(void)
 
 static int sequencer_continue(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
 
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
-	if (read_populate_opts(opts) ||
-			read_populate_todo(&todo_list, opts))
+	if (read_populate_opts(opts))
 		return -1;
+	if ((res = read_populate_todo(&todo_list, opts)))
+		goto release_todo_list;
 
 	/* Verify that the conflict has been resolved */
 	if (file_exists(git_path_cherry_pick_head()) ||
 	    file_exists(git_path_revert_head())) {
-		int ret = continue_single_pick();
-		if (ret)
-			return ret;
+		res = continue_single_pick();
+		if (res)
+			goto release_todo_list;
 	}
-	if (index_differs_from("HEAD", 0))
-		return error_dirty_index(opts);
-	todo_list = todo_list->next;
-	return pick_commits(todo_list, opts);
+	if (index_differs_from("HEAD", 0)) {
+		res = error_dirty_index(opts);
+		goto release_todo_list;
+	}
+	todo_list.current++;
+	res = pick_commits(&todo_list, opts);
+release_todo_list:
+	todo_list_release(&todo_list);
+	return res;
 }
 
 static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
-	return do_pick_commit(cmit, opts);
+	return do_pick_commit(opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT, cmit, opts);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
 	unsigned char sha1[20];
-	int i;
+	int i, res;
 
 	if (opts->subcommand == REPLAY_NONE)
 		assert(opts->revs);
@@ -1185,7 +1229,9 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 		return -1;
 	if (save_opts(opts))
 		return -1;
-	return pick_commits(todo_list, opts);
+	res = pick_commits(&todo_list, opts);
+	todo_list_release(&todo_list);
+	return res;
 }
 
 void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 08/25] sequencer: strip CR from the todo script

[Johannes Schindelin]

It is not unheard of that editors on Windows write CR/LF even if the
file originally had only LF. This is particularly awkward for exec lines
of a rebase -i todo sheet. Take for example the insn "exec echo": The
shell script parser splits at the LF and leaves the CR attached to
"echo", which leads to the unknown command "echo\r".

Work around that by stripping CR when reading the todo commands, as we
already do for LF.

This happens to fix t9903.14 and .15 in MSYS1 environments (with the
rebase--helper patches based on this patch series): the todo script
constructed in such a setup contains CR/LF thanks to MSYS1 runtime's
cleverness.

Based on a report and a patch by Johannes Sixt.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 678fdf3..cee7e50 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -774,6 +774,9 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 
 		next_p = *eol ? eol + 1 /* skip LF */ : eol;
 
+		if (p != eol && eol[-1] == '\r')
+			eol--; /* skip Carriage Return */
+
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
 		if (parse_insn_line(item, p, eol)) {
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 09/25] sequencer: avoid completely different messages for different actions

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index cee7e50..443a238 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -224,11 +224,8 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(action_name(opts));
 
-	/* Different translation strings for cherry-pick and revert */
-	if (opts->action == REPLAY_PICK)
-		error(_("Your local changes would be overwritten by cherry-pick."));
-	else
-		error(_("Your local changes would be overwritten by revert."));
+	error(_("Your local changes would be overwritten by %s."),
+		action_name(opts));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 10/25] sequencer: get rid of the subcommand field

[Johannes Schindelin]

The subcommands are used exactly once, at the very beginning of
sequencer_pick_revisions(), to determine what to do. This is an
unnecessary level of indirection: we can simply call the correct
function to begin with. So let's do that.

While at it, ensure that the subcommands return an error code so that
they do not have to die() all over the place (bad practice for library
functions...).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 36 ++++++++++++++++--------------------
 sequencer.c      | 35 +++++++++++------------------------
 sequencer.h      | 13 ++++---------
 3 files changed, 31 insertions(+), 53 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index fce9c75..0a7b5f4 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -71,7 +71,7 @@ static void verify_opt_compatible(const char *me, const char *base_opt, ...)
 		die(_("%s: %s cannot be used with %s"), me, this_opt, base_opt);
 }
 
-static void parse_args(int argc, const char **argv, struct replay_opts *opts)
+static int run_sequencer(int argc, const char **argv, struct replay_opts *opts)
 {
 	const char * const * usage_str = revert_or_cherry_pick_usage(opts);
 	const char *me = action_name(opts);
@@ -115,25 +115,15 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 	if (opts->keep_redundant_commits)
 		opts->allow_empty = 1;
 
-	/* Set the subcommand */
-	if (cmd == 'q')
-		opts->subcommand = REPLAY_REMOVE_STATE;
-	else if (cmd == 'c')
-		opts->subcommand = REPLAY_CONTINUE;
-	else if (cmd == 'a')
-		opts->subcommand = REPLAY_ROLLBACK;
-	else
-		opts->subcommand = REPLAY_NONE;
-
 	/* Check for incompatible command line arguments */
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		char *this_operation;
-		if (opts->subcommand == REPLAY_REMOVE_STATE)
+		if (cmd == 'q')
 			this_operation = "--quit";
-		else if (opts->subcommand == REPLAY_CONTINUE)
+		else if (cmd == 'c')
 			this_operation = "--continue";
 		else {
-			assert(opts->subcommand == REPLAY_ROLLBACK);
+			assert(cmd == 'a');
 			this_operation = "--abort";
 		}
 
@@ -156,7 +146,7 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 				"--edit", opts->edit,
 				NULL);
 
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		opts->revs = NULL;
 	} else {
 		struct setup_revision_opt s_r_opt;
@@ -180,6 +170,14 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 		opts->gpg_sign = xstrdup(opts->gpg_sign);
 	if (opts->strategy)
 		opts->strategy = xstrdup(opts->strategy);
+
+	if (cmd == 'q')
+		return sequencer_remove_state(opts);
+	if (cmd == 'c')
+		return sequencer_continue(opts);
+	if (cmd == 'a')
+		return sequencer_rollback(opts);
+	return sequencer_pick_revisions(opts);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
@@ -191,8 +189,7 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("revert failed"));
 	return res;
@@ -205,8 +202,7 @@ int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("cherry-pick failed"));
 	return res;
diff --git a/sequencer.c b/sequencer.c
index 443a238..14b1746 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -119,7 +119,7 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(const struct replay_opts *opts)
+int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
 	int i;
@@ -133,6 +133,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
 	strbuf_release(&dir);
+
+	return 0;
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -977,7 +979,7 @@ static int rollback_single_pick(void)
 	return reset_for_rollback(head_sha1);
 }
 
-static int sequencer_rollback(struct replay_opts *opts)
+int sequencer_rollback(struct replay_opts *opts)
 {
 	FILE *f;
 	unsigned char sha1[20];
@@ -1012,9 +1014,8 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state(opts);
 	strbuf_release(&buf);
-	return 0;
+	return sequencer_remove_state(opts);
 fail:
 	strbuf_release(&buf);
 	return -1;
@@ -1099,8 +1100,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state(opts);
-	return 0;
+	return sequencer_remove_state(opts);
 }
 
 static int continue_single_pick(void)
@@ -1113,11 +1113,14 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
-static int sequencer_continue(struct replay_opts *opts)
+int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
 	int res;
 
+	if (read_and_refresh_cache(opts))
+		return -1;
+
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
@@ -1156,26 +1159,10 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	unsigned char sha1[20];
 	int i, res;
 
-	if (opts->subcommand == REPLAY_NONE)
-		assert(opts->revs);
-
+	assert(opts->revs);
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	/*
-	 * Decide what to do depending on the arguments; a fresh
-	 * cherry-pick should be handled differently from an existing
-	 * one that is being continued
-	 */
-	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state(opts);
-		return 0;
-	}
-	if (opts->subcommand == REPLAY_ROLLBACK)
-		return sequencer_rollback(opts);
-	if (opts->subcommand == REPLAY_CONTINUE)
-		return sequencer_continue(opts);
-
 	for (i = 0; i < opts->revs->pending.nr; i++) {
 		unsigned char sha1[20];
 		const char *name = opts->revs->pending.objects[i].name;
diff --git a/sequencer.h b/sequencer.h
index 8453669..7a513c5 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -10,16 +10,8 @@ enum replay_action {
 	REPLAY_PICK
 };
 
-enum replay_subcommand {
-	REPLAY_NONE,
-	REPLAY_REMOVE_STATE,
-	REPLAY_CONTINUE,
-	REPLAY_ROLLBACK
-};
-
 struct replay_opts {
 	enum replay_action action;
-	enum replay_subcommand subcommand;
 
 	/* Boolean options */
 	int edit;
@@ -44,9 +36,12 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
-#define REPLAY_OPTS_INIT { -1, -1 }
+#define REPLAY_OPTS_INIT { -1 }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
+int sequencer_continue(struct replay_opts *opts);
+int sequencer_rollback(struct replay_opts *opts);
+int sequencer_remove_state(struct replay_opts *opts);
 
 extern const char sign_off_header[];
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 11/25] sequencer: refactor the code to obtain a short commit name

[Johannes Schindelin]

Not only does this DRY up the code (providing a better documentation what
the code is about, as well as allowing to change the behavior in a single
place), it also makes it substantially shorter to use the same
functionality in functions to be introduced when we teach the sequencer to
process interactive-rebase's git-rebase-todo file.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 14b1746..afc494e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -149,13 +149,18 @@ struct commit_message {
 	const char *message;
 };
 
+static const char *short_commit_name(struct commit *commit)
+{
+	return find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+}
+
 static int get_message(struct commit *commit, struct commit_message *out)
 {
 	const char *abbrev, *subject;
 	int subject_len;
 
 	out->message = logmsg_reencode(commit, NULL, get_commit_output_encoding());
-	abbrev = find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+	abbrev = short_commit_name(commit);
 
 	subject_len = find_commit_subject(out->message, &subject);
 
@@ -642,8 +647,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
-		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
-		      msg.subject);
+		      short_commit_name(commit), msg.subject);
 		print_advice(res == 1, opts);
 		rerere(opts->allow_rerere_auto);
 		goto leave;
@@ -910,9 +914,7 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
-			find_unique_abbrev(commit->object.oid.hash,
-				DEFAULT_ABBREV),
-			subject_len, subject);
+			short_commit_name(commit), subject_len, subject);
 		unuse_commit_buffer(commit, commit_buffer);
 	}
 	return 0;
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 12/25] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

The `git-rebase-todo` file contains a list of commands. Most of those
commands have the form

	<verb> <sha1> <oneline>

The <oneline> is displayed primarily for the user's convenience, as
rebase -i really interprets only the <verb> <sha1> part. However, there
are *some* places in interactive rebase where the <oneline> is used to
display messages, e.g. for reporting at which commit we stopped.

So let's just remember it when parsing the todo file; we keep a copy of
the entire todo file anyway (to write out the new `done` and
`git-rebase-todo` file just before processing each command), so all we
need to do is remember the begin offsets and lengths.

As we will have to parse and remember the command-line for `exec` commands
later, we do not call the field "oneline" but rather "arg" (and will reuse
that for exec's command-line).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index afc494e..7ba5e07 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -708,6 +708,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	const char *arg;
+	int arg_len;
 	size_t offset_in_buf;
 };
 
@@ -759,6 +761,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
+	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_len = (int)(eol - item->arg);
+
 	if (status < 0)
 		return -1;
 
@@ -911,6 +916,8 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
+		item->arg = NULL;
+		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 13/25] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

In interactive rebases, we commit a little bit differently than the
sequencer did so far: we heed the "author-script", the "message" and the
"amend" files in the .git/rebase-merge/ subdirectory.

Likewise, we may want to edit the commit message *even* when providing a
file containing the suggested commit message. Therefore we change the
code to not even provide a default message when we do not want any, and
to call the editor explicitly.

Also, in "interactive rebase" mode we want to skip reading the options
in the state directory of the cherry-pick/revert commands.

Finally, as interactive rebase's GPG settings are configured differently
from how cherry-pick (and therefore sequencer) handles them, we will
leave support for that to the next commit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 89 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 7ba5e07..b694ace 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,19 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+/*
+ * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
+ * GIT_AUTHOR_DATE that will be used for the commit that is currently
+ * being rebased.
+ */
+static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+
+/* We will introduce the 'interactive rebase' mode later */
+static inline int is_rebase_i(const struct replay_opts *opts)
+{
+	return 0;
+}
+
 static const char *get_dir(const struct replay_opts *opts)
 {
 	return git_path_seq_dir();
@@ -370,19 +383,79 @@ static int is_index_unchanged(void)
 }
 
 /*
+ * Read the author-script file into an environment block, ready for use in
+ * run_command(), that can be free()d afterwards.
+ */
+static char **read_author_script(void)
+{
+	struct strbuf script = STRBUF_INIT;
+	int i, count = 0;
+	char *p, *p2, **env;
+	size_t env_size;
+
+	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
+		return NULL;
+
+	for (p = script.buf; *p; p++)
+		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
+			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
+		else if (*p == '\'')
+			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
+		else if (*p == '\n') {
+			*p = '\0';
+			count++;
+		}
+
+	env_size = (count + 1) * sizeof(*env);
+	strbuf_grow(&script, env_size);
+	memmove(script.buf + env_size, script.buf, script.len);
+	p = script.buf + env_size;
+	env = (char **)strbuf_detach(&script, NULL);
+
+	for (i = 0; i < count; i++) {
+		env[i] = p;
+		p += strlen(p) + 1;
+	}
+	env[count] = NULL;
+
+	return env;
+}
+
+/*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
  * author date and name.
+ *
  * If we are revert, or if our cherry-pick results in a hand merge,
  * we had better say that the current user is responsible for that.
+ *
+ * An exception is when run_git_commit() is called during an
+ * interactive rebase: in that case, we will want to retain the
+ * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty)
 {
+	char **env = NULL;
 	struct argv_array array;
 	int rc;
 	const char *value;
 
+	if (is_rebase_i(opts)) {
+		env = read_author_script();
+		if (!env)
+			return error("You have staged changes in your working "
+				"tree. If these changes are meant to be\n"
+				"squashed into the previous commit, run:\n\n"
+				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"If they are meant to go into a new commit, "
+				"run:\n\n"
+				"  git commit $gpg_sign_opt_quoted\n\n"
+				"In both cases, once you're done, continue "
+				"with:\n\n"
+				"  git rebase --continue\n");
+	}
+
 	argv_array_init(&array);
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
@@ -391,14 +464,13 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
 		argv_array_push(&array, "-s");
-	if (!opts->edit) {
-		argv_array_push(&array, "-F");
-		argv_array_push(&array, defmsg);
-		if (!opts->signoff &&
-		    !opts->record_origin &&
-		    git_config_get_value("commit.cleanup", &value))
-			argv_array_push(&array, "--cleanup=verbatim");
-	}
+	if (defmsg)
+		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (opts->edit)
+		argv_array_push(&array, "-e");
+	else if (!opts->signoff && !opts->record_origin &&
+		 git_config_get_value("commit.cleanup", &value))
+		argv_array_push(&array, "--cleanup=verbatim");
 
 	if (allow_empty)
 		argv_array_push(&array, "--allow-empty");
@@ -406,8 +478,11 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt(array.argv, RUN_GIT_CMD);
+	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+			(const char *const *)env);
 	argv_array_clear(&array);
+	free(env);
+
 	return rc;
 }
 
@@ -659,7 +734,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(git_path_merge_msg(), opts, allow);
+		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
+				     opts, allow);
 
 leave:
 	free_message(commit, &msg);
@@ -883,6 +959,9 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return 0;
+
 	if (!file_exists(git_path_opts_file()))
 		return 0;
 	/*
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 14/25] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

As we are slowly teaching the sequencer to perform the hard work for
the interactive rebase, we need to read files that were written by
shell scripts.

These files typically contain a single line and are invariably ended
by a line feed (and possibly a carriage return before that). Let's use
a helper to read such files and to remove the line ending.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index b694ace..681552a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -234,6 +234,37 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
 	return 0;
 }
 
+/*
+ * Reads a file that was presumably written by a shell script, i.e.
+ * with an end-of-line marker that needs to be stripped.
+ *
+ * Returns 1 if the file was read, 0 if it could not be read or does not exist.
+ */
+static int read_oneliner(struct strbuf *buf,
+	const char *path, int skip_if_empty)
+{
+	int orig_len = buf->len;
+
+	if (!file_exists(path))
+		return 0;
+
+	if (strbuf_read_file(buf, path, 0) < 0) {
+		warning_errno(_("could not read '%s'"), path);
+		return 0;
+	}
+
+	if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
+		if (--buf->len > orig_len && buf->buf[buf->len - 1] == '\r')
+			--buf->len;
+		buf->buf[buf->len] = '\0';
+	}
+
+	if (skip_if_empty && buf->len == orig_len)
+		return 0;
+
+	return 1;
+}
+
 static struct tree *empty_tree(void)
 {
 	return lookup_tree(EMPTY_TREE_SHA1_BIN);
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 15/25] sequencer: allow editing the commit message on a case-by-case basis

[Johannes Schindelin]

In the upcoming commits, we will implement more and more of rebase -i's
functionality inside the sequencer. One particular feature of the
commands to come is that some of them allow editing the commit message
while others don't, i.e. we cannot define in the replay_opts whether the
commit message should be edited or not.

Let's add a new parameter to the run_git_commit() function. Previously,
it was the duty of the caller to ensure that the opts->edit setting
indicates whether to let the user edit the commit message or not,
indicating that it is an "all or nothing" setting, i.e. that the
sequencer wants to let the user edit *all* commit message, or none at
all. In the upcoming rebase -i mode, it will depend on the particular
command that is currently executed, though.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 48 ++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 40 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 681552a..b621f4b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -15,6 +15,7 @@
 #include "merge-recursive.h"
 #include "refs.h"
 #include "argv-array.h"
+#include "quote.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -33,6 +34,11 @@ static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
  * being rebased.
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+/*
+ * The following files are written by git-rebase just after parsing the
+ * command-line (and are only consumed, not modified, by the sequencer).
+ */
+static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -132,6 +138,16 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
+static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
+{
+	static struct strbuf buf = STRBUF_INIT;
+
+	strbuf_reset(&buf);
+	if (opts->gpg_sign)
+		sq_quotef(&buf, "-S%s", opts->gpg_sign);
+	return buf.buf;
+}
+
 int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
@@ -465,7 +481,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty)
+			  int allow_empty, int edit)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -474,17 +490,20 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 
 	if (is_rebase_i(opts)) {
 		env = read_author_script();
-		if (!env)
+		if (!env) {
+			const char *gpg_opt = gpg_sign_opt_quoted(opts);
+
 			return error("You have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"  git commit --amend %s\n\n"
 				"If they are meant to go into a new commit, "
 				"run:\n\n"
-				"  git commit $gpg_sign_opt_quoted\n\n"
+				"  git commit %s\n\n"
 				"In both cases, once you're done, continue "
 				"with:\n\n"
-				"  git rebase --continue\n");
+				"  git rebase --continue\n", gpg_opt, gpg_opt);
+		}
 	}
 
 	argv_array_init(&array);
@@ -497,7 +516,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
-	if (opts->edit)
+	if (edit)
 		argv_array_push(&array, "-e");
 	else if (!opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
@@ -766,7 +785,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow);
+				     opts, allow, opts->edit);
 
 leave:
 	free_message(commit, &msg);
@@ -990,8 +1009,21 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
-	if (is_rebase_i(opts))
+	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
+		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
+			if (!starts_with(buf.buf, "-S"))
+				strbuf_reset(&buf);
+			else {
+				free(opts->gpg_sign);
+				opts->gpg_sign = xstrdup(buf.buf + 2);
+			}
+		}
+		strbuf_release(&buf);
+
 		return 0;
+	}
 
 	if (!file_exists(git_path_opts_file()))
 		return 0;
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 16/25] sequencer: support amending commits

[Johannes Schindelin]

This teaches the run_git_commit() function to take an argument that will
allow us to implement "todo" commands that need to amend the commit
messages ("fixup", "squash" and "reword").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index b621f4b..403a4f0 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -481,7 +481,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit)
+			  int allow_empty, int edit, int amend)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -510,6 +510,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
 
+	if (amend)
+		argv_array_push(&array, "--amend");
 	if (opts->gpg_sign)
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
@@ -785,7 +787,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit);
+				     opts, allow, opts->edit, 0);
 
 leave:
 	free_message(commit, &msg);
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 17/25] sequencer: support cleaning up commit messages

[Johannes Schindelin]

The run_git_commit() function already knows how to amend commits, and
with this new option, it can also clean up commit messages (i.e. strip
out commented lines). This is needed to implement rebase -i's 'fixup'
and 'squash' commands as sequencer commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 403a4f0..108bca8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -481,7 +481,8 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit, int amend)
+			  int allow_empty, int edit, int amend,
+			  int cleanup_commit_message)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -518,9 +519,12 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (cleanup_commit_message)
+		argv_array_push(&array, "--cleanup=strip");
 	if (edit)
 		argv_array_push(&array, "-e");
-	else if (!opts->signoff && !opts->record_origin &&
+	else if (!cleanup_commit_message &&
+		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
 		argv_array_push(&array, "--cleanup=verbatim");
 
@@ -787,7 +791,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit, 0);
+				     opts, allow, opts->edit, 0, 0);
 
 leave:
 	free_message(commit, &msg);
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 18/25] sequencer: do not try to commit when there were merge conflicts

[Johannes Schindelin]

The return value of do_recursive_merge() may be positive (indicating merge
conflicts), or 0 (indicating success). It also may be negative, indicating
a fatal error that requires us to abort.

Now, if the return value indicates that there are merge conflicts, we
should not try to commit those changes, of course.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index 108bca8..23fe7db 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -789,7 +789,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		res = allow;
 		goto leave;
 	}
-	if (!opts->no_commit)
+	if (!res && !opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
 				     opts, allow, opts->edit, 0, 0);
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 19/25] sequencer: left-trim lines read from the script

[Johannes Schindelin]

Interactive rebase's scripts may be indented; we need to handle this
case, too, now that we prepare the sequencer to process interactive
rebases.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 23fe7db..45a3651 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -873,6 +873,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	/* left-trim */
+	bol += strspn(bol, " \t");
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 20/25] sequencer: refactor write_message()

[Johannes Schindelin]

The write_message() function safely writes an strbuf to a file.
Sometimes it is inconvenient to require an strbuf, though: the text to
be written may not be stored in a strbuf, or the strbuf should not be
released after writing.

Let's refactor "safely writing string to a file" into
write_with_lock_file(), and make write_message() use it. The new
function makes it easy to create new convenience function
write_file_gently(); as some of the upcoming callers of this new
function would want to append a newline character, add a flag for it in
write_file_gently(), and thus in write_with_lock_file().

While at it, roll back the locked files in case of failure, as pointed
out by Hannes Sixt.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 31 ++++++++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 45a3651..5cca8d8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -234,22 +234,43 @@ static void print_advice(int show_hint, struct replay_opts *opts)
 	}
 }
 
-static int write_message(struct strbuf *msgbuf, const char *filename)
+static int write_with_lock_file(const char *filename,
+				const void *buf, size_t len, int append_eol)
 {
 	static struct lock_file msg_file;
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
 		return error_errno(_("Could not lock '%s'"), filename);
-	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
-		return error_errno(_("Could not write to %s"), filename);
-	strbuf_release(msgbuf);
-	if (commit_lock_file(&msg_file) < 0)
+	if (write_in_full(msg_fd, buf, len) < 0) {
+		rollback_lock_file(&msg_file);
+		return error_errno(_("Could not write to '%s'"), filename);
+	}
+	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		rollback_lock_file(&msg_file);
+		return error_errno(_("Could not write eol to '%s"), filename);
+	}
+	if (commit_lock_file(&msg_file) < 0) {
+		rollback_lock_file(&msg_file);
 		return error(_("Error wrapping up %s."), filename);
+	}
 
 	return 0;
 }
 
+static int write_message(struct strbuf *msgbuf, const char *filename)
+{
+	int res = write_with_lock_file(filename, msgbuf->buf, msgbuf->len, 0);
+	strbuf_release(msgbuf);
+	return res;
+}
+
+static int write_file_gently(const char *filename,
+			     const char *text, int append_eol)
+{
+	return write_with_lock_file(filename, text, strlen(text), append_eol);
+}
+
 /*
  * Reads a file that was presumably written by a shell script, i.e.
  * with an end-of-line marker that needs to be stripped.
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 21/25] sequencer: remove overzealous assumption in rebase -i mode

[Johannes Schindelin]

The sequencer was introduced to make the cherry-pick and revert
functionality available as library function, with the original idea
being to extend the sequencer to also implement the rebase -i
functionality.

The test to ensure that all of the commands in the script are identical
to the overall operation does not mesh well with that.

Therefore let's disable the test in rebase -i mode.

While at it, error out early if the "instruction sheet" (i.e. the todo
script) could not be parsed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 5cca8d8..ffce095 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -971,7 +971,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (!res) {
+	if (res)
+		return error(_("Unusable instruction sheet: %s"), todo_file);
+
+	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
 		int i;
@@ -985,8 +988,6 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("Cannot revert during a cherry-pick."));
 	}
 
-	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 22/25] sequencer: mark action_name() for translation

[Johannes Schindelin]

The definition of this function goes back all the way to 043a449
(sequencer: factor code out of revert builtin, 2012-01-11), long before a
serious effort was made to translate all the error messages.

It is slightly out of the context of the current patch series (whose
purpose it is to re-implement the performance critical parts of the
interactive rebase in C) to make the error messages in the sequencer
translatable, but what the heck. We'll just do it while we're looking at
this part of the code.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ffce095..40ef33c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -168,7 +168,7 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? "revert" : "cherry-pick";
+	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
 }
 
 struct commit_message {
@@ -310,10 +310,10 @@ static struct tree *empty_tree(void)
 static int error_dirty_index(struct replay_opts *opts)
 {
 	if (read_cache_unmerged())
-		return error_resolve_conflict(action_name(opts));
+		return error_resolve_conflict(_(action_name(opts)));
 
 	error(_("Your local changes would be overwritten by %s."),
-		action_name(opts));
+		_(action_name(opts)));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
@@ -331,7 +331,7 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	if (checkout_fast_forward(from, to, 1))
 		return -1; /* the callee should have complained already */
 
-	strbuf_addf(&sb, _("%s: fast-forward"), action_name(opts));
+	strbuf_addf(&sb, _("%s: fast-forward"), _(action_name(opts)));
 
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
@@ -407,7 +407,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
 		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
 		return error(_("%s: Unable to write new index file"),
-			action_name(opts));
+			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
 
 	if (opts->signoff)
@@ -844,14 +844,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	if (read_index_preload(&the_index, NULL) < 0) {
 		rollback_lock_file(&index_lock);
 		return error(_("git %s: failed to read the index"),
-			action_name(opts));
+			_(action_name(opts)));
 	}
 	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, NULL, NULL, NULL);
 	if (the_index.cache_changed && index_fd >= 0) {
 		if (write_locked_index(&the_index, &index_lock, COMMIT_LOCK)) {
 			rollback_lock_file(&index_lock);
 			return error(_("git %s: failed to refresh the index"),
-				action_name(opts));
+				_(action_name(opts)));
 		}
 	}
 	rollback_lock_file(&index_lock);
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 23/25] sequencer: quote filenames in error messages

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3609 bytes --]

This makes the code consistent by fixing quite a couple of error messages.

Suggested by Jakub Narębski.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 40ef33c..4596540 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -252,7 +252,7 @@ static int write_with_lock_file(const char *filename,
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
-		return error(_("Error wrapping up %s."), filename);
+		return error(_("Error wrapping up '%s'."), filename);
 	}
 
 	return 0;
@@ -963,16 +963,16 @@ static int read_populate_todo(struct todo_list *todo_list,
 	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"), todo_file);
+		return error_errno(_("Could not open '%s'"), todo_file);
 	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		return error(_("Could not read %s."), todo_file);
+		return error(_("Could not read '%s'."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
+		return error(_("Unusable instruction sheet: '%s'"), todo_file);
 
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
@@ -1065,7 +1065,7 @@ static int read_populate_opts(struct replay_opts *opts)
 	 * are pretty certain that it is syntactically correct.
 	 */
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
-		return error(_("Malformed options sheet: %s"),
+		return error(_("Malformed options sheet: '%s'"),
 			git_path_opts_file());
 	return 0;
 }
@@ -1108,7 +1108,7 @@ static int create_seq_dir(void)
 		return -1;
 	}
 	else if (mkdir(git_path_seq_dir(), 0777) < 0)
-		return error_errno(_("Could not create sequencer directory %s"),
+		return error_errno(_("Could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
 }
@@ -1127,12 +1127,12 @@ static int save_head(const char *head)
 	strbuf_addf(&buf, "%s\n", head);
 	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not write to %s"),
+		return error_errno(_("Could not write to '%s'"),
 				   git_path_head_file());
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-		return error(_("Error wrapping up %s."), git_path_head_file());
+		return error(_("Error wrapping up '%s'."), git_path_head_file());
 	}
 	return 0;
 }
@@ -1177,9 +1177,9 @@ int sequencer_rollback(struct replay_opts *opts)
 		return rollback_single_pick();
 	}
 	if (!f)
-		return error_errno(_("cannot open %s"), git_path_head_file());
+		return error_errno(_("cannot open '%s'"), git_path_head_file());
 	if (strbuf_getline_lf(&buf, f)) {
-		error(_("cannot read %s: %s"), git_path_head_file(),
+		error(_("cannot read '%s': %s"), git_path_head_file(),
 		      ferror(f) ?  strerror(errno) : _("unexpected end of file"));
 		fclose(f);
 		goto fail;
@@ -1218,7 +1218,7 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 			todo_list->buf.len - offset) < 0)
 		return error_errno(_("Could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
-		return error(_("Error wrapping up %s."), todo_path);
+		return error(_("Error wrapping up '%s'."), todo_path);
 	return 0;
 }
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 24/25] sequencer: start error messages consistently with lower case

[Johannes Schindelin]

Quite a few error messages touched by this developer during the work to
speed up rebase -i started with an upper case letter, violating our
current conventions. Instead of sneaking in this fix (and forgetting
quite a few error messages), let's just have one wholesale patch fixing
all of the error messages in the sequencer.

While at it, the funny "error: Error wrapping up..." was changed to a
less funny, but more helpful, "error: failed to finalize...".

Pointed out by Junio Hamano.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                   | 68 +++++++++++++++++++++----------------------
 t/t3501-revert-cherry-pick.sh |  2 +-
 2 files changed, 35 insertions(+), 35 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4596540..676f16c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -241,18 +241,18 @@ static int write_with_lock_file(const char *filename,
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
-		return error_errno(_("Could not lock '%s'"), filename);
+		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
 		rollback_lock_file(&msg_file);
-		return error_errno(_("Could not write to '%s'"), filename);
+		return error_errno(_("could not write to '%s'"), filename);
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
 		rollback_lock_file(&msg_file);
-		return error_errno(_("Could not write eol to '%s"), filename);
+		return error_errno(_("could not write eol to '%s"), filename);
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
-		return error(_("Error wrapping up '%s'."), filename);
+		return error(_("failed to finalize '%s'."), filename);
 	}
 
 	return 0;
@@ -312,11 +312,11 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(_(action_name(opts)));
 
-	error(_("Your local changes would be overwritten by %s."),
+	error(_("your local changes would be overwritten by %s."),
 		_(action_name(opts)));
 
 	if (advice_commit_before_merge)
-		advise(_("Commit your changes or stash them to proceed."));
+		advise(_("commit your changes or stash them to proceed."));
 	return -1;
 }
 
@@ -425,7 +425,7 @@ static int is_index_unchanged(void)
 	struct commit *head_commit;
 
 	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
-		return error(_("Could not resolve HEAD commit\n"));
+		return error(_("could not resolve HEAD commit\n"));
 
 	head_commit = lookup_commit(head_sha1);
 
@@ -445,7 +445,7 @@ static int is_index_unchanged(void)
 
 	if (!cache_tree_fully_valid(active_cache_tree))
 		if (cache_tree_update(&the_index, 0))
-			return error(_("Unable to update cache tree\n"));
+			return error(_("unable to update cache tree\n"));
 
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
@@ -515,7 +515,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error("You have staged changes in your working "
+			return error("you have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
 				"  git commit --amend %s\n\n"
@@ -568,12 +568,12 @@ static int is_original_commit_empty(struct commit *commit)
 	const unsigned char *ptree_sha1;
 
 	if (parse_commit(commit))
-		return error(_("Could not parse commit %s\n"),
+		return error(_("could not parse commit %s\n"),
 			     oid_to_hex(&commit->object.oid));
 	if (commit->parents) {
 		struct commit *parent = commit->parents->item;
 		if (parse_commit(parent))
-			return error(_("Could not parse parent commit %s\n"),
+			return error(_("could not parse parent commit %s\n"),
 				oid_to_hex(&parent->object.oid));
 		ptree_sha1 = parent->tree->object.oid.hash;
 	} else {
@@ -657,7 +657,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		 * to work on.
 		 */
 		if (write_cache_as_tree(head, 0, NULL))
-			return error(_("Your index file is unmerged."));
+			return error(_("your index file is unmerged."));
 	} else {
 		unborn = get_sha1("HEAD", head);
 		if (unborn)
@@ -676,7 +676,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct commit_list *p;
 
 		if (!opts->mainline)
-			return error(_("Commit %s is a merge but no -m option was given."),
+			return error(_("commit %s is a merge but no -m option was given."),
 				oid_to_hex(&commit->object.oid));
 
 		for (cnt = 1, p = commit->parents;
@@ -684,11 +684,11 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		     cnt++)
 			p = p->next;
 		if (cnt != opts->mainline || !p)
-			return error(_("Commit %s does not have parent %d"),
+			return error(_("commit %s does not have parent %d"),
 				oid_to_hex(&commit->object.oid), opts->mainline);
 		parent = p->item;
 	} else if (0 < opts->mainline)
-		return error(_("Mainline was specified but commit %s is not a merge."),
+		return error(_("mainline was specified but commit %s is not a merge."),
 			oid_to_hex(&commit->object.oid));
 	else
 		parent = commit->parents->item;
@@ -708,7 +708,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
-		return error(_("Cannot get commit message for %s"),
+		return error(_("cannot get commit message for %s"),
 			oid_to_hex(&commit->object.oid));
 
 	/*
@@ -944,13 +944,13 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
 		if (parse_insn_line(item, p, eol)) {
-			res = error(_("Invalid line %d: %.*s"),
+			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
 			item->command = -1;
 		}
 	}
 	if (!todo_list->nr)
-		return error(_("No commits parsed."));
+		return error(_("no commits parsed."));
 	return res;
 }
 
@@ -963,16 +963,16 @@ static int read_populate_todo(struct todo_list *todo_list,
 	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open '%s'"), todo_file);
+		return error_errno(_("could not open '%s'"), todo_file);
 	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		return error(_("Could not read '%s'."), todo_file);
+		return error(_("could not read '%s'."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
-		return error(_("Unusable instruction sheet: '%s'"), todo_file);
+		return error(_("unusable instruction sheet: '%s'"), todo_file);
 
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
@@ -983,9 +983,9 @@ static int read_populate_todo(struct todo_list *todo_list,
 			if (valid == todo_list->items[i].command)
 				continue;
 			else if (valid == TODO_PICK)
-				return error(_("Cannot cherry-pick during a revert."));
+				return error(_("cannot cherry-pick during a revert."));
 			else
-				return error(_("Cannot revert during a cherry-pick."));
+				return error(_("cannot revert during a cherry-pick."));
 	}
 
 	return 0;
@@ -1030,10 +1030,10 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
 		opts->xopts[opts->xopts_nr++] = xstrdup(value);
 	} else
-		return error(_("Invalid key: %s"), key);
+		return error(_("invalid key: %s"), key);
 
 	if (!error_flag)
-		return error(_("Invalid value for %s: %s"), key, value);
+		return error(_("invalid value for %s: %s"), key, value);
 
 	return 0;
 }
@@ -1065,7 +1065,7 @@ static int read_populate_opts(struct replay_opts *opts)
 	 * are pretty certain that it is syntactically correct.
 	 */
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
-		return error(_("Malformed options sheet: '%s'"),
+		return error(_("malformed options sheet: '%s'"),
 			git_path_opts_file());
 	return 0;
 }
@@ -1108,7 +1108,7 @@ static int create_seq_dir(void)
 		return -1;
 	}
 	else if (mkdir(git_path_seq_dir(), 0777) < 0)
-		return error_errno(_("Could not create sequencer directory '%s'"),
+		return error_errno(_("could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
 }
@@ -1122,17 +1122,17 @@ static int save_head(const char *head)
 	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
 	if (fd < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not lock HEAD"));
+		return error_errno(_("could not lock HEAD"));
 	}
 	strbuf_addf(&buf, "%s\n", head);
 	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not write to '%s'"),
+		return error_errno(_("could not write to '%s'"),
 				   git_path_head_file());
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-		return error(_("Error wrapping up '%s'."), git_path_head_file());
+		return error(_("failed to finalize '%s'."), git_path_head_file());
 	}
 	return 0;
 }
@@ -1211,14 +1211,14 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
-		return error_errno(_("Could not lock '%s'"), todo_path);
+		return error_errno(_("could not lock '%s'"), todo_path);
 	offset = next < todo_list->nr ?
 		todo_list->items[next].offset_in_buf : todo_list->buf.len;
 	if (write_in_full(fd, todo_list->buf.buf + offset,
 			todo_list->buf.len - offset) < 0)
-		return error_errno(_("Could not write to '%s'"), todo_path);
+		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
-		return error(_("Error wrapping up '%s'."), todo_path);
+		return error(_("failed to finalize '%s'."), todo_path);
 	return 0;
 }
 
@@ -1393,7 +1393,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 			create_seq_dir() < 0)
 		return -1;
 	if (get_sha1("HEAD", sha1) && (opts->action == REPLAY_REVERT))
-		return error(_("Can't revert as initial commit"));
+		return error(_("can't revert as initial commit"));
 	if (save_head(sha1_to_hex(sha1)))
 		return -1;
 	if (save_opts(opts))
diff --git a/t/t3501-revert-cherry-pick.sh b/t/t3501-revert-cherry-pick.sh
index 51f3bbb..394f000 100755
--- a/t/t3501-revert-cherry-pick.sh
+++ b/t/t3501-revert-cherry-pick.sh
@@ -96,7 +96,7 @@ test_expect_success 'revert forbidden on dirty working tree' '
 	echo content >extra_file &&
 	git add extra_file &&
 	test_must_fail git revert HEAD 2>errors &&
-	test_i18ngrep "Your local changes would be overwritten by " errors
+	test_i18ngrep "your local changes would be overwritten by " errors
 
 '
 
-- 
2.10.0.windows.1.325.ge6089c1

[PATCH v3 25/25] sequencer: mark all error messages for translation

[Johannes Schindelin]

There was actually only one error message that was not yet marked for
translation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 676f16c..86d86ce 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -515,16 +515,19 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error("you have staged changes in your working "
-				"tree. If these changes are meant to be\n"
-				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend %s\n\n"
-				"If they are meant to go into a new commit, "
-				"run:\n\n"
-				"  git commit %s\n\n"
-				"In both cases, once you're done, continue "
-				"with:\n\n"
-				"  git rebase --continue\n", gpg_opt, gpg_opt);
+			return error(_("you have staged changes in your "
+				       "working tree. If these changes are "
+				       "meant to be\n"
+				       "squashed into the previous commit, "
+				       "run:\n\n"
+				       "  git commit --amend %s\n\n"
+				       "If they are meant to go into a new "
+				       "commit, run:\n\n"
+				       "  git commit %s\n\n"
+				       "In both cases, once you're done, "
+				       "continue with:\n\n"
+				       "  git rebase --continue\n"),
+				     gpg_opt, gpg_opt);
 		}
 	}
 
-- 
2.10.0.windows.1.325.ge6089c1

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Let's just bite the bullet and rewrite the entire parser; the code now
> ...
> In particular, we choose to maintain the list of commands in an array
> instead of a linked list: this is flexible enough to allow us later on to
> even implement rebase -i's reordering of fixup!/squash! commits very
> easily (and with a very nice speed bonus, at least on Windows).
>
> While at it, do not stop at the first problem, but list *all* of the
> problems. This will help the user when the sequencer will do `rebase
> -i`'s work by allowing to address all issues in one go rather than going
> back and forth until the todo list is valid.

All sounds sensible.

>  	if (parent && parse_commit(parent) < 0)
> -		/* TRANSLATORS: The first %s will be "revert" or
> -		   "cherry-pick", the second %s a SHA1 */
> +		/*
> +		 * TRANSLATORS: The first %s will be a "todo" command like
> +		 * "revert" or "pick", the second %s a SHA1.
> +		 */

You may want to double check this with i18n folks; IIRC the tool
that extracts TRANSLATORS: comment was somewhat particular about
where that magic "TRANSLATORS:" token resides on a comment line and
that is why we have this multi-line comment formatted in an unusual
way.

Ahh, no you do not have to bug i18n folks.  47fbfded53 ("i18n: only
extract comments marked with "TRANSLATORS:"", 2014-04-17) is an
example of such an adjustment.

I just found it in CodingGuidelines, cbcfd4e3ea ("i18n: mention
"TRANSLATORS:" marker in Documentation/CodingGuidelines",
2014-04-18).

> +	while ((commit = get_revision(opts->revs))) {
> +		struct todo_item *item = append_new_todo(todo_list);
> +		const char *commit_buffer = get_commit_buffer(commit, NULL);
> +		const char *subject;
> +		int subject_len;
> +
> +		item->command = command;
> +		item->commit = commit;
> +		item->offset_in_buf = todo_list->buf.len;
> +		subject_len = find_commit_subject(commit_buffer, &subject);
> +		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
> +			find_unique_abbrev(commit->object.oid.hash,
> +				DEFAULT_ABBREV),
> +			subject_len, subject);

I am personally fine with this line; two things come to mind:

 - This would work just fine as-is with Linus's change to turn
   DEFAULT_ABBREV to -1.

 - It appears that it is more fashionable to use
   strbuf_add_unique_abbrev() these days.

Thanks.

Re: [PATCH v3 01/25] sequencer: use static initializers for replay_opts

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This change is not completely faithful: instead of initializing all fields
> to 0, we choose to initialize command and subcommand to -1 (instead of
> defaulting to REPLAY_REVERT and REPLAY_NONE, respectively). Practically,
> it makes no difference at all, but future-proofs the code to require
> explicit assignments for both fields.

The assignments to opts.action immediately following, I would say
this is quite faithful conversion that looks good.

>  int cmd_revert(int argc, const char **argv, const char *prefix)
>  {
> -	struct replay_opts opts;
> +	struct replay_opts opts = REPLAY_OPTS_INIT;
>  	int res;
>  
> -	memset(&opts, 0, sizeof(opts));
>  	if (isatty(0))
>  		opts.edit = 1;
>  	opts.action = REPLAY_REVERT;
> @@ -195,10 +194,9 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
>  
>  int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
>  {
> -	struct replay_opts opts;
> +	struct replay_opts opts = REPLAY_OPTS_INIT;
>  	int res;
>  
> -	memset(&opts, 0, sizeof(opts));
>  	opts.action = REPLAY_PICK;
>  	git_config(git_default_config, NULL);
>  	parse_args(argc, argv, &opts);
> diff --git a/sequencer.h b/sequencer.h
> index 5ed5cb1..db425ad 100644
> --- a/sequencer.h
> +++ b/sequencer.h
> @@ -47,6 +47,7 @@ struct replay_opts {
>  	/* Only used by REPLAY_NONE */
>  	struct rev_info *revs;
>  };
> +#define REPLAY_OPTS_INIT { -1, -1 }
>  
>  int sequencer_pick_revisions(struct replay_opts *opts);

Re: [PATCH v3 03/25] sequencer: avoid unnecessary indirection

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> We really do not need the *pointer to a* pointer to the options in
> the read_populate_opts() function.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

I vaguely recall seeing this in the previous round and finding it
pretty sensible.  And I still do ;-)


[the remainder left as-is to help those who are reading from
sidelines]

> diff --git a/sequencer.c b/sequencer.c
> index cb16cbd..c2fbf6f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -813,7 +813,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
>  	return 0;
>  }
>  
> -static int read_populate_opts(struct replay_opts **opts)
> +static int read_populate_opts(struct replay_opts *opts)
>  {
>  	if (!file_exists(git_path_opts_file()))
>  		return 0;
> @@ -823,7 +823,7 @@ static int read_populate_opts(struct replay_opts **opts)
>  	 * about this case, though, because we wrote that file ourselves, so we
>  	 * are pretty certain that it is syntactically correct.
>  	 */
> -	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
> +	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
>  		return error(_("Malformed options sheet: %s"),
>  			git_path_opts_file());
>  	return 0;
> @@ -1054,7 +1054,7 @@ static int sequencer_continue(struct replay_opts *opts)
>  
>  	if (!file_exists(git_path_todo_file()))
>  		return continue_single_pick();
> -	if (read_populate_opts(&opts) ||
> +	if (read_populate_opts(opts) ||
>  			read_populate_todo(&todo_list, opts))
>  		return -1;

Re: [PATCH v3 05/25] sequencer: eventually release memory allocated for the option values

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> diff --git a/builtin/revert.c b/builtin/revert.c
> index 7365559..fce9c75 100644
> --- a/builtin/revert.c
> +++ b/builtin/revert.c
> @@ -174,6 +174,12 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
>  
>  	if (argc > 1)
>  		usage_with_options(usage_str, options);
> +
> +	/* These option values will be free()d */
> +	if (opts->gpg_sign)
> +		opts->gpg_sign = xstrdup(opts->gpg_sign);
> +	if (opts->strategy)
> +		opts->strategy = xstrdup(opts->strategy);
>  }

This certainly is good, but I wonder if a new variant of OPT_STRING
and OPTION_STRING that does the strdup for you, something along the
lines of ...

diff --git a/parse-options.c b/parse-options.c
index 312a85dbde..6aab6b0b05 100644
--- a/parse-options.c
+++ b/parse-options.c
@@ -138,6 +138,21 @@ static int get_value(struct parse_opt_ctx_t *p,
 			return get_arg(p, opt, flags, (const char **)opt->value);
 		return 0;
 
+	case OPTION_STRDUP:
+		err = 0;
+		free(opt->value);
+		if (unset)
+			*(const char **)opt->value = NULL;
+		else if (opt->flags & PARSE_OPT_OPTARG && !p->opt)
+			*(const char **)opt->value = xstrdup(opt->defval);
+		else {
+			const char *v;
+			err = get_arg(p, opt, flags, &v);
+			if (!err)
+				*(const char **)opt->value = xstrdup(v);
+		}
+		return err;
+
 	case OPTION_FILENAME:
 		err = 0;
 		if (unset)

... may make it even more pleasant to use?  Only for two fields in
this patch that may probably be an overkill, but we may eventually 
benefit from such an approach when we audit and plug leaks in
parse-options users.  I dunno.

It is a sign that the caller wants to _own_ the memory to mark a
variable or field with OPTION_STRDUP, which is why I added the
free() at the beginning there.

The remainder of this patch looks sensible.  The code that frees
these fields when we are done with the struct (and when we are
re-assigning) looked all good.

Re: [PATCH v3 05/25] sequencer: eventually release memory allocated for the option values

[Johannes Schindelin]

Hi Junio,

On Mon, 10 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/builtin/revert.c b/builtin/revert.c
> > index 7365559..fce9c75 100644
> > --- a/builtin/revert.c
> > +++ b/builtin/revert.c
> > @@ -174,6 +174,12 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
> >  
> >  	if (argc > 1)
> >  		usage_with_options(usage_str, options);
> > +
> > +	/* These option values will be free()d */
> > +	if (opts->gpg_sign)
> > +		opts->gpg_sign = xstrdup(opts->gpg_sign);
> > +	if (opts->strategy)
> > +		opts->strategy = xstrdup(opts->strategy);
> >  }
> 
> This certainly is good, but I wonder if a new variant of OPT_STRING
> and OPTION_STRING that does the strdup for you, something along the
> lines of ...
> 
> diff --git a/parse-options.c b/parse-options.c
> index 312a85dbde..6aab6b0b05 100644
> --- a/parse-options.c
> +++ b/parse-options.c
> @@ -138,6 +138,21 @@ static int get_value(struct parse_opt_ctx_t *p,
>  			return get_arg(p, opt, flags, (const char **)opt->value);
>  		return 0;
>  
> +	case OPTION_STRDUP:
> +		err = 0;
> +		free(opt->value);

Probably

		free(*(char **)opt->value);

instead.

> +		if (unset)
> +			*(const char **)opt->value = NULL;
> +		else if (opt->flags & PARSE_OPT_OPTARG && !p->opt)
> +			*(const char **)opt->value = xstrdup(opt->defval);
> +		else {
> +			const char *v;
> +			err = get_arg(p, opt, flags, &v);
> +			if (!err)
> +				*(const char **)opt->value = xstrdup(v);
> +		}
> +		return err;
> +
>  	case OPTION_FILENAME:
>  		err = 0;
>  		if (unset)
> 
> ... may make it even more pleasant to use?

With s/even// I would agree.

I will keep this patch in mind and will try to come back to it, once the
rebase--helper patches are well on target for `master`.

Ciao,
Dscho

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

Hi Junio,

On Mon, 10 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> >  	if (parent && parse_commit(parent) < 0)
> > -		/* TRANSLATORS: The first %s will be "revert" or
> > -		   "cherry-pick", the second %s a SHA1 */
> > +		/*
> > +		 * TRANSLATORS: The first %s will be a "todo" command like
> > +		 * "revert" or "pick", the second %s a SHA1.
> > +		 */
> 
> You may want to double check this with i18n folks; IIRC the tool
> that extracts TRANSLATORS: comment was somewhat particular about
> where that magic "TRANSLATORS:" token resides on a comment line and
> that is why we have this multi-line comment formatted in an unusual
> way.
> 
> Ahh, no you do not have to bug i18n folks.  47fbfded53 ("i18n: only
> extract comments marked with "TRANSLATORS:"", 2014-04-17) is an
> example of such an adjustment.

Urgh. Thanks for pointing this out to me, though. Will be fixed in the
next iteration.

> > +	while ((commit = get_revision(opts->revs))) {
> > +		struct todo_item *item = append_new_todo(todo_list);
> > +		const char *commit_buffer = get_commit_buffer(commit, NULL);
> > +		const char *subject;
> > +		int subject_len;
> > +
> > +		item->command = command;
> > +		item->commit = commit;
> > +		item->offset_in_buf = todo_list->buf.len;
> > +		subject_len = find_commit_subject(commit_buffer, &subject);
> > +		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
> > +			find_unique_abbrev(commit->object.oid.hash,
> > +				DEFAULT_ABBREV),
> > +			subject_len, subject);
> 
> I am personally fine with this line; two things come to mind:
> 
>  - This would work just fine as-is with Linus's change to turn
>    DEFAULT_ABBREV to -1.
> 
>  - It appears that it is more fashionable to use
>    strbuf_add_unique_abbrev() these days.

Right, I actually looked at this place when I tried to decide where I
could use that function. Somehow I thought I'd not break up the flow here.

But since you asked so nicely, I'll squash this in (I personally find it
uglier, and longer, but it does use strbuf_add_unique_abbrev() now):

-- snipsnap --
@@ -906,11 +904,13 @@ static int walk_revs_populate_todo(struct todo_list
*todo_list,
                item->command = command;
                item->commit = commit;
                item->offset_in_buf = todo_list->buf.len;
+               strbuf_addstr(&todo_list->buf, command);
+               strbuf_addch(&todo_list->buf, ' ');
+               strbuf_add_unique_abbrev(&todo_list->buf,
+                                        commit->object.oid.hash,
+                                        DEFAULT_ABBREV);
                subject_len = find_commit_subject(commit_buffer, &subject);
-               strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
                command_string,
-                       find_unique_abbrev(commit->object.oid.hash,
-                               DEFAULT_ABBREV),
-                       subject_len, subject);
+               strbuf_add(&todo_list->buf, subject, subject_len);
                unuse_commit_buffer(commit, commit_buffer);
        }
        return 0;

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

Hi Junio,

On Tue, 11 Oct 2016, Johannes Schindelin wrote:

> -- snipsnap --
> @@ -906,11 +904,13 @@ static int walk_revs_populate_todo(struct todo_list
> *todo_list,
>                 item->command = command;
>                 item->commit = commit;
>                 item->offset_in_buf = todo_list->buf.len;
> +               strbuf_addstr(&todo_list->buf, command);

This would be command_string instead of command, of course.

> +               strbuf_addch(&todo_list->buf, ' ');
> +               strbuf_add_unique_abbrev(&todo_list->buf,
> +                                        commit->object.oid.hash,
> +                                        DEFAULT_ABBREV);
>                 subject_len = find_commit_subject(commit_buffer, &subject);
> -               strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
>                 command_string,
> -                       find_unique_abbrev(commit->object.oid.hash,
> -                               DEFAULT_ABBREV),
> -                       subject_len, subject);
> +               strbuf_add(&todo_list->buf, subject, subject_len);
>                 unuse_commit_buffer(commit, commit_buffer);
>         }
>         return 0;

Ciao,
Dscho

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

Hi Junio,

On Tue, 11 Oct 2016, Johannes Schindelin wrote:

> On Tue, 11 Oct 2016, Johannes Schindelin wrote:
> 
> > -- snipsnap --
> > @@ -906,11 +904,13 @@ static int walk_revs_populate_todo(struct todo_list
> > *todo_list,
> >                 item->command = command;
> >                 item->commit = commit;
> >                 item->offset_in_buf = todo_list->buf.len;
> > +               strbuf_addstr(&todo_list->buf, command);
> 
> This would be command_string instead of command, of course.
> 
> > +               strbuf_addch(&todo_list->buf, ' ');
> > +               strbuf_add_unique_abbrev(&todo_list->buf,
> > +                                        commit->object.oid.hash,
> > +                                        DEFAULT_ABBREV);
> >                 subject_len = find_commit_subject(commit_buffer, &subject);
> > -               strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
> >                 command_string,
> > -                       find_unique_abbrev(commit->object.oid.hash,
> > -                               DEFAULT_ABBREV),
> > -                       subject_len, subject);
> > +               strbuf_add(&todo_list->buf, subject, subject_len);
> >                 unuse_commit_buffer(commit, commit_buffer);
> >         }
> >         return 0;

In the end, I decided to actually *not* use strbuf_add_unique_abbrev()
here because it really makes the code very much too ugly after the
introduction of short_commit_name():

-- snip --
@@ -1093,10 +1093,16 @@ static int walk_revs_populate_todo(struct
todo_list *todo_list,
                item->arg = NULL;
                item->arg_len = 0;
                item->offset_in_buf = todo_list->buf.len;
+               strbuf_addstr(&todo_list->buf, command_string);
+               strbuf_addch(&todo_list->buf, ' ');
+               strbuf_add_unique_abbrev(&todo_list->buf,
+                                        commit->object.oid.hash,
+                                        DEFAULT_ABBREV);
+               strbuf_addch(&todo_list->buf, ' ');
                subject_len = find_commit_subject(commit_buffer, &subject);
-               strbuf_addf(&todo_list->buf, "%s %s %.*s\n",
                command_string,
-                       short_commit_name(commit), subject_len, subject);
+               strbuf_add(&todo_list->buf, subject, subject_len);
                unuse_commit_buffer(commit, commit_buffer);
+               strbuf_addch(&todo_list->buf, '\n');
        }
        return 0;
 }
-- snap --

I hope you will forgive me disagreeing with you here.

To make it easier to accept, I reordered the short_commit_name() patch so
it comes before revamping the todo parsing.

Ciao,
Dscho

Re: [PATCH v3 05/25] sequencer: eventually release memory allocated for the option values

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> This certainly is good, but I wonder if a new variant of OPT_STRING
> and OPTION_STRING that does the strdup for you, something along the
> lines of ...
> ... may make it even more pleasant to use?  Only for two fields in
> this patch that may probably be an overkill, but we may eventually 
> benefit from such an approach when we audit and plug leaks in
> parse-options users.  I dunno.

After sleeping on it, I do think this is an overkill.  The pattern I
would expect for the most normal (boring) code to use is rather:

    struct app_specific app;
    const char *opt_x = NULL;
    struct option options[] = {
	...
	OPT_STRING(0, "xopt", &opt_x, N_("x option"), ...),
        ...
	OPT_END()
    };

    parse_options(ac, av, prefix, options, ...);
    app.x_field = xstrdup_or_null(opt_x);
    ... other values set to app's field based on
    ... not just command line options but from
    ... other sources.

The only reason why the OPT_STRDUP appeared convenient was because
options[] element happened to use a field in the structure directly.
The patch under discussion does an equivalent of

    app.x_field = xstrdup_or_null(opt_x);

but the "opt_x" happens to be the same "app.x_field" in this case,
so in that sense, it follows the normal and boring pattern.

The "struct app_specific" may not even exist in the same scope as
the caller of parse_options(), but may have to be initialized in a
function that is three-level deep in the callchain, with opt_x
variable passed through as a parameter.  So OPT_STRDUP may not be a
bad or horrible idea, but it is not such a great one, either.

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I am personally fine with this line; two things come to mind:
>> 
>>  - This would work just fine as-is with Linus's change to turn
>>    DEFAULT_ABBREV to -1.
>> 
>>  - It appears that it is more fashionable to use
>>    strbuf_add_unique_abbrev() these days.
>
> Right, I actually looked at this place when I tried to decide where I
> could use that function. Somehow I thought I'd not break up the flow here.
>
> But since you asked so nicely,...

When I say "I am fine", I am not asking you to change anything.

Some of the places that have been updated recently to use
strbuf_add_unique_abbrev() in other topics did improve the
readability of the code, but I do not think it would universally
be true.

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> In the end, I decided to actually *not* use strbuf_add_unique_abbrev()
> here because it really makes the code very much too ugly after the
> introduction of short_commit_name():

It's perfectly fine not to use that function when it does not make
sense; we shouldn't use it (or anything for that matter) just for
the sake of using it.

Re: [PATCH v3 08/25] sequencer: strip CR from the todo script

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> It is not unheard of that editors on Windows write CR/LF even if the
> file originally had only LF. This is particularly awkward for exec lines
> of a rebase -i todo sheet. Take for example the insn "exec echo": The
> shell script parser splits at the LF and leaves the CR attached to
> "echo", which leads to the unknown command "echo\r".
>
> Work around that by stripping CR when reading the todo commands, as we
> already do for LF.
>
> This happens to fix t9903.14 and .15 in MSYS1 environments (with the
> rebase--helper patches based on this patch series): the todo script
> constructed in such a setup contains CR/LF thanks to MSYS1 runtime's
> cleverness.
>
> Based on a report and a patch by Johannes Sixt.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/sequencer.c b/sequencer.c
> index 678fdf3..cee7e50 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -774,6 +774,9 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
>  
>  		next_p = *eol ? eol + 1 /* skip LF */ : eol;
>  
> +		if (p != eol && eol[-1] == '\r')
> +			eol--; /* skip Carriage Return */

micronit: s/skip/strip/ ;-)

> +
>  		item = append_new_todo(todo_list);
>  		item->offset_in_buf = p - todo_list->buf.buf;
>  		if (parse_insn_line(item, p, eol)) {

Re: [PATCH v3 12/25] sequencer: remember the onelines when parsing the todo file

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The `git-rebase-todo` file contains a list of commands. Most of those
> commands have the form
>
> 	<verb> <sha1> <oneline>
>
> The <oneline> is displayed primarily for the user's convenience, as
> rebase -i really interprets only the <verb> <sha1> part. However, there
> are *some* places in interactive rebase where the <oneline> is used to
> display messages, e.g. for reporting at which commit we stopped.
>
> So let's just remember it when parsing the todo file; we keep a copy of
> the entire todo file anyway (to write out the new `done` and
> `git-rebase-todo` file just before processing each command), so all we
> need to do is remember the begin offsets and lengths.
>
> As we will have to parse and remember the command-line for `exec` commands
> later, we do not call the field "oneline" but rather "arg" (and will reuse
> that for exec's command-line).
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/sequencer.c b/sequencer.c
> index afc494e..7ba5e07 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -708,6 +708,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
>  struct todo_item {
>  	enum todo_command command;
>  	struct commit *commit;
> +	const char *arg;
> +	int arg_len;
>  	size_t offset_in_buf;

micronit: you can make it to size_t and lose the cast below, no?

>  };
>  
> @@ -759,6 +761,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
>  	status = get_sha1(bol, commit_sha1);
>  	*end_of_object_name = saved;
>  
> +	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
> +	item->arg_len = (int)(eol - item->arg);
> +
>  	if (status < 0)
>  		return -1;
>  
> @@ -911,6 +916,8 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
>  
>  		item->command = command;
>  		item->commit = commit;
> +		item->arg = NULL;
> +		item->arg_len = 0;
>  		item->offset_in_buf = todo_list->buf.len;
>  		subject_len = find_commit_subject(commit_buffer, &subject);
>  		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,

Re: [PATCH v3 13/25] sequencer: prepare for rebase -i's commit functionality

[Junio C Hamano]

> @@ -370,19 +383,79 @@ static int is_index_unchanged(void)
>  }
>  
>  /*
> + * Read the author-script file into an environment block, ready for use in
> + * run_command(), that can be free()d afterwards.
> + */
> +static char **read_author_script(void)
> +{
> +	struct strbuf script = STRBUF_INIT;
> +	int i, count = 0;
> +	char *p, *p2, **env;
> +	size_t env_size;
> +
> +	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
> +		return NULL;
> +
> +	for (p = script.buf; *p; p++)
> +		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
> +			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
> +		else if (*p == '\'')
> +			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
> +		else if (*p == '\n') {
> +			*p = '\0';
> +			count++;
> +		}

Hmph, didn't we recently add parse_key_value_squoted() to build
read_author_script() in builtin/am.c on top of it, so that this
piece of code can also take advantage of and share the parser?

> +/*

Offtopic: this line and the beginning of the new comment block that
begins with "Read the author-script" above show a suboptimal marking
of what is added and what is left.  I wonder "diff-indent-heuristic"
topic by Michael can help to make it look better.

>   * If we are cherry-pick, and if the merge did not result in
>   * hand-editing, we will hit this commit and inherit the original
>   * author date and name.
> + *
>   * If we are revert, or if our cherry-pick results in a hand merge,
>   * we had better say that the current user is responsible for that.
> + *
> + * An exception is when run_git_commit() is called during an
> + * interactive rebase: in that case, we will want to retain the
> + * author metadata.
>   */

Re: [PATCH v3 08/25] sequencer: strip CR from the todo script

[Johannes Schindelin]

Hi Junio,

On Tue, 11 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/sequencer.c b/sequencer.c
> > index 678fdf3..cee7e50 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -774,6 +774,9 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
> >  
> >  		next_p = *eol ? eol + 1 /* skip LF */ : eol;
> >  
> > +		if (p != eol && eol[-1] == '\r')
> > +			eol--; /* skip Carriage Return */
> 
> micronit: s/skip/strip/ ;-)

Okay,
Dscho

Re: [PATCH v3 12/25] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

Hi Junio,

On Tue, 11 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > diff --git a/sequencer.c b/sequencer.c
> > index afc494e..7ba5e07 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -708,6 +708,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
> >  struct todo_item {
> >  	enum todo_command command;
> >  	struct commit *commit;
> > +	const char *arg;
> > +	int arg_len;
> >  	size_t offset_in_buf;
> 
> micronit: you can make it to size_t and lose the cast below, no?

No. The primary users of arg_len call a printf() style function with %.*s,
expecting an int. So your suggestion would lose one cast, but introduce at
least four casts in return.

Ciao,
Dscho

Re: [PATCH v3 13/25] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

Hi Junio,

On Tue, 11 Oct 2016, Junio C Hamano wrote:

> > @@ -370,19 +383,79 @@ static int is_index_unchanged(void)
> >  }
> >  
> >  /*
> > + * Read the author-script file into an environment block, ready for use in
> > + * run_command(), that can be free()d afterwards.
> > + */
> > +static char **read_author_script(void)
> > +{
> > +	struct strbuf script = STRBUF_INIT;
> > +	int i, count = 0;
> > +	char *p, *p2, **env;
> > +	size_t env_size;
> > +
> > +	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
> > +		return NULL;
> > +
> > +	for (p = script.buf; *p; p++)
> > +		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
> > +			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
> > +		else if (*p == '\'')
> > +			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
> > +		else if (*p == '\n') {
> > +			*p = '\0';
> > +			count++;
> > +		}
> 
> Hmph, didn't we recently add parse_key_value_squoted() to build
> read_author_script() in builtin/am.c on top of it, so that this
> piece of code can also take advantage of and share the parser?

I already pointed out that the author-script file may *not* be quoted.
sq_dequote() would return NULL and parse_key_value_squoted() would *fail*.

To complicate things further, the sequencer does not even need to access
the values at all. It needs to pass them to run_command() as an
environment block, which means that we would have to reconstruct the lines
after parse_key_value_squoted() painstakingly untangled the key names from
the values.

In short, this is another instance where using a function just because it
exists and is nominally related would make the resulting patch *more*
complicated than it currently is.

> > +/*
> 
> Offtopic: this line and the beginning of the new comment block that
> begins with "Read the author-script" above show a suboptimal marking
> of what is added and what is left.  I wonder "diff-indent-heuristic"
> topic by Michael can help to make it look better.

Maybe. I'll try to look into that once the more serious questions about
this patch series have been addressed.

Ciao,
Dscho

Re: [PATCH v3 05/25] sequencer: eventually release memory allocated for the option values

[Johannes Schindelin]

Hi Junio,

On Tue, 11 Oct 2016, Junio C Hamano wrote:

> The only reason why the OPT_STRDUP appeared convenient was because
> options[] element happened to use a field in the structure directly.
> The patch under discussion does an equivalent of
> 
>     app.x_field = xstrdup_or_null(opt_x);

Oh, that xstrdup_or_null() function slipped by me. My local patches use it
now.

Ciao,
Dscho

Re: [PATCH v3 12/25] sequencer: remember the onelines when parsing the todo file

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> > +	const char *arg;
>> > +	int arg_len;
>> >  	size_t offset_in_buf;
>> 
>> micronit: you can make it to size_t and lose the cast below, no?
>
> No. The primary users of arg_len call a printf() style function with %.*s,
> expecting an int. So your suggestion would lose one cast, but introduce at
> least four casts in return.

Actually my point was not the number of casts required, but more
about using the correct type to store things.  Granted, I do not
expect each of the lines would ever get too long to exceed "int"
(but fit in "size_t") in practice, and from that point of view, one
may be able to argue that "int" and "size_t" are both correct types,
but that argument applies equally to offset_in_buf, so...

Re: [PATCH v3 13/25] sequencer: prepare for rebase -i's commit functionality

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> Hmph, didn't we recently add parse_key_value_squoted() to build
>> read_author_script() in builtin/am.c on top of it, so that this
>> piece of code can also take advantage of and share the parser?
>
> I already pointed out that the author-script file may *not* be quoted.

I think my puzzlement comes from here.  What makes it OK for "am" to
expect the contents of author-script file to be quoted but it is not
OK to expect the same here?  What makes it not quoted for _this_
reader, in other words?

I am not sure what you meant by "nominally related", but the purpose
of the author-script in these two codepaths is the same, isn't it?
Somebody leaves the author information from the source (either from
an e-mailed patch or an existing commit), so that a later step can
use that pieces of information left in the file when (re)creating a
commit to record the tree made by using pieces of information from
the source.

Are our use in the author-script in these two codepaths _already_
inconsistent?  IOW, "am" never writes malformed unquoted values,
while the sequencer writes out in a way that is randomly quoted or
not quoted, iow, if you fed such an author-file to "am", it wouldn't
understand it?

I fully support your position to use different codepaths, if the
file that has the same name and that is used for the same purpose
uses different format in these two separate codepaths and the users
already expect them to be different.  We obviously need to have two
separate parsers.

But if that is not the case, IOW, if "am"'s author-script shares the
same issue (i.e. "'am' initially writes the file properly quoted,
but this or that can happen to change its quoting and we need to
read from such a file"), then perhaps sharing needs to happen the
other way around?  This patch may prepare "rebase -i" side for the
"this or that" (I still do not know what they are) to allow the
resulting file read correctly, but the same "this or that" can break
what "am" has used and is in use there if that is the case, no?

What makes it OK for "am" to expect the contents of author-script
file to be quoted but it is not OK to expect the same here?  What
makes it not quoted for _this_ reader, and doesn't "am" share the
same issue?

>> > +/*
>> 
>> Offtopic: this line and the beginning of the new comment block that
>> begins with "Read the author-script" above show a suboptimal marking
>> of what is added and what is left.  I wonder "diff-indent-heuristic"
>> topic by Michael can help to make it look better.
>
> Maybe. I'll try to look into that once the more serious questions about
> this patch series have been addressed.

You do not have to; the remark was meant for Michael (newly cc'ed in
the message you are responding to).

Re* [PATCH v3 05/25] sequencer: eventually release memory allocated for the option values

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Tue, 11 Oct 2016, Junio C Hamano wrote:
>
>> The only reason why the OPT_STRDUP appeared convenient was because
>> options[] element happened to use a field in the structure directly.
>> The patch under discussion does an equivalent of
>> 
>>     app.x_field = xstrdup_or_null(opt_x);
>
> Oh, that xstrdup_or_null() function slipped by me. My local patches use it
> now.

It has slipped many people ;-)

-- >8 --
Subject: cocci: refactor common patterns to use xstrdup_or_null()

d64ea0f83b ("git-compat-util: add xstrdup_or_null helper",
2015-01-12) added a handy wrapper that allows us to get a duplicate
of a string or NULL if the original is NULL, but a handful of
codepath predate its introduction or just weren't aware of it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 contrib/coccinelle/xstrdup_or_null.cocci | 7 +++++++
 git.c                                    | 3 +--
 imap-send.c                              | 6 ++----
 mailmap.c                                | 6 ++----
 refs.c                                   | 3 +--
 send-pack.c                              | 3 +--
 trailer.c                                | 9 +++------
 7 files changed, 17 insertions(+), 20 deletions(-)

diff --git a/contrib/coccinelle/xstrdup_or_null.cocci b/contrib/coccinelle/xstrdup_or_null.cocci
new file mode 100644
index 0000000000..3fceef132b
--- /dev/null
+++ b/contrib/coccinelle/xstrdup_or_null.cocci
@@ -0,0 +1,7 @@
+@@
+expression E;
+expression V;
+@@
+- if (E)
+-    V = xstrdup(E);
++ V = xstrdup_or_null(E);
diff --git a/git.c b/git.c
index 0f1937fd0c..f914490e14 100644
--- a/git.c
+++ b/git.c
@@ -35,8 +35,7 @@ static void save_env_before_alias(void)
 	orig_cwd = xgetcwd();
 	for (i = 0; i < ARRAY_SIZE(env_names); i++) {
 		orig_env[i] = getenv(env_names[i]);
-		if (orig_env[i])
-			orig_env[i] = xstrdup(orig_env[i]);
+		orig_env[i] = xstrdup_or_null(orig_env[i]);
 	}
 }
 
diff --git a/imap-send.c b/imap-send.c
index 0f5f4760e9..9514ddc565 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -1082,10 +1082,8 @@ static struct imap_store *imap_open_store(struct imap_server_conf *srvc, char *f
 			cred.protocol = xstrdup(srvc->use_ssl ? "imaps" : "imap");
 			cred.host = xstrdup(srvc->host);
 
-			if (srvc->user)
-				cred.username = xstrdup(srvc->user);
-			if (srvc->pass)
-				cred.password = xstrdup(srvc->pass);
+			cred.username = xstrdup_or_null(srvc->user);
+			cred.password = xstrdup_or_null(srvc->pass);
 
 			credential_fill(&cred);
 
diff --git a/mailmap.c b/mailmap.c
index b5c521fdea..c1a79c100c 100644
--- a/mailmap.c
+++ b/mailmap.c
@@ -103,10 +103,8 @@ static void add_mapping(struct string_list *map,
 	} else {
 		struct mailmap_info *mi = xcalloc(1, sizeof(struct mailmap_info));
 		debug_mm("mailmap: adding (complex) entry for '%s'\n", old_email);
-		if (new_name)
-			mi->name = xstrdup(new_name);
-		if (new_email)
-			mi->email = xstrdup(new_email);
+		mi->name = xstrdup_or_null(new_name);
+		mi->email = xstrdup_or_null(new_email);
 		string_list_insert(&me->namemap, old_name)->util = mi;
 	}
 
diff --git a/refs.c b/refs.c
index b4e7cac7b2..62055ab091 100644
--- a/refs.c
+++ b/refs.c
@@ -791,8 +791,7 @@ struct ref_update *ref_transaction_add_update(
 		hashcpy(update->new_sha1, new_sha1);
 	if (flags & REF_HAVE_OLD)
 		hashcpy(update->old_sha1, old_sha1);
-	if (msg)
-		update->msg = xstrdup(msg);
+	update->msg = xstrdup_or_null(msg);
 	return update;
 }
 
diff --git a/send-pack.c b/send-pack.c
index 90f2ac51a7..6195b43e9a 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -181,8 +181,7 @@ static int receive_status(int in, struct ref *refs)
 			hint->status = REF_STATUS_REMOTE_REJECT;
 			ret = -1;
 		}
-		if (msg)
-			hint->remote_status = xstrdup(msg);
+		hint->remote_status = xstrdup_or_null(msg);
 		/* start our next search from the next ref */
 		hint = hint->next;
 	}
diff --git a/trailer.c b/trailer.c
index c6ea9ac64d..aecaf9232a 100644
--- a/trailer.c
+++ b/trailer.c
@@ -428,12 +428,9 @@ static int set_if_missing(struct conf_info *item, const char *value)
 static void duplicate_conf(struct conf_info *dst, struct conf_info *src)
 {
 	*dst = *src;
-	if (src->name)
-		dst->name = xstrdup(src->name);
-	if (src->key)
-		dst->key = xstrdup(src->key);
-	if (src->command)
-		dst->command = xstrdup(src->command);
+	dst->name = xstrdup_or_null(src->name);
+	dst->key = xstrdup_or_null(src->key);
+	dst->command = xstrdup_or_null(src->command);
 }
 
 static struct trailer_item *get_conf_item(const char *name)

Re: [PATCH v3 25/25] sequencer: mark all error messages for translation

[Johannes Sixt]

Am 10.10.2016 um 19:26 schrieb Johannes Schindelin:
> There was actually only one error message that was not yet marked for
> translation.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 23 +++++++++++++----------
>  1 file changed, 13 insertions(+), 10 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 676f16c..86d86ce 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -515,16 +515,19 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
>  		if (!env) {
>  			const char *gpg_opt = gpg_sign_opt_quoted(opts);
>  
> -			return error("you have staged changes in your working "
> -				"tree. If these changes are meant to be\n"
> -				"squashed into the previous commit, run:\n\n"
> -				"  git commit --amend %s\n\n"
> -				"If they are meant to go into a new commit, "
> -				"run:\n\n"
> -				"  git commit %s\n\n"
> -				"In both cases, once you're done, continue "
> -				"with:\n\n"
> -				"  git rebase --continue\n", gpg_opt, gpg_opt);
> +			return error(_("you have staged changes in your "
> +				       "working tree. If these changes are "
> +				       "meant to be\n"
> +				       "squashed into the previous commit, "
> +				       "run:\n\n"
> +				       "  git commit --amend %s\n\n"
> +				       "If they are meant to go into a new "
> +				       "commit, run:\n\n"
> +				       "  git commit %s\n\n"
> +				       "In both cases, once you're done, "
> +				       "continue with:\n\n"
> +				       "  git rebase --continue\n"),
> +				     gpg_opt, gpg_opt);
>  		}
>  	}
>  
> 

Can we please have the following change instead? I think it makes sense
to deviate from the usual conventions in a case like this.

Note that this is an error() text, hence, there should not be a
fullstop on the first line. That's now a good excuse to start the next
sentence on a new line; hence, this is not a faithful conversion to _()
anymore (a will happily take authorship and all blame if you don't
want to for this reason). Also note that _( is not moved to the
beginning of the line because it would be picked up as hunk header by
git diff.

---- 8< ----
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: [PATCH] sequencer: mark all error messages for translation

There was actually only one error message that was not yet marked for
translation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 sequencer.c | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 95a382e..79f7aa4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -515,16 +515,20 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error("you have staged changes in your working "
-				"tree. If these changes are meant to be\n"
-				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend %s\n\n"
-				"If they are meant to go into a new commit, "
-				"run:\n\n"
-				"  git commit %s\n\n"
-				"In both cases, once you're done, continue "
-				"with:\n\n"
-				"  git rebase --continue\n", gpg_opt, gpg_opt);
+			return error(_(
+"you have staged changes in your working tree\n"
+"If these changes are meant to be squashed into the previous commit, run:\n"
+"\n"
+"  git commit --amend %s\n"
+"\n"
+"If they are meant to go into a new commit, run:\n"
+"\n"
+"  git commit %s\n"
+"\n"
+"In both cases, once you're done, continue with:\n"
+"\n"
+"  git rebase --continue\n"),
+				     gpg_opt, gpg_opt);
 		}
 	}
 
-- 
2.10.0.343.g37bc62b

Re: [PATCH v3 25/25] sequencer: mark all error messages for translation

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> Can we please have the following change instead? I think it makes sense
> to deviate from the usual conventions in a case like this.

You have at least two independent changes relative to Dscho's
version.  

 (1) Show line breaks more prominently by avoiding "\n\n" and
     breaking the string at "\n"; this matches the way how the
     result would be displayed more closely to how the source looks
     like.

 (2) Ignore the usual indentation rule and have messages start at
     the left end of the source.

Which one are you saying "makes sense" to?  Both?

I guess both can be grouped together into one theme: match the way
the final output and the source code look like.

If that is the motivation behind "makes sense", I'd prefer to see
the change explained explicitly with that rationale in the log
message.

Thanks.  I personally agree with that motivation (if the one I
guessed above is your motivation, that is).

> Note that this is an error() text, hence, there should not be a
> fullstop on the first line. That's now a good excuse to start the next
> sentence on a new line; hence, this is not a faithful conversion to _()
> anymore (a will happily take authorship and all blame if you don't
> want to for this reason). Also note that _( is not moved to the
> beginning of the line because it would be picked up as hunk header by
> git diff.
>
> ---- 8< ----
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> Subject: [PATCH] sequencer: mark all error messages for translation
>
> There was actually only one error message that was not yet marked for
> translation.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> Signed-off-by: Johannes Sixt <j6t@kdbg.org>
> ---
>  sequencer.c | 24 ++++++++++++++----------
>  1 file changed, 14 insertions(+), 10 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index 95a382e..79f7aa4 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -515,16 +515,20 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
>  		if (!env) {
>  			const char *gpg_opt = gpg_sign_opt_quoted(opts);
>  
> -			return error("you have staged changes in your working "
> -				"tree. If these changes are meant to be\n"
> -				"squashed into the previous commit, run:\n\n"
> -				"  git commit --amend %s\n\n"
> -				"If they are meant to go into a new commit, "
> -				"run:\n\n"
> -				"  git commit %s\n\n"
> -				"In both cases, once you're done, continue "
> -				"with:\n\n"
> -				"  git rebase --continue\n", gpg_opt, gpg_opt);
> +			return error(_(
> +"you have staged changes in your working tree\n"
> +"If these changes are meant to be squashed into the previous commit, run:\n"
> +"\n"
> +"  git commit --amend %s\n"
> +"\n"
> +"If they are meant to go into a new commit, run:\n"
> +"\n"
> +"  git commit %s\n"
> +"\n"
> +"In both cases, once you're done, continue with:\n"
> +"\n"
> +"  git rebase --continue\n"),
> +				     gpg_opt, gpg_opt);
>  		}
>  	}

Re: [PATCH v3 12/25] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

Hi Junio,

On Wed, 12 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> > +	const char *arg;
> >> > +	int arg_len;
> >> >  	size_t offset_in_buf;
> >> 
> >> micronit: you can make it to size_t and lose the cast below, no?
> >
> > No. The primary users of arg_len call a printf() style function with %.*s,
> > expecting an int. So your suggestion would lose one cast, but introduce at
> > least four casts in return.
> 
> Actually my point was not the number of casts required, but more
> about using the correct type to store things.  Granted, I do not
> expect each of the lines would ever get too long to exceed "int"
> (but fit in "size_t") in practice, and from that point of view, one
> may be able to argue that "int" and "size_t" are both correct types,
> but that argument applies equally to offset_in_buf, so...

You cannot make it a size_t without changing the printf() standard to
expect size_t for %.*s, because that is the intended usage of that field
in these here patches.

Ciao,
Dscho

Re: [PATCH v3 13/25] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

Hi Junio,

On Wed, 12 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> Hmph, didn't we recently add parse_key_value_squoted() to build
> >> read_author_script() in builtin/am.c on top of it, so that this
> >> piece of code can also take advantage of and share the parser?
> >
> > I already pointed out that the author-script file may *not* be quoted.
> 
> I think my puzzlement comes from here.  What makes it OK for "am" to
> expect the contents of author-script file to be quoted but it is not
> OK to expect the same here?  What makes it not quoted for _this_
> reader, in other words?

The `git am` command is inherently *not* interactive, while the
interactive rebase, well, is.

As such, we must assume that enterprisey users did come up with scripts
that edit, or create, author-script files, and exploited the fact that the
interactive rebase previously sourced them.

Come to think of it, there is a bigger problem here, as users might have
abused the author-script to execute commands in rebase -i's own context.
Not sure we can do anything about that.

But the point stands, if anybody used unquoted, or differently quoted,
values in author-script, we should at least attempt within reason to
support that.

> I am not sure what you meant by "nominally related", but the purpose
> of the author-script in these two codepaths is the same, isn't it?

The purpose is, but the means aren't. As I pointed out above, the
interactive rebase is inherently much more interactive, and needs to be
much more forgiving in its input, than `git am`.

> Somebody leaves the author information from the source (either from
> an e-mailed patch or an existing commit), so that a later step can
> use that pieces of information left in the file when (re)creating a
> commit to record the tree made by using pieces of information from
> the source.
> 
> Are our use in the author-script in these two codepaths _already_
> inconsistent?  IOW, "am" never writes malformed unquoted values,
> while the sequencer writes out in a way that is randomly quoted or
> not quoted, iow, if you fed such an author-file to "am", it wouldn't
> understand it?

We heed Postel's Law: what the sequencer writes is in a very strict
format, but what the sequencer accepts need not be.

> I fully support your position to use different codepaths, if the
> file that has the same name and that is used for the same purpose
> uses different format in these two separate codepaths and the users
> already expect them to be different.  We obviously need to have two
> separate parsers.

Well, traditionally we *do* have separate parsers. I do not say that we
need to keep that, but given what I said above, it might not be a bad idea
to keep the lenient parser required by `git rebase -i` separate from the
one used by `git am` so that the latter can be faster (by making
assumptions the other parser cannot).

> But if that is not the case, IOW, if "am"'s author-script shares the
> same issue (i.e. "'am' initially writes the file properly quoted,
> but this or that can happen to change its quoting and we need to
> read from such a file"), then perhaps sharing needs to happen the
> other way around?  This patch may prepare "rebase -i" side for the
> "this or that" (I still do not know what they are) to allow the
> resulting file read correctly, but the same "this or that" can break
> what "am" has used and is in use there if that is the case, no?
> 
> What makes it OK for "am" to expect the contents of author-script
> file to be quoted but it is not OK to expect the same here?  What
> makes it not quoted for _this_ reader, and doesn't "am" share the
> same issue?

The fact that `git am` is *non-interactive*.

Ciao,
Dscho

Re: Re* [PATCH v3 05/25] sequencer: eventually release memory allocated for the option values

[Johannes Schindelin]

Hi Junio,

On Wed, 12 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Tue, 11 Oct 2016, Junio C Hamano wrote:
> >
> >> The only reason why the OPT_STRDUP appeared convenient was because
> >> options[] element happened to use a field in the structure directly.
> >> The patch under discussion does an equivalent of
> >> 
> >>     app.x_field = xstrdup_or_null(opt_x);
> >
> > Oh, that xstrdup_or_null() function slipped by me. My local patches use it
> > now.
> 
> It has slipped many people ;-)

Thanks, I feel better now ;-)

Ciao,
Dscho

Re: [PATCH v3 25/25] sequencer: mark all error messages for translation

[Johannes Schindelin]

Hi,

On Wed, 12 Oct 2016, Junio C Hamano wrote:

> Johannes Sixt <j6t@kdbg.org> writes:
> 
> > Can we please have the following change instead? I think it makes sense
> > to deviate from the usual conventions in a case like this.
> 
> You have at least two independent changes relative to Dscho's
> version.  
> 
>  (1) Show line breaks more prominently by avoiding "\n\n" and
>      breaking the string at "\n"; this matches the way how the
>      result would be displayed more closely to how the source looks
>      like.
> 
>  (2) Ignore the usual indentation rule and have messages start at
>      the left end of the source.
> 
> Which one are you saying "makes sense" to?  Both?
> 
> I guess both can be grouped together into one theme: match the way
> the final output and the source code look like.
> 
> If that is the motivation behind "makes sense", I'd prefer to see
> the change explained explicitly with that rationale in the log
> message.
> 
> Thanks.  I personally agree with that motivation (if the one I
> guessed above is your motivation, that is).

I agree with that motivation, but I decided to go about it in a way that
is more in line with the existing source code:

-- snipsnap --
diff --git a/sequencer.c b/sequencer.c
index 8e10bb5..1cf70f7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -483,6 +483,20 @@ static char **read_author_script(void)
 	return env;
 }
 
+static const char staged_changes_advice[] =
+N_("you have staged changes in your working tree\n"
+"If these changes are meant to be squashed into the previous commit, run:\n"
+"\n"
+"  git commit --amend %s\n"
+"\n"
+"If they are meant to go into a new commit, run:\n"
+"\n"
+"  git commit %s\n"
+"\n"
+"In both cases, once you're done, continue with:\n"
+"\n"
+"  git rebase --continue\n");
+
 /*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
@@ -509,18 +523,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error(_("you have staged changes in your "
-				       "working tree. If these changes are "
-				       "meant to be\n"
-				       "squashed into the previous commit, "
-				       "run:\n\n"
-				       "  git commit --amend %s\n\n"
-				       "If they are meant to go into a new "
-				       "commit, run:\n\n"
-				       "  git commit %s\n\n"
-				       "In both cases, once you're done, "
-				       "continue with:\n\n"
-				       "  git rebase --continue\n"),
+			return error(_(staged_changes_advice),
 				     gpg_opt, gpg_opt);
 		}
 	}

Re: [PATCH v3 25/25] sequencer: mark all error messages for translation

[Johannes Sixt]

Am 13.10.2016 um 16:56 schrieb Johannes Schindelin:
> On Wed, 12 Oct 2016, Junio C Hamano wrote:
>> You have at least two independent changes relative to Dscho's
>> version.
>>
>>  (1) Show line breaks more prominently by avoiding "\n\n" and
>>      breaking the string at "\n"; this matches the way how the
>>      result would be displayed more closely to how the source looks
>>      like.
>>
>>  (2) Ignore the usual indentation rule and have messages start at
>>      the left end of the source.
>>
>> Which one are you saying "makes sense" to?  Both?
>>
>> I guess both can be grouped together into one theme: match the way
>> the final output and the source code look like.

Yes, that summarizes it pretty well.

> I agree with that motivation, but I decided to go about it in a way that
> is more in line with the existing source code:
>
> -- snipsnap --
> diff --git a/sequencer.c b/sequencer.c
> index 8e10bb5..1cf70f7 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -483,6 +483,20 @@ static char **read_author_script(void)
>  	return env;
>  }
>
> +static const char staged_changes_advice[] =
> +N_("you have staged changes in your working tree\n"
> +"If these changes are meant to be squashed into the previous commit, run:\n"
> +"\n"
> +"  git commit --amend %s\n"
> +"\n"
> +"If they are meant to go into a new commit, run:\n"
> +"\n"
> +"  git commit %s\n"
> +"\n"
> +"In both cases, once you're done, continue with:\n"
> +"\n"
> +"  git rebase --continue\n");

Much better! Thank you.

-- Hannes

[PATCH v4 00/25] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

This patch series marks the '4' in the countdown to speed up rebase -i
by implementing large parts in C (read: there will be three more patch
series after that before the full benefit hits git.git: sequencer-i,
rebase--helper and rebase-i-extra). It is based on the `libify-sequencer`
patch series I submitted earlier.

The patches in this series merely prepare the sequencer code for the
next patch series that actually teaches the sequencer to run rebase -i's
commands.

The reason to split these two patch series is simple: to keep them at a
sensible size.

The two patch series after that are much smaller: a two-patch "series"
that switches rebase -i to use the sequencer (except with --root or
--preserve-merges), and a couple of patches to move several pretty
expensive script processing steps to C (think: autosquash).

The end game of this patch series is a git-rebase--helper that makes
rebase -i 5x faster on Windows (according to t/perf/p3404). Travis says
that even MacOSX and Linux benefit (4x and 3x, respectively).

I have been working on this since early February, whenever time allowed,
and it is time to put it into the users' hands. To that end, I already
integrated the whole shebang into Git for Windows 2.10.0 and 2.10.1
where it has been running without complaints (and some quite positive
feedback).

Changes vs v3:

- fixed TRANSLATORS: comment to help the tool extracting those comments.

- reordered the patch introducing the short_commit_name() function so it
  can be used in the patch revamping the todo parsing right away, as
  opposed to fixing up the find_unique_abbrev() ugliness in a later
  patch.

- backed out the write_message_gently() function of this patch series:
  it is not used by the end of this patch series and would therefore let
  the build fail with DEVELOPER=1. That function is now introduced as
  part of the patch in the sequencer-i series that adds support for the
  'edit' command.

- edited "skip CR/skip LF" to say "strip" instead.

- used xstrdup_or_null() where appropriate.

- abstracted out git_config_string_dup() instead of adding
  near-duplicate code.

- marked the append_new_todo() function as file-local, pointed out by
  Ramsay.

- made the "you have staged changes" advice prettier by moving it out of
  the run_git_commit() function, based on a suggestion by Hannes Sixt.


Johannes Schindelin (25):
  sequencer: use static initializers for replay_opts
  sequencer: use memoized sequencer directory path
  sequencer: avoid unnecessary indirection
  sequencer: future-proof remove_sequencer_state()
  sequencer: eventually release memory allocated for the option values
  sequencer: future-proof read_populate_todo()
  sequencer: refactor the code to obtain a short commit name
  sequencer: completely revamp the "todo" script parsing
  sequencer: strip CR from the todo script
  sequencer: avoid completely different messages for different actions
  sequencer: get rid of the subcommand field
  sequencer: remember the onelines when parsing the todo file
  sequencer: prepare for rebase -i's commit functionality
  sequencer: introduce a helper to read files written by scripts
  sequencer: allow editing the commit message on a case-by-case basis
  sequencer: support amending commits
  sequencer: support cleaning up commit messages
  sequencer: do not try to commit when there were merge conflicts
  sequencer: left-trim lines read from the script
  sequencer: refactor write_message()
  sequencer: remove overzealous assumption in rebase -i mode
  sequencer: mark action_name() for translation
  sequencer: quote filenames in error messages
  sequencer: start error messages consistently with lower case
  sequencer: mark all error messages for translation

 builtin/commit.c              |   2 +-
 builtin/revert.c              |  46 ++-
 sequencer.c                   | 679 ++++++++++++++++++++++++++++--------------
 sequencer.h                   |  23 +-
 t/t3501-revert-cherry-pick.sh |   2 +-
 5 files changed, 492 insertions(+), 260 deletions(-)


base-commit: 3cdd5d19178a54d2e51b5098d43b57571241d0ab
Published-As: https://github.com/dscho/git/releases/tag/prepare-sequencer-v4
Fetch-It-Via: git fetch https://github.com/dscho/git prepare-sequencer-v4

Interdiff vs v3:

 diff --git a/builtin/revert.c b/builtin/revert.c
 index 0a7b5f4..4ca5b51 100644
 --- a/builtin/revert.c
 +++ b/builtin/revert.c
 @@ -166,10 +166,8 @@ static int run_sequencer(int argc, const char **argv, struct replay_opts *opts)
  		usage_with_options(usage_str, options);
  
  	/* These option values will be free()d */
 -	if (opts->gpg_sign)
 -		opts->gpg_sign = xstrdup(opts->gpg_sign);
 -	if (opts->strategy)
 -		opts->strategy = xstrdup(opts->strategy);
 +	opts->gpg_sign = xstrdup_or_null(opts->gpg_sign);
 +	opts->strategy = xstrdup_or_null(opts->strategy);
  
  	if (cmd == 'q')
  		return sequencer_remove_state(opts);
 diff --git a/sequencer.c b/sequencer.c
 index 86d86ce..1cf70f7 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -265,12 +265,6 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
  	return res;
  }
  
 -static int write_file_gently(const char *filename,
 -			     const char *text, int append_eol)
 -{
 -	return write_with_lock_file(filename, text, strlen(text), append_eol);
 -}
 -
  /*
   * Reads a file that was presumably written by a shell script, i.e.
   * with an end-of-line marker that needs to be stripped.
 @@ -489,6 +483,20 @@ static char **read_author_script(void)
  	return env;
  }
  
 +static const char staged_changes_advice[] =
 +N_("you have staged changes in your working tree\n"
 +"If these changes are meant to be squashed into the previous commit, run:\n"
 +"\n"
 +"  git commit --amend %s\n"
 +"\n"
 +"If they are meant to go into a new commit, run:\n"
 +"\n"
 +"  git commit %s\n"
 +"\n"
 +"In both cases, once you're done, continue with:\n"
 +"\n"
 +"  git rebase --continue\n");
 +
  /*
   * If we are cherry-pick, and if the merge did not result in
   * hand-editing, we will hit this commit and inherit the original
 @@ -515,18 +523,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
  		if (!env) {
  			const char *gpg_opt = gpg_sign_opt_quoted(opts);
  
 -			return error(_("you have staged changes in your "
 -				       "working tree. If these changes are "
 -				       "meant to be\n"
 -				       "squashed into the previous commit, "
 -				       "run:\n\n"
 -				       "  git commit --amend %s\n\n"
 -				       "If they are meant to go into a new "
 -				       "commit, run:\n\n"
 -				       "  git commit %s\n\n"
 -				       "In both cases, once you're done, "
 -				       "continue with:\n\n"
 -				       "  git rebase --continue\n"),
 +			return error(_(staged_changes_advice),
  				     gpg_opt, gpg_opt);
  		}
  	}
 @@ -702,10 +699,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
  
  	if (parent && parse_commit(parent) < 0)
 -		/*
 -		 * TRANSLATORS: The first %s will be a "todo" command like
 -		 * "revert" or "pick", the second %s a SHA1.
 -		 */
 +		/* TRANSLATORS: The first %s will be a "todo" command like
 +		   "revert" or "pick", the second %s a SHA1. */
  		return error(_("%s: cannot parse parent commit %s"),
  			command_to_string(command),
  			oid_to_hex(&parent->object.oid));
 @@ -885,7 +880,7 @@ static void todo_list_release(struct todo_list *todo_list)
  	todo_list->nr = todo_list->alloc = 0;
  }
  
 -struct todo_item *append_new_todo(struct todo_list *todo_list)
 +static struct todo_item *append_new_todo(struct todo_list *todo_list)
  {
  	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
  	return todo_list->items + todo_list->nr++;
 @@ -939,10 +934,10 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  	for (i = 1; *p; i++, p = next_p) {
  		char *eol = strchrnul(p, '\n');
  
 -		next_p = *eol ? eol + 1 /* skip LF */ : eol;
 +		next_p = *eol ? eol + 1 /* strip LF */ : eol;
  
  		if (p != eol && eol[-1] == '\r')
 -			eol--; /* skip Carriage Return */
 +			eol--; /* strip Carriage Return */
  
  		item = append_new_todo(todo_list);
  		item->offset_in_buf = p - todo_list->buf.buf;
 @@ -994,6 +989,16 @@ static int read_populate_todo(struct todo_list *todo_list,
  	return 0;
  }
  
 +static int git_config_string_dup(char **dest,
 +				 const char *var, const char *value)
 +{
 +	if (!value)
 +		return config_error_nonbool(var);
 +	free(*dest);
 +	*dest = xstrdup(value);
 +	return 0;
 +}
 +
  static int populate_opts_cb(const char *key, const char *value, void *data)
  {
  	struct replay_opts *opts = data;
 @@ -1013,22 +1018,10 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
  		opts->allow_ff = git_config_bool_or_int(key, value, &error_flag);
  	else if (!strcmp(key, "options.mainline"))
  		opts->mainline = git_config_int(key, value);
 -	else if (!strcmp(key, "options.strategy")) {
 -		if (!value)
 -			config_error_nonbool(key);
 -		else {
 -			free(opts->strategy);
 -			opts->strategy = xstrdup(value);
 -		}
 -	}
 -	else if (!strcmp(key, "options.gpg-sign")) {
 -		if (!value)
 -			config_error_nonbool(key);
 -		else {
 -			free(opts->gpg_sign);
 -			opts->gpg_sign = xstrdup(value);
 -		}
 -	}
 +	else if (!strcmp(key, "options.strategy"))
 +		git_config_string_dup(&opts->strategy, key, value);
 +	else if (!strcmp(key, "options.gpg-sign"))
 +		git_config_string_dup(&opts->gpg_sign, key, value);
  	else if (!strcmp(key, "options.strategy-option")) {
  		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
  		opts->xopts[opts->xopts_nr++] = xstrdup(value);

-- 
2.10.1.513.g00ef6dd

[PATCH v4 01/25] sequencer: use static initializers for replay_opts

[Johannes Schindelin]

This change is not completely faithful: instead of initializing all fields
to 0, we choose to initialize command and subcommand to -1 (instead of
defaulting to REPLAY_REVERT and REPLAY_NONE, respectively). Practically,
it makes no difference at all, but future-proofs the code to require
explicit assignments for both fields.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 6 ++----
 sequencer.h      | 1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 4e69380..7365559 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -178,10 +178,9 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	if (isatty(0))
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
@@ -195,10 +194,9 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 
 int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
 	parse_args(argc, argv, &opts);
diff --git a/sequencer.h b/sequencer.h
index 5ed5cb1..db425ad 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -47,6 +47,7 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
+#define REPLAY_OPTS_INIT { -1, -1 }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 02/25] sequencer: use memoized sequencer directory path

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/commit.c |  2 +-
 sequencer.c      | 11 ++++++-----
 sequencer.h      |  5 +----
 3 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 1cba3b7..9fddb19 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -183,7 +183,7 @@ static void determine_whence(struct wt_status *s)
 		whence = FROM_MERGE;
 	else if (file_exists(git_path_cherry_pick_head())) {
 		whence = FROM_CHERRY_PICK;
-		if (file_exists(git_path(SEQ_DIR)))
+		if (file_exists(git_path_seq_dir()))
 			sequencer_in_use = 1;
 	}
 	else
diff --git a/sequencer.c b/sequencer.c
index eec8a60..cb16cbd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,10 +21,11 @@
 const char sign_off_header[] = "Signed-off-by: ";
 static const char cherry_picked_prefix[] = "(cherry picked from commit ";
 
-static GIT_PATH_FUNC(git_path_todo_file, SEQ_TODO_FILE)
-static GIT_PATH_FUNC(git_path_opts_file, SEQ_OPTS_FILE)
-static GIT_PATH_FUNC(git_path_seq_dir, SEQ_DIR)
-static GIT_PATH_FUNC(git_path_head_file, SEQ_HEAD_FILE)
+GIT_PATH_FUNC(git_path_seq_dir, "sequencer")
+
+static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
+static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
+static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
 static int is_rfc2822_line(const char *buf, int len)
 {
@@ -112,7 +113,7 @@ static void remove_sequencer_state(void)
 {
 	struct strbuf seq_dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path(SEQ_DIR));
+	strbuf_addstr(&seq_dir, git_path_seq_dir());
 	remove_dir_recursively(&seq_dir, 0);
 	strbuf_release(&seq_dir);
 }
diff --git a/sequencer.h b/sequencer.h
index db425ad..dd4d33a 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -1,10 +1,7 @@
 #ifndef SEQUENCER_H
 #define SEQUENCER_H
 
-#define SEQ_DIR		"sequencer"
-#define SEQ_HEAD_FILE	"sequencer/head"
-#define SEQ_TODO_FILE	"sequencer/todo"
-#define SEQ_OPTS_FILE	"sequencer/opts"
+const char *git_path_seq_dir(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 03/25] sequencer: avoid unnecessary indirection

[Johannes Schindelin]

We really do not need the *pointer to a* pointer to the options in
the read_populate_opts() function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index cb16cbd..c2fbf6f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -813,7 +813,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
-static int read_populate_opts(struct replay_opts **opts)
+static int read_populate_opts(struct replay_opts *opts)
 {
 	if (!file_exists(git_path_opts_file()))
 		return 0;
@@ -823,7 +823,7 @@ static int read_populate_opts(struct replay_opts **opts)
 	 * about this case, though, because we wrote that file ourselves, so we
 	 * are pretty certain that it is syntactically correct.
 	 */
-	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
+	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
 		return error(_("Malformed options sheet: %s"),
 			git_path_opts_file());
 	return 0;
@@ -1054,7 +1054,7 @@ static int sequencer_continue(struct replay_opts *opts)
 
 	if (!file_exists(git_path_todo_file()))
 		return continue_single_pick();
-	if (read_populate_opts(&opts) ||
+	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
 		return -1;
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 04/25] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

In a couple of commits, we will teach the sequencer to handle the
nitty gritty of the interactive rebase, which keeps its state in a
different directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index c2fbf6f..8d272fb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,11 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static const char *get_dir(const struct replay_opts *opts)
+{
+	return git_path_seq_dir();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -109,13 +114,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(void)
+static void remove_sequencer_state(const struct replay_opts *opts)
 {
-	struct strbuf seq_dir = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path_seq_dir());
-	remove_dir_recursively(&seq_dir, 0);
-	strbuf_release(&seq_dir);
+	strbuf_addf(&dir, "%s", get_dir(opts));
+	remove_dir_recursively(&dir, 0);
+	strbuf_release(&dir);
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -940,7 +945,7 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	strbuf_release(&buf);
 	return 0;
 fail:
@@ -1034,7 +1039,7 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	return 0;
 }
 
@@ -1095,7 +1100,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	 * one that is being continued
 	 */
 	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state();
+		remove_sequencer_state(opts);
 		return 0;
 	}
 	if (opts->subcommand == REPLAY_ROLLBACK)
-- 
2.10.1.513.g00ef6dd

[PATCH v4 05/25] sequencer: eventually release memory allocated for the option values

[Johannes Schindelin]

The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
like a one-shot command when it reads its configuration: memory is
allocated and released only when the command exits.

This is kind of okay for git-cherry-pick, which *is* a one-shot
command. All the work to make the sequencer its work horse was
done to allow using the functionality as a library function, though,
including proper clean-up after use.

To remedy that, we now take custody of the option values in question,
requiring those values to be malloc()ed or strdup()ed (an alternative
approach, to add a list of pointers to malloc()ed data and to ask the
sequencer to release all of them in the end, was proposed earlier but
rejected during review).

Sadly, the current approach makes the code uglier, as we now have to
take care to strdup() the values passed via the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c |  4 ++++
 sequencer.c      | 26 ++++++++++++++++++++++----
 sequencer.h      |  6 +++---
 3 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 7365559..ba5a88c 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -174,6 +174,10 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 	if (argc > 1)
 		usage_with_options(usage_str, options);
+
+	/* These option values will be free()d */
+	opts->gpg_sign = xstrdup_or_null(opts->gpg_sign);
+	opts->strategy = xstrdup_or_null(opts->strategy);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
diff --git a/sequencer.c b/sequencer.c
index 8d272fb..04c55f2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -117,6 +117,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 static void remove_sequencer_state(const struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
+	int i;
+
+	free(opts->gpg_sign);
+	free(opts->strategy);
+	for (i = 0; i < opts->xopts_nr; i++)
+		free(opts->xopts[i]);
+	free(opts->xopts);
 
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
@@ -280,7 +287,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	struct merge_options o;
 	struct tree *result, *next_tree, *base_tree, *head_tree;
 	int clean;
-	const char **xopt;
+	char **xopt;
 	static struct lock_file index_lock;
 
 	hold_locked_index(&index_lock, 1);
@@ -583,7 +590,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 
 		commit_list_insert(base, &common);
 		commit_list_insert(next, &remotes);
-		res |= try_merge_command(opts->strategy, opts->xopts_nr, opts->xopts,
+		res |= try_merge_command(opts->strategy,
+					 opts->xopts_nr, (const char **)opts->xopts,
 					common, sha1_to_hex(head), remotes);
 		free_commit_list(common);
 		free_commit_list(remotes);
@@ -783,6 +791,16 @@ static int read_populate_todo(struct commit_list **todo_list,
 	return 0;
 }
 
+static int git_config_string_dup(char **dest,
+				 const char *var, const char *value)
+{
+	if (!value)
+		return config_error_nonbool(var);
+	free(*dest);
+	*dest = xstrdup(value);
+	return 0;
+}
+
 static int populate_opts_cb(const char *key, const char *value, void *data)
 {
 	struct replay_opts *opts = data;
@@ -803,9 +821,9 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	else if (!strcmp(key, "options.mainline"))
 		opts->mainline = git_config_int(key, value);
 	else if (!strcmp(key, "options.strategy"))
-		git_config_string(&opts->strategy, key, value);
+		git_config_string_dup(&opts->strategy, key, value);
 	else if (!strcmp(key, "options.gpg-sign"))
-		git_config_string(&opts->gpg_sign, key, value);
+		git_config_string_dup(&opts->gpg_sign, key, value);
 	else if (!strcmp(key, "options.strategy-option")) {
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
 		opts->xopts[opts->xopts_nr++] = xstrdup(value);
diff --git a/sequencer.h b/sequencer.h
index dd4d33a..8453669 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -34,11 +34,11 @@ struct replay_opts {
 
 	int mainline;
 
-	const char *gpg_sign;
+	char *gpg_sign;
 
 	/* Merge strategy */
-	const char *strategy;
-	const char **xopts;
+	char *strategy;
+	char **xopts;
 	size_t xopts_nr, xopts_alloc;
 
 	/* Only used by REPLAY_NONE */
-- 
2.10.1.513.g00ef6dd

[PATCH v4 06/25] sequencer: future-proof read_populate_todo()

[Johannes Schindelin]

Over the next commits, we will work on improving the sequencer to the
point where it can process the todo script of an interactive rebase. To
that end, we will need to teach the sequencer to read interactive
rebase's todo file. In preparation, we consolidate all places where
that todo file is needed to call a function that we will later extend.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 04c55f2..fb0b94b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -32,6 +32,11 @@ static const char *get_dir(const struct replay_opts *opts)
 	return git_path_seq_dir();
 }
 
+static const char *get_todo_path(const struct replay_opts *opts)
+{
+	return git_path_todo_file();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -769,25 +774,24 @@ static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
 static int read_populate_todo(struct commit_list **todo_list,
 			struct replay_opts *opts)
 {
+	const char *todo_file = get_todo_path(opts);
 	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
-	fd = open(git_path_todo_file(), O_RDONLY);
+	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"),
-				   git_path_todo_file());
+		return error_errno(_("Could not open %s"), todo_file);
 	if (strbuf_read(&buf, fd, 0) < 0) {
 		close(fd);
 		strbuf_release(&buf);
-		return error(_("Could not read %s."), git_path_todo_file());
+		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(buf.buf, todo_list, opts);
 	strbuf_release(&buf);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"),
-			git_path_todo_file());
+		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
@@ -1075,7 +1079,7 @@ static int sequencer_continue(struct replay_opts *opts)
 {
 	struct commit_list *todo_list = NULL;
 
-	if (!file_exists(git_path_todo_file()))
+	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
-- 
2.10.1.513.g00ef6dd

[PATCH v4 07/25] sequencer: refactor the code to obtain a short commit name

[Johannes Schindelin]

Not only does this DRY up the code (providing a better documentation what
the code is about, as well as allowing to change the behavior in a single
place), it also makes it substantially shorter to use the same
functionality in functions to be introduced when we teach the sequencer to
process interactive-rebase's git-rebase-todo file.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index fb0b94b..499f5ee 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -147,13 +147,18 @@ struct commit_message {
 	const char *message;
 };
 
+static const char *short_commit_name(struct commit *commit)
+{
+	return find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+}
+
 static int get_message(struct commit *commit, struct commit_message *out)
 {
 	const char *abbrev, *subject;
 	int subject_len;
 
 	out->message = logmsg_reencode(commit, NULL, get_commit_output_encoding());
-	abbrev = find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+	abbrev = short_commit_name(commit);
 
 	subject_len = find_commit_subject(out->message, &subject);
 
@@ -621,8 +626,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		error(opts->action == REPLAY_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
-		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
-		      msg.subject);
+		      short_commit_name(commit), msg.subject);
 		print_advice(res == 1, opts);
 		rerere(opts->allow_rerere_auto);
 		goto leave;
-- 
2.10.1.513.g00ef6dd

[PATCH v4 08/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

When we came up with the "sequencer" idea, we really wanted to have
kind of a plumbing equivalent of the interactive rebase. Hence the
choice of words: the "todo" script, a "pick", etc.

However, when it came time to implement the entire shebang, somehow this
idea got lost and the sequencer was used as working horse for
cherry-pick and revert instead. So as not to interfere with the
interactive rebase, it even uses a separate directory to store its
state.

Furthermore, it also is stupidly strict about the "todo" script it
accepts: while it parses commands in a way that was *designed* to be
similar to the interactive rebase, it then goes on to *error out* if the
commands disagree with the overall action (cherry-pick or revert).

Finally, the sequencer code chose to deviate from the interactive rebase
code insofar that when it comes to writing the file with the remaining
commands, it *reformats* the "todo" script instead of just writing the
part of the parsed script that were not yet processed. This is not only
unnecessary churn, but might well lose information that is valuable to
the user (i.e. comments after the commands).

Let's just bite the bullet and rewrite the entire parser; the code now
becomes not only more elegant: it allows us to go on and teach the
sequencer how to parse *true* "todo" scripts as used by the interactive
rebase itself. In a way, the sequencer is about to grow up to do its
older brother's job. Better.

In particular, we choose to maintain the list of commands in an array
instead of a linked list: this is flexible enough to allow us later on to
even implement rebase -i's reordering of fixup!/squash! commits very
easily (and with a very nice speed bonus, at least on Windows).

While at it, do not stop at the first problem, but list *all* of the
problems. This will help the user when the sequencer will do `rebase
-i`'s work by allowing to address all issues in one go rather than going
back and forth until the todo list is valid.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 284 ++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 163 insertions(+), 121 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 499f5ee..f797e8a 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -470,7 +470,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
+enum todo_command {
+	TODO_PICK = 0,
+	TODO_REVERT
+};
+
+static const char *todo_command_strings[] = {
+	"pick",
+	"revert"
+};
+
+static const char *command_to_string(const enum todo_command command)
+{
+	if (command < ARRAY_SIZE(todo_command_strings))
+		return todo_command_strings[command];
+	die("Unknown command: %d", command);
+}
+
+
+static int do_pick_commit(enum todo_command command, struct commit *commit,
+		struct replay_opts *opts)
 {
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
@@ -529,10 +548,11 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
 
 	if (parent && parse_commit(parent) < 0)
-		/* TRANSLATORS: The first %s will be "revert" or
-		   "cherry-pick", the second %s a SHA1 */
+		/* TRANSLATORS: The first %s will be a "todo" command like
+		   "revert" or "pick", the second %s a SHA1. */
 		return error(_("%s: cannot parse parent commit %s"),
-			action_name(opts), oid_to_hex(&parent->object.oid));
+			command_to_string(command),
+			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
 		return error(_("Cannot get commit message for %s"),
@@ -545,7 +565,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * reverse of it if we are revert.
 	 */
 
-	if (opts->action == REPLAY_REVERT) {
+	if (command == TODO_REVERT) {
 		base = commit;
 		base_label = msg.label;
 		next = parent;
@@ -586,7 +606,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
+	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
@@ -613,17 +633,17 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * However, if the merge did not even start, then we don't want to
 	 * write it at all.
 	 */
-	if (opts->action == REPLAY_PICK && !opts->no_commit && (res == 0 || res == 1) &&
+	if (command == TODO_PICK && !opts->no_commit && (res == 0 || res == 1) &&
 	    update_ref(NULL, "CHERRY_PICK_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
-	if (opts->action == REPLAY_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
+	if (command == TODO_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
 	    update_ref(NULL, "REVERT_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
 
 	if (res) {
-		error(opts->action == REPLAY_REVERT
+		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
 		      short_commit_name(commit), msg.subject);
@@ -684,116 +704,122 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
-static int format_todo(struct strbuf *buf, struct commit_list *todo_list,
-		struct replay_opts *opts)
+struct todo_item {
+	enum todo_command command;
+	struct commit *commit;
+	size_t offset_in_buf;
+};
+
+struct todo_list {
+	struct strbuf buf;
+	struct todo_item *items;
+	int nr, alloc, current;
+};
+
+#define TODO_LIST_INIT { STRBUF_INIT }
+
+static void todo_list_release(struct todo_list *todo_list)
 {
-	struct commit_list *cur = NULL;
-	const char *sha1_abbrev = NULL;
-	const char *action_str = opts->action == REPLAY_REVERT ? "revert" : "pick";
-	const char *subject;
-	int subject_len;
+	strbuf_release(&todo_list->buf);
+	free(todo_list->items);
+	todo_list->items = NULL;
+	todo_list->nr = todo_list->alloc = 0;
+}
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		const char *commit_buffer = get_commit_buffer(cur->item, NULL);
-		sha1_abbrev = find_unique_abbrev(cur->item->object.oid.hash, DEFAULT_ABBREV);
-		subject_len = find_commit_subject(commit_buffer, &subject);
-		strbuf_addf(buf, "%s %s %.*s\n", action_str, sha1_abbrev,
-			subject_len, subject);
-		unuse_commit_buffer(cur->item, commit_buffer);
-	}
-	return 0;
+static struct todo_item *append_new_todo(struct todo_list *todo_list)
+{
+	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
+	return todo_list->items + todo_list->nr++;
 }
 
-static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
+static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 {
 	unsigned char commit_sha1[20];
-	enum replay_action action;
 	char *end_of_object_name;
-	int saved, status, padding;
-
-	if (starts_with(bol, "pick")) {
-		action = REPLAY_PICK;
-		bol += strlen("pick");
-	} else if (starts_with(bol, "revert")) {
-		action = REPLAY_REVERT;
-		bol += strlen("revert");
-	} else
-		return NULL;
+	int i, saved, status, padding;
+
+	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
+		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_strings))
+		return -1;
 
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
-		return NULL;
+		return -1;
 	bol += padding;
 
-	end_of_object_name = bol + strcspn(bol, " \t\n");
+	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
-	/*
-	 * Verify that the action matches up with the one in
-	 * opts; we don't support arbitrary instructions
-	 */
-	if (action != opts->action) {
-		if (action == REPLAY_REVERT)
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot revert during another revert.")
-			    : _("Cannot revert during a cherry-pick."));
-		else
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot cherry-pick during a revert.")
-			    : _("Cannot cherry-pick during another cherry-pick."));
-		return NULL;
-	}
-
 	if (status < 0)
-		return NULL;
+		return -1;
 
-	return lookup_commit_reference(commit_sha1);
+	item->commit = lookup_commit_reference(commit_sha1);
+	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
-			struct replay_opts *opts)
+static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
-	struct commit_list **next = todo_list;
-	struct commit *commit;
-	char *p = buf;
-	int i;
+	struct todo_item *item;
+	char *p = buf, *next_p;
+	int i, res = 0;
 
-	for (i = 1; *p; i++) {
+	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
-		commit = parse_insn_line(p, eol, opts);
-		if (!commit)
-			return error(_("Could not parse line %d."), i);
-		next = commit_list_append(commit, next);
-		p = *eol ? eol + 1 : eol;
+
+		next_p = *eol ? eol + 1 /* strip LF */ : eol;
+
+		item = append_new_todo(todo_list);
+		item->offset_in_buf = p - todo_list->buf.buf;
+		if (parse_insn_line(item, p, eol)) {
+			res = error(_("Invalid line %d: %.*s"),
+				i, (int)(eol - p), p);
+			item->command = -1;
+		}
 	}
-	if (!*todo_list)
+	if (!todo_list->nr)
 		return error(_("No commits parsed."));
-	return 0;
+	return res;
 }
 
-static int read_populate_todo(struct commit_list **todo_list,
+static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
 	const char *todo_file = get_todo_path(opts);
-	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
+	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
 		return error_errno(_("Could not open %s"), todo_file);
-	if (strbuf_read(&buf, fd, 0) < 0) {
+	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		strbuf_release(&buf);
 		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
-	res = parse_insn_buffer(buf.buf, todo_list, opts);
-	strbuf_release(&buf);
+	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
+	if (!res) {
+		enum todo_command valid =
+			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
+		int i;
+
+		for (i = 0; i < todo_list->nr; i++)
+			if (valid == todo_list->items[i].command)
+				continue;
+			else if (valid == TODO_PICK)
+				return error(_("Cannot cherry-pick during a revert."));
+			else
+				return error(_("Cannot revert during a cherry-pick."));
+	}
+
 	if (res)
 		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
@@ -860,18 +886,31 @@ static int read_populate_opts(struct replay_opts *opts)
 	return 0;
 }
 
-static int walk_revs_populate_todo(struct commit_list **todo_list,
+static int walk_revs_populate_todo(struct todo_list *todo_list,
 				struct replay_opts *opts)
 {
+	enum todo_command command = opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT;
+	const char *command_string = todo_command_strings[command];
 	struct commit *commit;
-	struct commit_list **next;
 
 	if (prepare_revs(opts))
 		return -1;
 
-	next = todo_list;
-	while ((commit = get_revision(opts->revs)))
-		next = commit_list_append(commit, next);
+	while ((commit = get_revision(opts->revs))) {
+		struct todo_item *item = append_new_todo(todo_list);
+		const char *commit_buffer = get_commit_buffer(commit, NULL);
+		const char *subject;
+		int subject_len;
+
+		item->command = command;
+		item->commit = commit;
+		item->offset_in_buf = todo_list->buf.len;
+		subject_len = find_commit_subject(commit_buffer, &subject);
+		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
+			short_commit_name(commit), subject_len, subject);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
 	return 0;
 }
 
@@ -979,30 +1018,22 @@ static int sequencer_rollback(struct replay_opts *opts)
 	return -1;
 }
 
-static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
+static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	static struct lock_file todo_lock;
-	struct strbuf buf = STRBUF_INIT;
-	int fd;
+	const char *todo_path = get_todo_path(opts);
+	int next = todo_list->current, offset, fd;
 
-	fd = hold_lock_file_for_update(&todo_lock, git_path_todo_file(), 0);
+	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
-		return error_errno(_("Could not lock '%s'"),
-				   git_path_todo_file());
-	if (format_todo(&buf, todo_list, opts) < 0) {
-		strbuf_release(&buf);
-		return error(_("Could not format %s."), git_path_todo_file());
-	}
-	if (write_in_full(fd, buf.buf, buf.len) < 0) {
-		strbuf_release(&buf);
-		return error_errno(_("Could not write to %s"),
-				   git_path_todo_file());
-	}
-	if (commit_lock_file(&todo_lock) < 0) {
-		strbuf_release(&buf);
-		return error(_("Error wrapping up %s."), git_path_todo_file());
-	}
-	strbuf_release(&buf);
+		return error_errno(_("Could not lock '%s'"), todo_path);
+	offset = next < todo_list->nr ?
+		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	if (write_in_full(fd, todo_list->buf.buf + offset,
+			todo_list->buf.len - offset) < 0)
+		return error_errno(_("Could not write to '%s'"), todo_path);
+	if (commit_lock_file(&todo_lock) < 0)
+		return error(_("Error wrapping up %s."), todo_path);
 	return 0;
 }
 
@@ -1041,9 +1072,8 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
-static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
+static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	struct commit_list *cur;
 	int res;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
@@ -1053,10 +1083,12 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		if (save_todo(cur, opts))
+	while (todo_list->current < todo_list->nr) {
+		struct todo_item *item = todo_list->items + todo_list->current;
+		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(cur->item, opts);
+		res = do_pick_commit(item->command, item->commit, opts);
+		todo_list->current++;
 		if (res)
 			return res;
 	}
@@ -1081,38 +1113,46 @@ static int continue_single_pick(void)
 
 static int sequencer_continue(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
 
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
-	if (read_populate_opts(opts) ||
-			read_populate_todo(&todo_list, opts))
+	if (read_populate_opts(opts))
 		return -1;
+	if ((res = read_populate_todo(&todo_list, opts)))
+		goto release_todo_list;
 
 	/* Verify that the conflict has been resolved */
 	if (file_exists(git_path_cherry_pick_head()) ||
 	    file_exists(git_path_revert_head())) {
-		int ret = continue_single_pick();
-		if (ret)
-			return ret;
+		res = continue_single_pick();
+		if (res)
+			goto release_todo_list;
 	}
-	if (index_differs_from("HEAD", 0))
-		return error_dirty_index(opts);
-	todo_list = todo_list->next;
-	return pick_commits(todo_list, opts);
+	if (index_differs_from("HEAD", 0)) {
+		res = error_dirty_index(opts);
+		goto release_todo_list;
+	}
+	todo_list.current++;
+	res = pick_commits(&todo_list, opts);
+release_todo_list:
+	todo_list_release(&todo_list);
+	return res;
 }
 
 static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
-	return do_pick_commit(cmit, opts);
+	return do_pick_commit(opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT, cmit, opts);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
 	unsigned char sha1[20];
-	int i;
+	int i, res;
 
 	if (opts->subcommand == REPLAY_NONE)
 		assert(opts->revs);
@@ -1187,7 +1227,9 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 		return -1;
 	if (save_opts(opts))
 		return -1;
-	return pick_commits(todo_list, opts);
+	res = pick_commits(&todo_list, opts);
+	todo_list_release(&todo_list);
+	return res;
 }
 
 void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
-- 
2.10.1.513.g00ef6dd

[PATCH v4 09/25] sequencer: strip CR from the todo script

[Johannes Schindelin]

It is not unheard of that editors on Windows write CR/LF even if the
file originally had only LF. This is particularly awkward for exec lines
of a rebase -i todo sheet. Take for example the insn "exec echo": The
shell script parser splits at the LF and leaves the CR attached to
"echo", which leads to the unknown command "echo\r".

Work around that by stripping CR when reading the todo commands, as we
already do for LF.

This happens to fix t9903.14 and .15 in MSYS1 environments (with the
rebase--helper patches based on this patch series): the todo script
constructed in such a setup contains CR/LF thanks to MSYS1 runtime's
cleverness.

Based on a report and a patch by Johannes Sixt.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index f797e8a..eac531b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -776,6 +776,9 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 
 		next_p = *eol ? eol + 1 /* strip LF */ : eol;
 
+		if (p != eol && eol[-1] == '\r')
+			eol--; /* strip Carriage Return */
+
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
 		if (parse_insn_line(item, p, eol)) {
-- 
2.10.1.513.g00ef6dd

[PATCH v4 10/25] sequencer: avoid completely different messages for different actions

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index eac531b..9bca056 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -229,11 +229,8 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(action_name(opts));
 
-	/* Different translation strings for cherry-pick and revert */
-	if (opts->action == REPLAY_PICK)
-		error(_("Your local changes would be overwritten by cherry-pick."));
-	else
-		error(_("Your local changes would be overwritten by revert."));
+	error(_("Your local changes would be overwritten by %s."),
+		action_name(opts));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
-- 
2.10.1.513.g00ef6dd

[PATCH v4 11/25] sequencer: get rid of the subcommand field

[Johannes Schindelin]

The subcommands are used exactly once, at the very beginning of
sequencer_pick_revisions(), to determine what to do. This is an
unnecessary level of indirection: we can simply call the correct
function to begin with. So let's do that.

While at it, ensure that the subcommands return an error code so that
they do not have to die() all over the place (bad practice for library
functions...).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 36 ++++++++++++++++--------------------
 sequencer.c      | 35 +++++++++++------------------------
 sequencer.h      | 13 ++++---------
 3 files changed, 31 insertions(+), 53 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index ba5a88c..4ca5b51 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -71,7 +71,7 @@ static void verify_opt_compatible(const char *me, const char *base_opt, ...)
 		die(_("%s: %s cannot be used with %s"), me, this_opt, base_opt);
 }
 
-static void parse_args(int argc, const char **argv, struct replay_opts *opts)
+static int run_sequencer(int argc, const char **argv, struct replay_opts *opts)
 {
 	const char * const * usage_str = revert_or_cherry_pick_usage(opts);
 	const char *me = action_name(opts);
@@ -115,25 +115,15 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 	if (opts->keep_redundant_commits)
 		opts->allow_empty = 1;
 
-	/* Set the subcommand */
-	if (cmd == 'q')
-		opts->subcommand = REPLAY_REMOVE_STATE;
-	else if (cmd == 'c')
-		opts->subcommand = REPLAY_CONTINUE;
-	else if (cmd == 'a')
-		opts->subcommand = REPLAY_ROLLBACK;
-	else
-		opts->subcommand = REPLAY_NONE;
-
 	/* Check for incompatible command line arguments */
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		char *this_operation;
-		if (opts->subcommand == REPLAY_REMOVE_STATE)
+		if (cmd == 'q')
 			this_operation = "--quit";
-		else if (opts->subcommand == REPLAY_CONTINUE)
+		else if (cmd == 'c')
 			this_operation = "--continue";
 		else {
-			assert(opts->subcommand == REPLAY_ROLLBACK);
+			assert(cmd == 'a');
 			this_operation = "--abort";
 		}
 
@@ -156,7 +146,7 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 				"--edit", opts->edit,
 				NULL);
 
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		opts->revs = NULL;
 	} else {
 		struct setup_revision_opt s_r_opt;
@@ -178,6 +168,14 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 	/* These option values will be free()d */
 	opts->gpg_sign = xstrdup_or_null(opts->gpg_sign);
 	opts->strategy = xstrdup_or_null(opts->strategy);
+
+	if (cmd == 'q')
+		return sequencer_remove_state(opts);
+	if (cmd == 'c')
+		return sequencer_continue(opts);
+	if (cmd == 'a')
+		return sequencer_rollback(opts);
+	return sequencer_pick_revisions(opts);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
@@ -189,8 +187,7 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("revert failed"));
 	return res;
@@ -203,8 +200,7 @@ int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("cherry-pick failed"));
 	return res;
diff --git a/sequencer.c b/sequencer.c
index 9bca056..5e4bf23 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -119,7 +119,7 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(const struct replay_opts *opts)
+int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
 	int i;
@@ -133,6 +133,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
 	strbuf_release(&dir);
+
+	return 0;
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -975,7 +977,7 @@ static int rollback_single_pick(void)
 	return reset_for_rollback(head_sha1);
 }
 
-static int sequencer_rollback(struct replay_opts *opts)
+int sequencer_rollback(struct replay_opts *opts)
 {
 	FILE *f;
 	unsigned char sha1[20];
@@ -1010,9 +1012,8 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state(opts);
 	strbuf_release(&buf);
-	return 0;
+	return sequencer_remove_state(opts);
 fail:
 	strbuf_release(&buf);
 	return -1;
@@ -1097,8 +1098,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state(opts);
-	return 0;
+	return sequencer_remove_state(opts);
 }
 
 static int continue_single_pick(void)
@@ -1111,11 +1111,14 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
-static int sequencer_continue(struct replay_opts *opts)
+int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
 	int res;
 
+	if (read_and_refresh_cache(opts))
+		return -1;
+
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
@@ -1154,26 +1157,10 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	unsigned char sha1[20];
 	int i, res;
 
-	if (opts->subcommand == REPLAY_NONE)
-		assert(opts->revs);
-
+	assert(opts->revs);
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	/*
-	 * Decide what to do depending on the arguments; a fresh
-	 * cherry-pick should be handled differently from an existing
-	 * one that is being continued
-	 */
-	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state(opts);
-		return 0;
-	}
-	if (opts->subcommand == REPLAY_ROLLBACK)
-		return sequencer_rollback(opts);
-	if (opts->subcommand == REPLAY_CONTINUE)
-		return sequencer_continue(opts);
-
 	for (i = 0; i < opts->revs->pending.nr; i++) {
 		unsigned char sha1[20];
 		const char *name = opts->revs->pending.objects[i].name;
diff --git a/sequencer.h b/sequencer.h
index 8453669..7a513c5 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -10,16 +10,8 @@ enum replay_action {
 	REPLAY_PICK
 };
 
-enum replay_subcommand {
-	REPLAY_NONE,
-	REPLAY_REMOVE_STATE,
-	REPLAY_CONTINUE,
-	REPLAY_ROLLBACK
-};
-
 struct replay_opts {
 	enum replay_action action;
-	enum replay_subcommand subcommand;
 
 	/* Boolean options */
 	int edit;
@@ -44,9 +36,12 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
-#define REPLAY_OPTS_INIT { -1, -1 }
+#define REPLAY_OPTS_INIT { -1 }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
+int sequencer_continue(struct replay_opts *opts);
+int sequencer_rollback(struct replay_opts *opts);
+int sequencer_remove_state(struct replay_opts *opts);
 
 extern const char sign_off_header[];
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 12/25] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

The `git-rebase-todo` file contains a list of commands. Most of those
commands have the form

	<verb> <sha1> <oneline>

The <oneline> is displayed primarily for the user's convenience, as
rebase -i really interprets only the <verb> <sha1> part. However, there
are *some* places in interactive rebase where the <oneline> is used to
display messages, e.g. for reporting at which commit we stopped.

So let's just remember it when parsing the todo file; we keep a copy of
the entire todo file anyway (to write out the new `done` and
`git-rebase-todo` file just before processing each command), so all we
need to do is remember the begin offsets and lengths.

As we will have to parse and remember the command-line for `exec` commands
later, we do not call the field "oneline" but rather "arg" (and will reuse
that for exec's command-line).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 5e4bf23..caef51e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -706,6 +706,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	const char *arg;
+	int arg_len;
 	size_t offset_in_buf;
 };
 
@@ -757,6 +759,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
+	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_len = (int)(eol - item->arg);
+
 	if (status < 0)
 		return -1;
 
@@ -907,6 +912,8 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
+		item->arg = NULL;
+		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
-- 
2.10.1.513.g00ef6dd

[PATCH v4 13/25] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

In interactive rebases, we commit a little bit differently than the
sequencer did so far: we heed the "author-script", the "message" and the
"amend" files in the .git/rebase-merge/ subdirectory.

Likewise, we may want to edit the commit message *even* when providing a
file containing the suggested commit message. Therefore we change the
code to not even provide a default message when we do not want any, and
to call the editor explicitly.

Also, in "interactive rebase" mode we want to skip reading the options
in the state directory of the cherry-pick/revert commands.

Finally, as interactive rebase's GPG settings are configured differently
from how cherry-pick (and therefore sequencer) handles them, we will
leave support for that to the next commit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 89 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index caef51e..99c39fb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,19 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+/*
+ * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
+ * GIT_AUTHOR_DATE that will be used for the commit that is currently
+ * being rebased.
+ */
+static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+
+/* We will introduce the 'interactive rebase' mode later */
+static inline int is_rebase_i(const struct replay_opts *opts)
+{
+	return 0;
+}
+
 static const char *get_dir(const struct replay_opts *opts)
 {
 	return git_path_seq_dir();
@@ -370,19 +383,79 @@ static int is_index_unchanged(void)
 }
 
 /*
+ * Read the author-script file into an environment block, ready for use in
+ * run_command(), that can be free()d afterwards.
+ */
+static char **read_author_script(void)
+{
+	struct strbuf script = STRBUF_INIT;
+	int i, count = 0;
+	char *p, *p2, **env;
+	size_t env_size;
+
+	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
+		return NULL;
+
+	for (p = script.buf; *p; p++)
+		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
+			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
+		else if (*p == '\'')
+			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
+		else if (*p == '\n') {
+			*p = '\0';
+			count++;
+		}
+
+	env_size = (count + 1) * sizeof(*env);
+	strbuf_grow(&script, env_size);
+	memmove(script.buf + env_size, script.buf, script.len);
+	p = script.buf + env_size;
+	env = (char **)strbuf_detach(&script, NULL);
+
+	for (i = 0; i < count; i++) {
+		env[i] = p;
+		p += strlen(p) + 1;
+	}
+	env[count] = NULL;
+
+	return env;
+}
+
+/*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
  * author date and name.
+ *
  * If we are revert, or if our cherry-pick results in a hand merge,
  * we had better say that the current user is responsible for that.
+ *
+ * An exception is when run_git_commit() is called during an
+ * interactive rebase: in that case, we will want to retain the
+ * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty)
 {
+	char **env = NULL;
 	struct argv_array array;
 	int rc;
 	const char *value;
 
+	if (is_rebase_i(opts)) {
+		env = read_author_script();
+		if (!env)
+			return error("You have staged changes in your working "
+				"tree. If these changes are meant to be\n"
+				"squashed into the previous commit, run:\n\n"
+				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"If they are meant to go into a new commit, "
+				"run:\n\n"
+				"  git commit $gpg_sign_opt_quoted\n\n"
+				"In both cases, once you're done, continue "
+				"with:\n\n"
+				"  git rebase --continue\n");
+	}
+
 	argv_array_init(&array);
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
@@ -391,14 +464,13 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
 		argv_array_push(&array, "-s");
-	if (!opts->edit) {
-		argv_array_push(&array, "-F");
-		argv_array_push(&array, defmsg);
-		if (!opts->signoff &&
-		    !opts->record_origin &&
-		    git_config_get_value("commit.cleanup", &value))
-			argv_array_push(&array, "--cleanup=verbatim");
-	}
+	if (defmsg)
+		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (opts->edit)
+		argv_array_push(&array, "-e");
+	else if (!opts->signoff && !opts->record_origin &&
+		 git_config_get_value("commit.cleanup", &value))
+		argv_array_push(&array, "--cleanup=verbatim");
 
 	if (allow_empty)
 		argv_array_push(&array, "--allow-empty");
@@ -406,8 +478,11 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt(array.argv, RUN_GIT_CMD);
+	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+			(const char *const *)env);
 	argv_array_clear(&array);
+	free(env);
+
 	return rc;
 }
 
@@ -657,7 +732,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(git_path_merge_msg(), opts, allow);
+		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
+				     opts, allow);
 
 leave:
 	free_message(commit, &msg);
@@ -879,6 +955,9 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return 0;
+
 	if (!file_exists(git_path_opts_file()))
 		return 0;
 	/*
-- 
2.10.1.513.g00ef6dd

[PATCH v4 14/25] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

As we are slowly teaching the sequencer to perform the hard work for
the interactive rebase, we need to read files that were written by
shell scripts.

These files typically contain a single line and are invariably ended
by a line feed (and possibly a carriage return before that). Let's use
a helper to read such files and to remove the line ending.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 99c39fb..de6b95f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -234,6 +234,37 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
 	return 0;
 }
 
+/*
+ * Reads a file that was presumably written by a shell script, i.e.
+ * with an end-of-line marker that needs to be stripped.
+ *
+ * Returns 1 if the file was read, 0 if it could not be read or does not exist.
+ */
+static int read_oneliner(struct strbuf *buf,
+	const char *path, int skip_if_empty)
+{
+	int orig_len = buf->len;
+
+	if (!file_exists(path))
+		return 0;
+
+	if (strbuf_read_file(buf, path, 0) < 0) {
+		warning_errno(_("could not read '%s'"), path);
+		return 0;
+	}
+
+	if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
+		if (--buf->len > orig_len && buf->buf[buf->len - 1] == '\r')
+			--buf->len;
+		buf->buf[buf->len] = '\0';
+	}
+
+	if (skip_if_empty && buf->len == orig_len)
+		return 0;
+
+	return 1;
+}
+
 static struct tree *empty_tree(void)
 {
 	return lookup_tree(EMPTY_TREE_SHA1_BIN);
-- 
2.10.1.513.g00ef6dd

[PATCH v4 15/25] sequencer: allow editing the commit message on a case-by-case basis

[Johannes Schindelin]

In the upcoming commits, we will implement more and more of rebase -i's
functionality inside the sequencer. One particular feature of the
commands to come is that some of them allow editing the commit message
while others don't, i.e. we cannot define in the replay_opts whether the
commit message should be edited or not.

Let's add a new parameter to the run_git_commit() function. Previously,
it was the duty of the caller to ensure that the opts->edit setting
indicates whether to let the user edit the commit message or not,
indicating that it is an "all or nothing" setting, i.e. that the
sequencer wants to let the user edit *all* commit message, or none at
all. In the upcoming rebase -i mode, it will depend on the particular
command that is currently executed, though.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 48 ++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 40 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index de6b95f..2ef80e7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -15,6 +15,7 @@
 #include "merge-recursive.h"
 #include "refs.h"
 #include "argv-array.h"
+#include "quote.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -33,6 +34,11 @@ static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
  * being rebased.
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+/*
+ * The following files are written by git-rebase just after parsing the
+ * command-line (and are only consumed, not modified, by the sequencer).
+ */
+static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -132,6 +138,16 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
+static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
+{
+	static struct strbuf buf = STRBUF_INIT;
+
+	strbuf_reset(&buf);
+	if (opts->gpg_sign)
+		sq_quotef(&buf, "-S%s", opts->gpg_sign);
+	return buf.buf;
+}
+
 int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
@@ -465,7 +481,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty)
+			  int allow_empty, int edit)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -474,17 +490,20 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 
 	if (is_rebase_i(opts)) {
 		env = read_author_script();
-		if (!env)
+		if (!env) {
+			const char *gpg_opt = gpg_sign_opt_quoted(opts);
+
 			return error("You have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"  git commit --amend %s\n\n"
 				"If they are meant to go into a new commit, "
 				"run:\n\n"
-				"  git commit $gpg_sign_opt_quoted\n\n"
+				"  git commit %s\n\n"
 				"In both cases, once you're done, continue "
 				"with:\n\n"
-				"  git rebase --continue\n");
+				"  git rebase --continue\n", gpg_opt, gpg_opt);
+		}
 	}
 
 	argv_array_init(&array);
@@ -497,7 +516,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
-	if (opts->edit)
+	if (edit)
 		argv_array_push(&array, "-e");
 	else if (!opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
@@ -764,7 +783,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow);
+				     opts, allow, opts->edit);
 
 leave:
 	free_message(commit, &msg);
@@ -986,8 +1005,21 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
-	if (is_rebase_i(opts))
+	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
+		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
+			if (!starts_with(buf.buf, "-S"))
+				strbuf_reset(&buf);
+			else {
+				free(opts->gpg_sign);
+				opts->gpg_sign = xstrdup(buf.buf + 2);
+			}
+		}
+		strbuf_release(&buf);
+
 		return 0;
+	}
 
 	if (!file_exists(git_path_opts_file()))
 		return 0;
-- 
2.10.1.513.g00ef6dd

[PATCH v4 16/25] sequencer: support amending commits

[Johannes Schindelin]

This teaches the run_git_commit() function to take an argument that will
allow us to implement "todo" commands that need to amend the commit
messages ("fixup", "squash" and "reword").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 2ef80e7..fa77c82 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -481,7 +481,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit)
+			  int allow_empty, int edit, int amend)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -510,6 +510,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
 
+	if (amend)
+		argv_array_push(&array, "--amend");
 	if (opts->gpg_sign)
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
@@ -783,7 +785,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit);
+				     opts, allow, opts->edit, 0);
 
 leave:
 	free_message(commit, &msg);
-- 
2.10.1.513.g00ef6dd

[PATCH v4 17/25] sequencer: support cleaning up commit messages

[Johannes Schindelin]

The run_git_commit() function already knows how to amend commits, and
with this new option, it can also clean up commit messages (i.e. strip
out commented lines). This is needed to implement rebase -i's 'fixup'
and 'squash' commands as sequencer commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index fa77c82..cbc3742 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -481,7 +481,8 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit, int amend)
+			  int allow_empty, int edit, int amend,
+			  int cleanup_commit_message)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -518,9 +519,12 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (cleanup_commit_message)
+		argv_array_push(&array, "--cleanup=strip");
 	if (edit)
 		argv_array_push(&array, "-e");
-	else if (!opts->signoff && !opts->record_origin &&
+	else if (!cleanup_commit_message &&
+		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
 		argv_array_push(&array, "--cleanup=verbatim");
 
@@ -785,7 +789,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit, 0);
+				     opts, allow, opts->edit, 0, 0);
 
 leave:
 	free_message(commit, &msg);
-- 
2.10.1.513.g00ef6dd

[PATCH v4 18/25] sequencer: do not try to commit when there were merge conflicts

[Johannes Schindelin]

The return value of do_recursive_merge() may be positive (indicating merge
conflicts), or 0 (indicating success). It also may be negative, indicating
a fatal error that requires us to abort.

Now, if the return value indicates that there are merge conflicts, we
should not try to commit those changes, of course.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index cbc3742..9ffc090 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -787,7 +787,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		res = allow;
 		goto leave;
 	}
-	if (!opts->no_commit)
+	if (!res && !opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
 				     opts, allow, opts->edit, 0, 0);
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 19/25] sequencer: left-trim lines read from the script

[Johannes Schindelin]

Interactive rebase's scripts may be indented; we need to handle this
case, too, now that we prepare the sequencer to process interactive
rebases.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 9ffc090..914a576 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -871,6 +871,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	/* left-trim */
+	bol += strspn(bol, " \t");
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
-- 
2.10.1.513.g00ef6dd

[PATCH v4 20/25] sequencer: refactor write_message()

[Johannes Schindelin]

The write_message() function safely writes an strbuf to a file.
Sometimes it is inconvenient to require an strbuf, though: the text to
be written may not be stored in a strbuf, or the strbuf should not be
released after writing.

Let's refactor "safely writing string to a file" into
write_with_lock_file(), and make write_message() use it.

The new function will make it easy to create a new convenience function
write_file_gently() that does not die(); as some of the upcoming callers
of this new function will want to append a newline character, we already
add that flag as parameter to write_with_lock_file().

While at it, roll back the locked files in case of failure, as pointed
out by Hannes Sixt.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 914a576..baccee9 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -234,22 +234,37 @@ static void print_advice(int show_hint, struct replay_opts *opts)
 	}
 }
 
-static int write_message(struct strbuf *msgbuf, const char *filename)
+static int write_with_lock_file(const char *filename,
+				const void *buf, size_t len, int append_eol)
 {
 	static struct lock_file msg_file;
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
 		return error_errno(_("Could not lock '%s'"), filename);
-	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
-		return error_errno(_("Could not write to %s"), filename);
-	strbuf_release(msgbuf);
-	if (commit_lock_file(&msg_file) < 0)
+	if (write_in_full(msg_fd, buf, len) < 0) {
+		rollback_lock_file(&msg_file);
+		return error_errno(_("Could not write to '%s'"), filename);
+	}
+	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		rollback_lock_file(&msg_file);
+		return error_errno(_("Could not write eol to '%s"), filename);
+	}
+	if (commit_lock_file(&msg_file) < 0) {
+		rollback_lock_file(&msg_file);
 		return error(_("Error wrapping up %s."), filename);
+	}
 
 	return 0;
 }
 
+static int write_message(struct strbuf *msgbuf, const char *filename)
+{
+	int res = write_with_lock_file(filename, msgbuf->buf, msgbuf->len, 0);
+	strbuf_release(msgbuf);
+	return res;
+}
+
 /*
  * Reads a file that was presumably written by a shell script, i.e.
  * with an end-of-line marker that needs to be stripped.
-- 
2.10.1.513.g00ef6dd

[PATCH v4 21/25] sequencer: remove overzealous assumption in rebase -i mode

[Johannes Schindelin]

The sequencer was introduced to make the cherry-pick and revert
functionality available as library function, with the original idea
being to extend the sequencer to also implement the rebase -i
functionality.

The test to ensure that all of the commands in the script are identical
to the overall operation does not mesh well with that.

Therefore let's disable the test in rebase -i mode.

While at it, error out early if the "instruction sheet" (i.e. the todo
script) could not be parsed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index baccee9..36c24b6 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -963,7 +963,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (!res) {
+	if (res)
+		return error(_("Unusable instruction sheet: %s"), todo_file);
+
+	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
 		int i;
@@ -977,8 +980,6 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("Cannot revert during a cherry-pick."));
 	}
 
-	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 22/25] sequencer: mark action_name() for translation

[Johannes Schindelin]

The definition of this function goes back all the way to 043a449
(sequencer: factor code out of revert builtin, 2012-01-11), long before a
serious effort was made to translate all the error messages.

It is slightly out of the context of the current patch series (whose
purpose it is to re-implement the performance critical parts of the
interactive rebase in C) to make the error messages in the sequencer
translatable, but what the heck. We'll just do it while we're looking at
this part of the code.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 36c24b6..af88753 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -168,7 +168,7 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? "revert" : "cherry-pick";
+	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
 }
 
 struct commit_message {
@@ -304,10 +304,10 @@ static struct tree *empty_tree(void)
 static int error_dirty_index(struct replay_opts *opts)
 {
 	if (read_cache_unmerged())
-		return error_resolve_conflict(action_name(opts));
+		return error_resolve_conflict(_(action_name(opts)));
 
 	error(_("Your local changes would be overwritten by %s."),
-		action_name(opts));
+		_(action_name(opts)));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
@@ -325,7 +325,7 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	if (checkout_fast_forward(from, to, 1))
 		return -1; /* the callee should have complained already */
 
-	strbuf_addf(&sb, _("%s: fast-forward"), action_name(opts));
+	strbuf_addf(&sb, _("%s: fast-forward"), _(action_name(opts)));
 
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
@@ -401,7 +401,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
 		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
 		return error(_("%s: Unable to write new index file"),
-			action_name(opts));
+			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
 
 	if (opts->signoff)
@@ -836,14 +836,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	if (read_index_preload(&the_index, NULL) < 0) {
 		rollback_lock_file(&index_lock);
 		return error(_("git %s: failed to read the index"),
-			action_name(opts));
+			_(action_name(opts)));
 	}
 	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, NULL, NULL, NULL);
 	if (the_index.cache_changed && index_fd >= 0) {
 		if (write_locked_index(&the_index, &index_lock, COMMIT_LOCK)) {
 			rollback_lock_file(&index_lock);
 			return error(_("git %s: failed to refresh the index"),
-				action_name(opts));
+				_(action_name(opts)));
 		}
 	}
 	rollback_lock_file(&index_lock);
-- 
2.10.1.513.g00ef6dd

[PATCH v4 23/25] sequencer: quote filenames in error messages

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3599 bytes --]

This makes the code consistent by fixing quite a couple of error messages.

Suggested by Jakub Narębski.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index af88753..3e26631 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -252,7 +252,7 @@ static int write_with_lock_file(const char *filename,
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
-		return error(_("Error wrapping up %s."), filename);
+		return error(_("Error wrapping up '%s'."), filename);
 	}
 
 	return 0;
@@ -955,16 +955,16 @@ static int read_populate_todo(struct todo_list *todo_list,
 	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"), todo_file);
+		return error_errno(_("Could not open '%s'"), todo_file);
 	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		return error(_("Could not read %s."), todo_file);
+		return error(_("Could not read '%s'."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
+		return error(_("Unusable instruction sheet: '%s'"), todo_file);
 
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
@@ -1055,7 +1055,7 @@ static int read_populate_opts(struct replay_opts *opts)
 	 * are pretty certain that it is syntactically correct.
 	 */
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
-		return error(_("Malformed options sheet: %s"),
+		return error(_("Malformed options sheet: '%s'"),
 			git_path_opts_file());
 	return 0;
 }
@@ -1098,7 +1098,7 @@ static int create_seq_dir(void)
 		return -1;
 	}
 	else if (mkdir(git_path_seq_dir(), 0777) < 0)
-		return error_errno(_("Could not create sequencer directory %s"),
+		return error_errno(_("Could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
 }
@@ -1117,12 +1117,12 @@ static int save_head(const char *head)
 	strbuf_addf(&buf, "%s\n", head);
 	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not write to %s"),
+		return error_errno(_("Could not write to '%s'"),
 				   git_path_head_file());
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-		return error(_("Error wrapping up %s."), git_path_head_file());
+		return error(_("Error wrapping up '%s'."), git_path_head_file());
 	}
 	return 0;
 }
@@ -1167,9 +1167,9 @@ int sequencer_rollback(struct replay_opts *opts)
 		return rollback_single_pick();
 	}
 	if (!f)
-		return error_errno(_("cannot open %s"), git_path_head_file());
+		return error_errno(_("cannot open '%s'"), git_path_head_file());
 	if (strbuf_getline_lf(&buf, f)) {
-		error(_("cannot read %s: %s"), git_path_head_file(),
+		error(_("cannot read '%s': %s"), git_path_head_file(),
 		      ferror(f) ?  strerror(errno) : _("unexpected end of file"));
 		fclose(f);
 		goto fail;
@@ -1208,7 +1208,7 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 			todo_list->buf.len - offset) < 0)
 		return error_errno(_("Could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
-		return error(_("Error wrapping up %s."), todo_path);
+		return error(_("Error wrapping up '%s'."), todo_path);
 	return 0;
 }
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 24/25] sequencer: start error messages consistently with lower case

[Johannes Schindelin]

Quite a few error messages touched by this developer during the work to
speed up rebase -i started with an upper case letter, violating our
current conventions. Instead of sneaking in this fix (and forgetting
quite a few error messages), let's just have one wholesale patch fixing
all of the error messages in the sequencer.

While at it, the funny "error: Error wrapping up..." was changed to a
less funny, but more helpful, "error: failed to finalize...".

Pointed out by Junio Hamano.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                   | 68 +++++++++++++++++++++----------------------
 t/t3501-revert-cherry-pick.sh |  2 +-
 2 files changed, 35 insertions(+), 35 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3e26631..57c5c0c 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -241,18 +241,18 @@ static int write_with_lock_file(const char *filename,
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
-		return error_errno(_("Could not lock '%s'"), filename);
+		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
 		rollback_lock_file(&msg_file);
-		return error_errno(_("Could not write to '%s'"), filename);
+		return error_errno(_("could not write to '%s'"), filename);
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
 		rollback_lock_file(&msg_file);
-		return error_errno(_("Could not write eol to '%s"), filename);
+		return error_errno(_("could not write eol to '%s"), filename);
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
-		return error(_("Error wrapping up '%s'."), filename);
+		return error(_("failed to finalize '%s'."), filename);
 	}
 
 	return 0;
@@ -306,11 +306,11 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(_(action_name(opts)));
 
-	error(_("Your local changes would be overwritten by %s."),
+	error(_("your local changes would be overwritten by %s."),
 		_(action_name(opts)));
 
 	if (advice_commit_before_merge)
-		advise(_("Commit your changes or stash them to proceed."));
+		advise(_("commit your changes or stash them to proceed."));
 	return -1;
 }
 
@@ -419,7 +419,7 @@ static int is_index_unchanged(void)
 	struct commit *head_commit;
 
 	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
-		return error(_("Could not resolve HEAD commit\n"));
+		return error(_("could not resolve HEAD commit\n"));
 
 	head_commit = lookup_commit(head_sha1);
 
@@ -439,7 +439,7 @@ static int is_index_unchanged(void)
 
 	if (!cache_tree_fully_valid(active_cache_tree))
 		if (cache_tree_update(&the_index, 0))
-			return error(_("Unable to update cache tree\n"));
+			return error(_("unable to update cache tree\n"));
 
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
@@ -509,7 +509,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error("You have staged changes in your working "
+			return error("you have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
 				"  git commit --amend %s\n\n"
@@ -562,12 +562,12 @@ static int is_original_commit_empty(struct commit *commit)
 	const unsigned char *ptree_sha1;
 
 	if (parse_commit(commit))
-		return error(_("Could not parse commit %s\n"),
+		return error(_("could not parse commit %s\n"),
 			     oid_to_hex(&commit->object.oid));
 	if (commit->parents) {
 		struct commit *parent = commit->parents->item;
 		if (parse_commit(parent))
-			return error(_("Could not parse parent commit %s\n"),
+			return error(_("could not parse parent commit %s\n"),
 				oid_to_hex(&parent->object.oid));
 		ptree_sha1 = parent->tree->object.oid.hash;
 	} else {
@@ -651,7 +651,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		 * to work on.
 		 */
 		if (write_cache_as_tree(head, 0, NULL))
-			return error(_("Your index file is unmerged."));
+			return error(_("your index file is unmerged."));
 	} else {
 		unborn = get_sha1("HEAD", head);
 		if (unborn)
@@ -670,7 +670,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct commit_list *p;
 
 		if (!opts->mainline)
-			return error(_("Commit %s is a merge but no -m option was given."),
+			return error(_("commit %s is a merge but no -m option was given."),
 				oid_to_hex(&commit->object.oid));
 
 		for (cnt = 1, p = commit->parents;
@@ -678,11 +678,11 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		     cnt++)
 			p = p->next;
 		if (cnt != opts->mainline || !p)
-			return error(_("Commit %s does not have parent %d"),
+			return error(_("commit %s does not have parent %d"),
 				oid_to_hex(&commit->object.oid), opts->mainline);
 		parent = p->item;
 	} else if (0 < opts->mainline)
-		return error(_("Mainline was specified but commit %s is not a merge."),
+		return error(_("mainline was specified but commit %s is not a merge."),
 			oid_to_hex(&commit->object.oid));
 	else
 		parent = commit->parents->item;
@@ -700,7 +700,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
-		return error(_("Cannot get commit message for %s"),
+		return error(_("cannot get commit message for %s"),
 			oid_to_hex(&commit->object.oid));
 
 	/*
@@ -936,13 +936,13 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
 		if (parse_insn_line(item, p, eol)) {
-			res = error(_("Invalid line %d: %.*s"),
+			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
 			item->command = -1;
 		}
 	}
 	if (!todo_list->nr)
-		return error(_("No commits parsed."));
+		return error(_("no commits parsed."));
 	return res;
 }
 
@@ -955,16 +955,16 @@ static int read_populate_todo(struct todo_list *todo_list,
 	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open '%s'"), todo_file);
+		return error_errno(_("could not open '%s'"), todo_file);
 	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		return error(_("Could not read '%s'."), todo_file);
+		return error(_("could not read '%s'."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
-		return error(_("Unusable instruction sheet: '%s'"), todo_file);
+		return error(_("unusable instruction sheet: '%s'"), todo_file);
 
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
@@ -975,9 +975,9 @@ static int read_populate_todo(struct todo_list *todo_list,
 			if (valid == todo_list->items[i].command)
 				continue;
 			else if (valid == TODO_PICK)
-				return error(_("Cannot cherry-pick during a revert."));
+				return error(_("cannot cherry-pick during a revert."));
 			else
-				return error(_("Cannot revert during a cherry-pick."));
+				return error(_("cannot revert during a cherry-pick."));
 	}
 
 	return 0;
@@ -1020,10 +1020,10 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
 		opts->xopts[opts->xopts_nr++] = xstrdup(value);
 	} else
-		return error(_("Invalid key: %s"), key);
+		return error(_("invalid key: %s"), key);
 
 	if (!error_flag)
-		return error(_("Invalid value for %s: %s"), key, value);
+		return error(_("invalid value for %s: %s"), key, value);
 
 	return 0;
 }
@@ -1055,7 +1055,7 @@ static int read_populate_opts(struct replay_opts *opts)
 	 * are pretty certain that it is syntactically correct.
 	 */
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
-		return error(_("Malformed options sheet: '%s'"),
+		return error(_("malformed options sheet: '%s'"),
 			git_path_opts_file());
 	return 0;
 }
@@ -1098,7 +1098,7 @@ static int create_seq_dir(void)
 		return -1;
 	}
 	else if (mkdir(git_path_seq_dir(), 0777) < 0)
-		return error_errno(_("Could not create sequencer directory '%s'"),
+		return error_errno(_("could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
 }
@@ -1112,17 +1112,17 @@ static int save_head(const char *head)
 	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
 	if (fd < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not lock HEAD"));
+		return error_errno(_("could not lock HEAD"));
 	}
 	strbuf_addf(&buf, "%s\n", head);
 	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not write to '%s'"),
+		return error_errno(_("could not write to '%s'"),
 				   git_path_head_file());
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-		return error(_("Error wrapping up '%s'."), git_path_head_file());
+		return error(_("failed to finalize '%s'."), git_path_head_file());
 	}
 	return 0;
 }
@@ -1201,14 +1201,14 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
-		return error_errno(_("Could not lock '%s'"), todo_path);
+		return error_errno(_("could not lock '%s'"), todo_path);
 	offset = next < todo_list->nr ?
 		todo_list->items[next].offset_in_buf : todo_list->buf.len;
 	if (write_in_full(fd, todo_list->buf.buf + offset,
 			todo_list->buf.len - offset) < 0)
-		return error_errno(_("Could not write to '%s'"), todo_path);
+		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
-		return error(_("Error wrapping up '%s'."), todo_path);
+		return error(_("failed to finalize '%s'."), todo_path);
 	return 0;
 }
 
@@ -1383,7 +1383,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 			create_seq_dir() < 0)
 		return -1;
 	if (get_sha1("HEAD", sha1) && (opts->action == REPLAY_REVERT))
-		return error(_("Can't revert as initial commit"));
+		return error(_("can't revert as initial commit"));
 	if (save_head(sha1_to_hex(sha1)))
 		return -1;
 	if (save_opts(opts))
diff --git a/t/t3501-revert-cherry-pick.sh b/t/t3501-revert-cherry-pick.sh
index 51f3bbb..394f000 100755
--- a/t/t3501-revert-cherry-pick.sh
+++ b/t/t3501-revert-cherry-pick.sh
@@ -96,7 +96,7 @@ test_expect_success 'revert forbidden on dirty working tree' '
 	echo content >extra_file &&
 	git add extra_file &&
 	test_must_fail git revert HEAD 2>errors &&
-	test_i18ngrep "Your local changes would be overwritten by " errors
+	test_i18ngrep "your local changes would be overwritten by " errors
 
 '
 
-- 
2.10.1.513.g00ef6dd

[PATCH v4 25/25] sequencer: mark all error messages for translation

[Johannes Schindelin]

There was actually only one error message that was not yet marked for
translation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 57c5c0c..1cf70f7 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -483,6 +483,20 @@ static char **read_author_script(void)
 	return env;
 }
 
+static const char staged_changes_advice[] =
+N_("you have staged changes in your working tree\n"
+"If these changes are meant to be squashed into the previous commit, run:\n"
+"\n"
+"  git commit --amend %s\n"
+"\n"
+"If they are meant to go into a new commit, run:\n"
+"\n"
+"  git commit %s\n"
+"\n"
+"In both cases, once you're done, continue with:\n"
+"\n"
+"  git rebase --continue\n");
+
 /*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
@@ -509,16 +523,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error("you have staged changes in your working "
-				"tree. If these changes are meant to be\n"
-				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend %s\n\n"
-				"If they are meant to go into a new commit, "
-				"run:\n\n"
-				"  git commit %s\n\n"
-				"In both cases, once you're done, continue "
-				"with:\n\n"
-				"  git rebase --continue\n", gpg_opt, gpg_opt);
+			return error(_(staged_changes_advice),
+				     gpg_opt, gpg_opt);
 		}
 	}
 
-- 
2.10.1.513.g00ef6dd

Re: [PATCH v3 13/25] sequencer: prepare for rebase -i's commit functionality

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I think my puzzlement comes from here.  What makes it OK for "am" to
>> expect the contents of author-script file to be quoted but it is not
>> OK to expect the same here?  What makes it not quoted for _this_
>> reader, in other words?
>
> The `git am` command is inherently *not* interactive, while the
> interactive rebase, well, is.
>
> As such, we must assume that enterprisey users did come up with scripts
> that edit, or create, author-script files, and exploited the fact that the
> interactive rebase previously sourced them.

It's like saying "While the program is waiting for user interaction,
the user can attach a debugger to tweak the contents of this string
variable.  Hence we must not assume that the string is always NUL
terminated to protect ourselves."

A correct response to such a user is to tell them not to do that,
and do so at two levels.  

The first level may be "it may be physically possible to tweak the
string via debugger but don't do that in such a way that breaks the
invariants our program relies on, or you are on your own", but more
important is the response at the second level.  Why is the user
futzing with that string in the first place with the debugger?  In
the context of author-script, the question is "Why is the user
futzing with the author-script file"?  To attribute the resulting
commit to a different author, of course.  But we need to step back
and think why the user needs to resort to editing that file to
achieve that.

If that is a common thing users would want to do, then we should
offer an official way to do so, and it should not be "you can futz
with author-script file with your editor".  Something like "have
'exec git commit --amend' in the todo file" may be more appropriate
and if it is important enough, the sequencer command language may
want to learn an extra verb to update the author.

Besides, the opportunity easiest for the user to futz with the
contents of author-script file (or "attach the debugger while we
wait for user interaction") arises when "rebase -i" or "am" stops
waiting for conflict resolution.  Even when you run "am" without its
"-i" option, it is equally susceptible to the "user futzing with the
file", which means your "am is not interactive but 'rebase -i' is"
is irrelevant to the issue.  Oh, also, did I say "am" has "-i"
option, which is a short-hand for "--interactive"?

What disturbs me the most is that I know you know the system well
enough to realize how bogus your argument to claim that "rebase -i"
and "am" are different was and to come up with what I wrote above
yourself.  Which means that I need to conclude that you have some
other reasons why you want to keep this parser different, but I
still do not know what they are.

I guess it entirely is possible that one of the reasons is because
some later patches in the larger "rebase-i to sequencer" series
writes author-script file in a syntax that cannot be read by the
recently refactored code "am" uses to read the author-script file,
and reusing the existing code may end up breaking the endgame
"rebase -i" you have.

As I do not feel like arguing with you on this any longer, and as I
certainly do not want to be blamed for breaking your "rebase -i", I
do not insist the code to be refactored to share with the existing
codepath.  But still I do not see the need to keep them separate (as
I already said in the previous message, I am OK if the one used in
"am" is updated to match).

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Torsten Bögershausen]

Not sure is this has been reported before:


sequencer.c:633:14: warning: comparison of constant 2 with expression of type 'const enum todo_command' is always true [-Wtautological-constant-out-of-range-compare]
        if (command < ARRAY_SIZE(todo_command_strings))
            ~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.


53f8024e (Johannes Schindelin   2016-10-10 19:25:07 +0200  633)         if (command < ARRAY_SIZE(todo_command_strings))

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Jeff King]

On Sat, Oct 15, 2016 at 07:03:46PM +0200, Torsten Bögershausen wrote:

> sequencer.c:633:14: warning: comparison of constant 2 with expression of type 'const enum todo_command' is always true [-Wtautological-constant-out-of-range-compare]
>         if (command < ARRAY_SIZE(todo_command_strings))
>             ~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 1 warning generated.
>
> 53f8024e (Johannes Schindelin   2016-10-10 19:25:07 +0200  633)         if (command < ARRAY_SIZE(todo_command_strings))
> 

Interesting. The compiler is right that this _should_ never happen, but
I think the patch is quite reasonable to be defensive in case the enum
happens to get a value outside of its acceptable range (which is
probably undefined behavior, but...).

I wonder if:

  if ((int)command < ARRAY_SIZE(todo_command_strings))

silences the warning (I suppose size_t is probably an even better type,
though obviously it does not matter in practice).

-Peff

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Torsten Bögershausen]

On 15.10.16 19:19, Jeff King wrote:
> On Sat, Oct 15, 2016 at 07:03:46PM +0200, Torsten Bögershausen wrote:
> 
>> sequencer.c:633:14: warning: comparison of constant 2 with expression of type 'const enum todo_command' is always true [-Wtautological-constant-out-of-range-compare]
>>         if (command < ARRAY_SIZE(todo_command_strings))
>>             ~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> 1 warning generated.
>>
>> 53f8024e (Johannes Schindelin   2016-10-10 19:25:07 +0200  633)         if (command < ARRAY_SIZE(todo_command_strings))
>>
> 
> Interesting. The compiler is right that this _should_ never happen, but
> I think the patch is quite reasonable to be defensive in case the enum
> happens to get a value outside of its acceptable range (which is
> probably undefined behavior, but...).
> 
> I wonder if:
> 
>   if ((int)command < ARRAY_SIZE(todo_command_strings))
> 
> silences the warning (I suppose size_t is probably an even better type,
> though obviously it does not matter in practice).
> 
> -Peff
> 
Both do (silence the warning)

enum may be signed or unsigned, right ?
So the size_t variant seams to be a better choice

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Jeff King]

On Sat, Oct 15, 2016 at 07:40:15PM +0200, Torsten Bögershausen wrote:

> > I wonder if:
> > 
> >   if ((int)command < ARRAY_SIZE(todo_command_strings))
> > 
> > silences the warning (I suppose size_t is probably an even better type,
> > though obviously it does not matter in practice).
> > 
> Both do (silence the warning)
> 
> enum may be signed or unsigned, right ?
> So the size_t variant seams to be a better choice

Good catch. It technically needs to check the lower bound, too. In
theory, if somebody wanted to add an enum value that is negative, you'd
use a signed cast and check against both 0 and ARRAY_SIZE(). In
practice, that is nonsense for this case, and using an unsigned type
means that any negative values become large, and the check catches them.

-Peff

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

Hi,

On Sat, 15 Oct 2016, Jeff King wrote:

> On Sat, Oct 15, 2016 at 07:40:15PM +0200, Torsten Bögershausen wrote:
> 
> > > I wonder if:
> > > 
> > >   if ((int)command < ARRAY_SIZE(todo_command_strings))
> > > 
> > > silences the warning (I suppose size_t is probably an even better type,
> > > though obviously it does not matter in practice).
> > > 
> > Both do (silence the warning)
> > 
> > enum may be signed or unsigned, right ?
> > So the size_t variant seams to be a better choice
> 
> Good catch. It technically needs to check the lower bound, too. In
> theory, if somebody wanted to add an enum value that is negative, you'd
> use a signed cast and check against both 0 and ARRAY_SIZE(). In
> practice, that is nonsense for this case, and using an unsigned type
> means that any negative values become large, and the check catches them.

I am pretty certain that I disagree with that warning: enums have been
used as equivalents of ints for a long time, and will be, for a long time
to come.

Given that this test is modified to `if (command < TODO_NOOP)` later, I
hope that you agree that it is not worth the trouble to appease that
compiler overreaction?

Ciao,
Dscho

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Jeff King]

On Sun, Oct 16, 2016 at 10:09:12AM +0200, Johannes Schindelin wrote:

> > Good catch. It technically needs to check the lower bound, too. In
> > theory, if somebody wanted to add an enum value that is negative, you'd
> > use a signed cast and check against both 0 and ARRAY_SIZE(). In
> > practice, that is nonsense for this case, and using an unsigned type
> > means that any negative values become large, and the check catches them.
> 
> I am pretty certain that I disagree with that warning: enums have been
> used as equivalents of ints for a long time, and will be, for a long time
> to come.

I'm not sure I agree. IIRC, Assigning values outside the range of an enum has
always been fishy according to the standard, and a compiler really is
allowed to allocate a single bit for storage for this enum.

> Given that this test is modified to `if (command < TODO_NOOP)` later, I
> hope that you agree that it is not worth the trouble to appease that
> compiler overreaction?

I don't mind if there are transient warnings on some compilers in the
middle of a series, but I'm not sure when "later" is. The tip of "pu"
has this warning right now when built with clang.

I'm happy to test the TODO_NOOP version against clang (and prepare a
patch on top if it still complains), but that doesn't seem to have
Junio's tree at all yet.

-Peff

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

Hi Peff,

On Sun, 16 Oct 2016, Jeff King wrote:

> On Sun, Oct 16, 2016 at 10:09:12AM +0200, Johannes Schindelin wrote:
> 
> > > Good catch. It technically needs to check the lower bound, too. In
> > > theory, if somebody wanted to add an enum value that is negative, you'd
> > > use a signed cast and check against both 0 and ARRAY_SIZE(). In
> > > practice, that is nonsense for this case, and using an unsigned type
> > > means that any negative values become large, and the check catches them.
> > 
> > I am pretty certain that I disagree with that warning: enums have been
> > used as equivalents of ints for a long time, and will be, for a long time
> > to come.
> 
> I'm not sure I agree. IIRC, Assigning values outside the range of an enum has
> always been fishy according to the standard, and a compiler really is
> allowed to allocate a single bit for storage for this enum.

Really? I did see my share of code that completely violated this freedom,
by assuming that it was really okay to cast a -1 to an enum and then test
for that value later, when -1 was not a legal enum value.

In any case, the fact that even one compiler used to build Git *may*
violate that standard, and that we therefore need such safety guards as
the one under discussion, still makes me think that this warning, while
certainly well-intentioned, is poison for cross-platform projects.

> > Given that this test is modified to `if (command < TODO_NOOP)` later, I
> > hope that you agree that it is not worth the trouble to appease that
> > compiler overreaction?
> 
> I don't mind if there are transient warnings on some compilers in the
> middle of a series, but I'm not sure when "later" is. The tip of "pu"
> has this warning right now when built with clang.

"Later" is the sequencer-i patch series I already sent out for review
[*1*], in particular the patch titled "sequencer (rebase -i):
differentiate between comments and 'noop'" [*2*].

> I'm happy to test the TODO_NOOP version against clang (and prepare a
> patch on top if it still complains), but that doesn't seem to have
> Junio's tree at all yet.

Junio chose to pick up only one patch series out of the rebase--helper
thicket at a time, it seems. I did send out at least one revision per
patch series prior to integrating them into Git for Windows v2.10.0,
though. Plus, I kept updating the `interactive-rebase` branch in my
repository on GitHub (https://github.com/dscho/git).

Ciao,
Dscho

Footnote *1*:
https://public-inbox.org/git/cover.1472633606.git.johannes.schindelin@gmx.de/

Footnote *2*:
https://public-inbox.org/git/736bcb8e860c876e32e8f89f68b0b901abedc187.1472633606.git.johannes.schindelin@gmx.de/t/#u

P.S.: I cannot wait for the day when somebody with an artistic touch
provides .css for the public-inbox.org site so it stops threatening
causing eye cancer to me.

Re: [PATCH v3 07/25] sequencer: completely revamp the "todo" script parsing

[Jeff King]

[tl;dr: the version in your repo is fine, and there's a trivial fix
 below if we want to silence the warning in the meantime]

On Mon, Oct 17, 2016 at 10:37:52AM +0200, Johannes Schindelin wrote:

> > I'm not sure I agree. IIRC, Assigning values outside the range of an enum has
> > always been fishy according to the standard, and a compiler really is
> > allowed to allocate a single bit for storage for this enum.
> 
> Really? I did see my share of code that completely violated this freedom,
> by assuming that it was really okay to cast a -1 to an enum and then test
> for that value later, when -1 was not a legal enum value.

I poked around a bit, and it seems we're both half-wrong. C99 says:

  6.7.2.2 Enumeration specifiers
  [...]
  The expression that defines the value of an enumeration constant shall
  be an integer constant expression that has a value representable as an
  int.
  [...]
  Each enumerated type shall be compatible with char, a signed integer
  type, or an unsigned integer type. The choice of type is
  implementation-defined, but shall be capable of representing the
  values of all the members of the enumeration.

My reading is that it can't be a single-bit bitfield as I claimed, but
it also isn't necessarily interchangeable with an int. But you get at
least a "char", and you can use all of those integer values even if they
aren't explicitly part of the set. And I'd assume that goes for values
even beyond the largest tag as long as you don't need more bits, so
that:

  enum { A = 1, B = 2, C = 4 } x = A | B | C;

is OK (though I didn't see anything particularly about that in the
standard).

Assigning "-1" works in the same way that normal "unsigned x = -1" works
(and is defined by the standard), though of course it may unexpectedly
conflict with an actual enum value if the compiler chooses a smaller
type (e.g., it may literally be 255 in many cases).

Anyway. Enough language lawyering. It seems like clang is being overly
strict in its interpretation of the standard (it should be giving us at
least a char's worth of values). But it matters less what the standard
says and more what real compilers do, and we have to deal with clang's
behavior.

> In any case, the fact that even one compiler used to build Git *may*
> violate that standard, and that we therefore need such safety guards as
> the one under discussion, still makes me think that this warning, while
> certainly well-intentioned, is poison for cross-platform projects.

Oh, I agree that the warning is annoying, and the code should not go
away. We just need to figure out how to silence clang.

> > I'm happy to test the TODO_NOOP version against clang (and prepare a
> > patch on top if it still complains), but that doesn't seem to have
> > Junio's tree at all yet.
> 
> Junio chose to pick up only one patch series out of the rebase--helper
> thicket at a time, it seems. I did send out at least one revision per
> patch series prior to integrating them into Git for Windows v2.10.0,
> though. Plus, I kept updating the `interactive-rebase` branch in my
> repository on GitHub (https://github.com/dscho/git).

Thanks, I was able to test that branch. It looks like clang is happy
with it because you compare against the max value. Unlike the
ARRAY_SIZE() check, this does mean if somebody modifies the enum without
touching the array, we might go out of bounds. But things would be
severely broken enough from the mismatch that I don't think it's worth
worrying about too much (and I see you have a nice comment warning
people about this).

If the rest of your interactive-rebase branch is coming soon, I think we
can probably ignore it for now. Otherwise something like:

diff --git a/sequencer.c b/sequencer.c
index d662c6b..1fdc35e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -620,7 +620,8 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 
 enum todo_command {
 	TODO_PICK = 0,
-	TODO_REVERT
+	TODO_REVERT,
+	TODO_MAX
 };
 

is probably the simplest portable fix.

As a more clever change, I wondered if switching the enum values from
(0,1) to (1,2) would silence the warning, and indeed it does. Which I
assume is because using bit-flags, we could now represent "1|2", or "3",
which is larger than the array (well, obviously "2" is, but we'd need to
subtract 1 when indexing the array). I don't think that's a good route,
though, because it loses the 0-indexing, the benefits of
zero-initialization, etc. I was mostly just poking at how clang
perceives the enum values.

> P.S.: I cannot wait for the day when somebody with an artistic touch
> provides .css for the public-inbox.org site so it stops threatening
> causing eye cancer to me.

Heh. I gently hinted something similar to Eric in the past, but I think
he actually likes how it looks. He has invited others to mirror
public-inbox and make their own interface, though. I just lack the
"artistic touch" you mentioned.

-Peff

Re: [PATCH v3 14/25] sequencer: introduce a helper to read files written by scripts

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> +/*
> + * Reads a file that was presumably written by a shell script, i.e.
> + * with an end-of-line marker that needs to be stripped.
> + *
> + * Returns 1 if the file was read, 0 if it could not be read or does not exist.
> + */
> +static int read_oneliner(struct strbuf *buf,
> +	const char *path, int skip_if_empty)
> +...
> +	if (strbuf_read_file(buf, path, 0) < 0) {
> +		warning_errno(_("could not read '%s'"), path);
> +		return 0;
> +	}
> +	if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
> +		if (--buf->len > orig_len && buf->buf[buf->len - 1] == '\r')
> +			--buf->len;
> +		buf->buf[buf->len] = '\0';
> +	}

The name says "oneliner" but this reads the whole thing and trims
only the last line of the input.  Which is correct?

Do we want to error out if we got more than one line?  That makes it
more strict.  Going in the other direction, do we want to just read
the first line and ignore the remainder?  That allows users to leave
cruft after what matters.  I _think_ the existing code is closer to
the latter, i.e. something along the lines of ...

	struct strbuf oneline = STRBUF_INIT;
	FILE *fp = fopen(path, "r");
	if (!fp) {
		warning_errno(_("could not open '%s'"), path);
		return 0;
	}
	if (strbuf_getline(&oneline, fp) < 0)
		; /* EOF - empty */
	else {
		strbuf_addbuf(buf, &oneline);
	}

Re: [PATCH v3 15/25] sequencer: allow editing the commit message on a case-by-case basis

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> In the upcoming commits, we will implement more and more of rebase -i's
> functionality inside the sequencer. One particular feature of the
> commands to come is that some of them allow editing the commit message
> while others don't, i.e. we cannot define in the replay_opts whether the
> commit message should be edited or not.
>
> Let's add a new parameter to the run_git_commit() function. Previously,
> it was the duty of the caller to ensure that the opts->edit setting
> indicates whether to let the user edit the commit message or not,
> indicating that it is an "all or nothing" setting, i.e. that the
> sequencer wants to let the user edit *all* commit message, or none at
> all. In the upcoming rebase -i mode, it will depend on the particular
> command that is currently executed, though.

Makes tons of sense.

Re: [PATCH v3 16/25] sequencer: support amending commits

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This teaches the run_git_commit() function to take an argument that will
> allow us to implement "todo" commands that need to amend the commit
> messages ("fixup", "squash" and "reword").

Likewise to 15/25, i.e. Good, though the growth by these two steps
starts to make me wonder if these three options should be crammed
into an unsigned "flags" bitword.

I see you have v4, so I'll ignore the remainder of this stale round
and start reading that updated one instead.

>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index b621f4b..403a4f0 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -481,7 +481,7 @@ static char **read_author_script(void)
>   * author metadata.
>   */
>  static int run_git_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit)
> +			  int allow_empty, int edit, int amend)
>  {
>  	char **env = NULL;
>  	struct argv_array array;
> @@ -510,6 +510,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
>  	argv_array_push(&array, "commit");
>  	argv_array_push(&array, "-n");
>  
> +	if (amend)
> +		argv_array_push(&array, "--amend");
>  	if (opts->gpg_sign)
>  		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
>  	if (opts->signoff)
> @@ -785,7 +787,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	}
>  	if (!opts->no_commit)
>  		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
> -				     opts, allow, opts->edit);
> +				     opts, allow, opts->edit, 0);
>  
>  leave:
>  	free_message(commit, &msg);

Re: [PATCH v4 05/25] sequencer: eventually release memory allocated for the option values

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
> like a one-shot command when it reads its configuration: memory is
> allocated and released only when the command exits.
>
> This is kind of okay for git-cherry-pick, which *is* a one-shot
> command. All the work to make the sequencer its work horse was
> done to allow using the functionality as a library function, though,
> including proper clean-up after use.
>
> To remedy that, we now take custody of the option values in question,
> requiring those values to be malloc()ed or strdup()ed

That is the approach this patch takes, so "eventually release" in
the title is no longer accurate, I would think.

> Sadly, the current approach makes the code uglier, as we now have to
> take care to strdup() the values passed via the command-line.

I obviously disagree with that statement and the _entrust was too
ugly to live, but it is obviously subjective, and it boils down to
who has a better taste.  Let's not go there.

> +
> +	/* These option values will be free()d */
> +	opts->gpg_sign = xstrdup_or_null(opts->gpg_sign);
> +	opts->strategy = xstrdup_or_null(opts->strategy);

xstrdup-or-null does make things cleaner.

> +static int git_config_string_dup(char **dest,
> +				 const char *var, const char *value)
> +{
> +	if (!value)
> +		return config_error_nonbool(var);
> +	free(*dest);
> +	*dest = xstrdup(value);
> +	return 0;
> +}

So does this.

Re: [PATCH v4 00/25] Prepare the sequencer for the upcoming rebase -i patches

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This patch series marks the '4' in the countdown to speed up rebase -i
> by implementing large parts in C (read: there will be three more patch
> series after that before the full benefit hits git.git: sequencer-i,
> rebase--helper and rebase-i-extra). It is based on the `libify-sequencer`
> patch series I submitted earlier.

The difference between the end results of v3 and v4 looked OK
(except the 08/25 "strip LF" change that is unneeded) to me, so I'll
skip the early part of the series from my review that correspond to
the ones in v3 that I've already reviewed and found good, and
continue from the later ones.

Thanks.

Re: [PATCH v4 08/25] sequencer: completely revamp the "todo" script parsing

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> -	for (i = 1; *p; i++) {
> +	for (i = 1; *p; i++, p = next_p) {
>  		char *eol = strchrnul(p, '\n');
> -		commit = parse_insn_line(p, eol, opts);
> -		if (!commit)
> -			return error(_("Could not parse line %d."), i);
> -		next = commit_list_append(commit, next);
> -		p = *eol ? eol + 1 : eol;
> +
> +		next_p = *eol ? eol + 1 /* strip LF */ : eol;

This one was explained as "skip LF" in the previous round, and that
is more correct than "strip", I think.  The +1 here is not done to
"strip" the LF out of the end result, but to "skip" one to move to
the beginning of the next line.

The one in v3 08/25 decremented the pointer to denote the end of the
line with the explicit purpose of not including the CR in the end
result, which was explained as "skip CR", but it was stripping CR.
Correcting that explanation to "strip" was a right fix and I think
your v4 09/25 still has it, which is good.

Other than this unnecessary change since the previous round, I
didn't spot a difference in this step, which was already good.

Thanks.

Re: [PATCH v3 14/25] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

Hi Junio,

On Mon, 17 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > +/*
> > + * Reads a file that was presumably written by a shell script, i.e.
> > + * with an end-of-line marker that needs to be stripped.
> > + *
> > + * Returns 1 if the file was read, 0 if it could not be read or does not exist.
> > + */
> > +static int read_oneliner(struct strbuf *buf,
> > +	const char *path, int skip_if_empty)
> > +...
> > +	if (strbuf_read_file(buf, path, 0) < 0) {
> > +		warning_errno(_("could not read '%s'"), path);
> > +		return 0;
> > +	}
> > +	if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
> > +		if (--buf->len > orig_len && buf->buf[buf->len - 1] == '\r')
> > +			--buf->len;
> > +		buf->buf[buf->len] = '\0';
> > +	}
> 
> The name says "oneliner" but this reads the whole thing and trims
> only the last line of the input.  Which is correct?

The latter. Basically, `read_oneliner()` is short-hand for "that thing
that shell does when you use `cat file` with backticks.

I do not like `read_stripping_last_eol()`, `read_what_the_shell_wrote()`
nor `read_skipping_last_lf()`. So if you come up with any brilliant idea,
I am all ears.

In the meantime, I'd be happy to just add a comment that this function is
intended for oneliners, but that it will also read multi-line files and
only strip off the EOL marker from the last line.

Would that work for you?
Dscho

Re: [PATCH v3 16/25] sequencer: support amending commits

[Johannes Schindelin]

Hi Junio,

On Mon, 17 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > This teaches the run_git_commit() function to take an argument that will
> > allow us to implement "todo" commands that need to amend the commit
> > messages ("fixup", "squash" and "reword").
> 
> Likewise to 15/25, i.e. Good, though the growth by these two steps
> starts to make me wonder if these three options should be crammed
> into an unsigned "flags" bitword.

After looking at the diff with the added complications of ORing and ANDing
the flags, I'd much rather prefer to stay with the three flags being kept
separately. It's not like we need to save bits, but we need to preserve
readability as much as possible, I'd wager.

> I see you have v4, so I'll ignore the remainder of this stale round
> and start reading that updated one instead.

Thanks,
Dscho

Re: [PATCH v4 05/25] sequencer: eventually release memory allocated for the option values

[Johannes Schindelin]

Hi Junio,

On Mon, 17 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
> > like a one-shot command when it reads its configuration: memory is
> > allocated and released only when the command exits.
> >
> > This is kind of okay for git-cherry-pick, which *is* a one-shot
> > command. All the work to make the sequencer its work horse was
> > done to allow using the functionality as a library function, though,
> > including proper clean-up after use.
> >
> > To remedy that, we now take custody of the option values in question,
> > requiring those values to be malloc()ed or strdup()ed
> 
> That is the approach this patch takes, so "eventually release" in
> the title is no longer accurate, I would think.

To the contrary, we now free() things in remove_state(), so we still
"eventually release" the memory.

> > Sadly, the current approach makes the code uglier, as we now have to
> > take care to strdup() the values passed via the command-line.
> 
> I obviously disagree with that statement and the _entrust was too
> ugly to live, but it is obviously subjective, and it boils down to
> who has a better taste.  Let's not go there.

Changed.

Thanks,
Dscho

Re: [PATCH v4 08/25] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

Hi Junio,

On Mon, 17 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > -	for (i = 1; *p; i++) {
> > +	for (i = 1; *p; i++, p = next_p) {
> >  		char *eol = strchrnul(p, '\n');
> > -		commit = parse_insn_line(p, eol, opts);
> > -		if (!commit)
> > -			return error(_("Could not parse line %d."), i);
> > -		next = commit_list_append(commit, next);
> > -		p = *eol ? eol + 1 : eol;
> > +
> > +		next_p = *eol ? eol + 1 /* strip LF */ : eol;
> 
> This one was explained as "skip LF" in the previous round, and that
> is more correct than "strip", I think.  The +1 here is not done to
> "strip" the LF out of the end result, but to "skip" one to move to
> the beginning of the next line.

Changed,
Dscho

Re: [PATCH v3 14/25] sequencer: introduce a helper to read files written by scripts

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> In the meantime, I'd be happy to just add a comment that this function is
> intended for oneliners, but that it will also read multi-line files and
> only strip off the EOL marker from the last line.
>
> Would that work for you?

That would be ideal, I would think.

Re: [PATCH v3 16/25] sequencer: support amending commits

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Mon, 17 Oct 2016, Junio C Hamano wrote:
>
>> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>> 
>> > This teaches the run_git_commit() function to take an argument that will
>> > allow us to implement "todo" commands that need to amend the commit
>> > messages ("fixup", "squash" and "reword").
>> 
>> Likewise to 15/25, i.e. Good, though the growth by these two steps
>> starts to make me wonder if these three options should be crammed
>> into an unsigned "flags" bitword.
>
> After looking at the diff with the added complications of ORing and ANDing
> the flags, I'd much rather prefer to stay with the three flags being kept
> separately. It's not like we need to save bits, but we need to preserve
> readability as much as possible, I'd wager.

That's OK.  I just wanted to make sure pros-and-cons have been
already considered.

The primary merit of using flags bitword is not to save bits; it is
done to limit the damage to the codebase when we need to add yet
another knob, by the way.

Re: [PATCH v4 05/25] sequencer: eventually release memory allocated for the option values

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> > To remedy that, we now take custody of the option values in question,
>> > requiring those values to be malloc()ed or strdup()ed
>> 
>> That is the approach this patch takes, so "eventually release" in
>> the title is no longer accurate, I would think.
>
> To the contrary, we now free() things in remove_state(), so we still
> "eventually release" the memory.

OK.  We call a change to teach remove_state() to free the resource
does more commonly as "plug leaks"; the word "eventually" gave me an
impression that we are emphasizing the fact that we do not free(3)
immediately but lazily do so at the end, hence my response.

Re: [PATCH v3 14/25] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

Hi Junio,

On Tue, 18 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > In the meantime, I'd be happy to just add a comment that this function is
> > intended for oneliners, but that it will also read multi-line files and
> > only strip off the EOL marker from the last line.
> >
> > Would that work for you?
> 
> That would be ideal, I would think.

Done,
Dscho

Re: [PATCH v4 05/25] sequencer: eventually release memory allocated for the option values

[Johannes Schindelin]

Hi Junio,

On Tue, 18 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> > To remedy that, we now take custody of the option values in question,
> >> > requiring those values to be malloc()ed or strdup()ed
> >> 
> >> That is the approach this patch takes, so "eventually release" in
> >> the title is no longer accurate, I would think.
> >
> > To the contrary, we now free() things in remove_state(), so we still
> > "eventually release" the memory.
> 
> OK.  We call a change to teach remove_state() to free the resource
> does more commonly as "plug leaks"; the word "eventually" gave me an
> impression that we are emphasizing the fact that we do not free(3)
> immediately but lazily do so at the end, hence my response.

I changed the commit subject, hopefully to your liking,
Dscho

Re: [PATCH v4 17/25] sequencer: support cleaning up commit messages

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The run_git_commit() function already knows how to amend commits, and
> with this new option, it can also clean up commit messages (i.e. strip
> out commented lines). This is needed to implement rebase -i's 'fixup'
> and 'squash' commands as sequencer commands.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index fa77c82..cbc3742 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -481,7 +481,8 @@ static char **read_author_script(void)
>   * author metadata.
>   */
>  static int run_git_commit(const char *defmsg, struct replay_opts *opts,
> -			  int allow_empty, int edit, int amend)
> +			  int allow_empty, int edit, int amend,
> +			  int cleanup_commit_message)
>  {
>  	char **env = NULL;
>  	struct argv_array array;

Looks OK to me.  

This starts to look like calling for a single flag word even more,
but it is a file-local helper so we can clean it up if it becomes
necessary without affecting too many things later.

> @@ -518,9 +519,12 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
>  		argv_array_push(&array, "-s");
>  	if (defmsg)
>  		argv_array_pushl(&array, "-F", defmsg, NULL);
> +	if (cleanup_commit_message)
> +		argv_array_push(&array, "--cleanup=strip");
>  	if (edit)
>  		argv_array_push(&array, "-e");
> -	else if (!opts->signoff && !opts->record_origin &&
> +	else if (!cleanup_commit_message &&
> +		 !opts->signoff && !opts->record_origin &&
>  		 git_config_get_value("commit.cleanup", &value))
>  		argv_array_push(&array, "--cleanup=verbatim");
>  
> @@ -785,7 +789,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  	}
>  	if (!opts->no_commit)
>  		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
> -				     opts, allow, opts->edit, 0);
> +				     opts, allow, opts->edit, 0, 0);
>  
>  leave:
>  	free_message(commit, &msg);

Re: [PATCH v4 18/25] sequencer: do not try to commit when there were merge conflicts

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The return value of do_recursive_merge() may be positive (indicating merge
> conflicts), or 0 (indicating success). It also may be negative, indicating
> a fatal error that requires us to abort.
>
> Now, if the return value indicates that there are merge conflicts, we
> should not try to commit those changes, of course.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index cbc3742..9ffc090 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -787,7 +787,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		res = allow;
>  		goto leave;
>  	}
> -	if (!opts->no_commit)
> +	if (!res && !opts->no_commit)
>  		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
>  				     opts, allow, opts->edit, 0, 0);

This by itself looks more like a bugfix than preparation for later
steps.  The only reason why it is not a bugfix is because there is
nothing in this function that makes res a non-zero value and reach
this if statement at this step.  We would have been caught by an 
"if (res) { ... rerere(); goto leave; }" or 
"if (allow < 0) { res = allow; goto leave; }" 
that appear before this part of the code.

So while it is not wrong per-se, I think this should be part of an
actual change that makes it possible for the control flow to reach
here with non-zero res.

Re: [PATCH v4 20/25] sequencer: refactor write_message()

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> The write_message() function safely writes an strbuf to a file.
> Sometimes it is inconvenient to require an strbuf, though: the text to
> be written may not be stored in a strbuf, or the strbuf should not be
> released after writing.
>
> Let's refactor "safely writing string to a file" into
> write_with_lock_file(), and make write_message() use it.
>
> The new function will make it easy to create a new convenience function
> write_file_gently() that does not die(); as some of the upcoming callers
> of this new function will want to append a newline character, we already
> add that flag as parameter to write_with_lock_file().
>
> While at it, roll back the locked files in case of failure, as pointed
> out by Hannes Sixt.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  sequencer.c | 25 ++++++++++++++++++++-----
>  1 file changed, 20 insertions(+), 5 deletions(-)

Once a helper function starts taking <buf, len> pair, not a strbuf,
it becomes obvious that it does not make much sense to calling
strbuf_release() from the helper.  It is caller's job to do the
memory management of the strbuf it uses to pass information to the
helper, and the current api into write_message() is klunky.

If I were doing this, I would make this into three separate steps:

    - move the strbuf_release(msgbuf) to the caller in
      do_pick_commit();

    - add the missing rollback_lock_file(), which is a bugfix; and
      then finally

    - allow the helper to take not a strbuf but <buf, len> pair as
      parameters.

The end result of this patch achieves two thirds of the above, but
especially given that write_message() only has two call sites in a
single function, I think it is OK and preferrable even to do all
three.

Re: [PATCH v4 20/25] sequencer: refactor write_message()

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> If I were doing this, I would make this into three separate steps:
>
>     - move the strbuf_release(msgbuf) to the caller in
>       do_pick_commit();
>
>     - add the missing rollback_lock_file(), which is a bugfix; and
>       then finally
>
>     - allow the helper to take not a strbuf but <buf, len> pair as
>       parameters.
>
> The end result of this patch achieves two thirds of the above, but
> especially given that write_message() only has two call sites in a
> single function, I think it is OK and preferrable even to do all
> three.

Ah, make that four steps.  The final one is:

    - add append_eol parameter that nobody uses at this step in the
      series.

This is a new feature to the helper.  While it is OK to have it as a
preparatory step in this series, it is easier to understand if it
kept as a separate step.  It is even more preferrable if it is made
as a preparatory step in a series that adds a caller that passes
true to append_eol to this helper, or if that real change is small
enough, part of that patch that adds such a caller, not as a
separate step.

Re: [PATCH v4 23/25] sequencer: quote filenames in error messages

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This makes the code consistent by fixing quite a couple of error messages.
>
> Suggested by Jakub Narębski.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

These finishing touches in 21-23 look all sensible to me.

Re: [PATCH v4 23/25] sequencer: quote filenames in error messages

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
>
>> This makes the code consistent by fixing quite a couple of error messages.
>>
>> Suggested by Jakub Narębski.
>>
>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>> ---
>
> These finishing touches in 21-23 look all sensible to me.

Make that 21-25.  I finished reading to the end and it was mostly a
pleasnt read, except for a few things I noticed and sent reviews
separately.

Thanks.

Re: [PATCH v4 18/25] sequencer: do not try to commit when there were merge conflicts

[Johannes Schindelin]

Hi Junio,

On Thu, 20 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > The return value of do_recursive_merge() may be positive (indicating merge
> > conflicts), or 0 (indicating success). It also may be negative, indicating
> > a fatal error that requires us to abort.
> >
> > Now, if the return value indicates that there are merge conflicts, we
> > should not try to commit those changes, of course.
> >
> > Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> > ---
> >  sequencer.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/sequencer.c b/sequencer.c
> > index cbc3742..9ffc090 100644
> > --- a/sequencer.c
> > +++ b/sequencer.c
> > @@ -787,7 +787,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
> >  		res = allow;
> >  		goto leave;
> >  	}
> > -	if (!opts->no_commit)
> > +	if (!res && !opts->no_commit)
> >  		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
> >  				     opts, allow, opts->edit, 0, 0);
> 
> This by itself looks more like a bugfix than preparation for later
> steps.  The only reason why it is not a bugfix is because there is
> nothing in this function that makes res a non-zero value and reach
> this if statement at this step.  We would have been caught by an 
> "if (res) { ... rerere(); goto leave; }" or 
> "if (allow < 0) { res = allow; goto leave; }" 
> that appear before this part of the code.
> 
> So while it is not wrong per-se, I think this should be part of an
> actual change that makes it possible for the control flow to reach
> here with non-zero res.

It looks like it is no longer needed (I *think* that it was made obsolete
by the change where I now "goto fast_forward_edit" only in case there were
no errors).

In any case, the patch's gone now,
Dscho

Re: [PATCH v4 20/25] sequencer: refactor write_message()

[Johannes Schindelin]

Hi Junio,

On Thu, 20 Oct 2016, Junio C Hamano wrote:

> Junio C Hamano <gitster@pobox.com> writes:
> 
> > If I were doing this, I would make this into three separate steps:
> >
> >     - move the strbuf_release(msgbuf) to the caller in
> >       do_pick_commit();
> >
> >     - add the missing rollback_lock_file(), which is a bugfix; and
> >       then finally
> >
> >     - allow the helper to take not a strbuf but <buf, len> pair as
> >       parameters.
> >
> > The end result of this patch achieves two thirds of the above, but
> > especially given that write_message() only has two call sites in a
> > single function, I think it is OK and preferrable even to do all
> > three.
> 
> Ah, make that four steps.  The final one is:
> 
>     - add append_eol parameter that nobody uses at this step in the
>       series.

Done,
Dscho

[PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

This patch series marks the '4' in the countdown to speed up rebase -i
by implementing large parts in C (read: there will be three more patch
series after that before the full benefit hits git.git: sequencer-i,
rebase--helper and rebase-i-extra). It is based on the `libify-sequencer`
patch series I submitted earlier.

The patches in this series merely prepare the sequencer code for the
next patch series that actually teaches the sequencer to run rebase -i's
commands.

The reason to split these two patch series is simple: to keep them at a
sensible size.

The two patch series after that are much smaller: a two-patch "series"
that switches rebase -i to use the sequencer (except with --root or
--preserve-merges), and a couple of patches to move several pretty
expensive script processing steps to C (think: autosquash).

The end game of this patch series is a git-rebase--helper that makes
rebase -i 5x faster on Windows (according to t/perf/p3404). Travis says
that even MacOSX and Linux benefit (4x and 3x, respectively).

I have been working on this since early February, whenever time allowed,
and it is time to put it into the users' hands. To that end, I already
integrated the whole shebang into Git for Windows 2.10.0 and 2.10.1
where it has been running without complaints (and some quite positive
feedback).

It would be *really* nice if we could get this patch series at least into
`next` soon, as it gets late and later for the rest of the patches to make
it into `master` in time for v2.11 (and it is not for lack of trying on my
end...).

Changes vs v4:

- clarified in a code comment that read_oneliner() is lenient when it
  comes to multi-line files: it still reads the entire file, but strips
  off only the final EOL (if any).

- rephrased a commit message to stop judging Junio's taste ;-)

- changed "strip LF" to "skip LF", as requested.

- rephrased a commit message to talk about pluggin memory leaks instead
  of stating that the memory is now eventually released.

- dropped the "sequencer: do not try to commit when there were merge
  conflicts" patch that appears to be no longer necessary.

- split and modified the commit refactoring write_message() according to
  Junio's suggestions.


Johannes Schindelin (27):
  sequencer: use static initializers for replay_opts
  sequencer: use memoized sequencer directory path
  sequencer: avoid unnecessary indirection
  sequencer: future-proof remove_sequencer_state()
  sequencer: plug memory leaks for the option values
  sequencer: future-proof read_populate_todo()
  sequencer: refactor the code to obtain a short commit name
  sequencer: completely revamp the "todo" script parsing
  sequencer: strip CR from the todo script
  sequencer: avoid completely different messages for different actions
  sequencer: get rid of the subcommand field
  sequencer: remember the onelines when parsing the todo file
  sequencer: prepare for rebase -i's commit functionality
  sequencer: introduce a helper to read files written by scripts
  sequencer: allow editing the commit message on a case-by-case basis
  sequencer: support amending commits
  sequencer: support cleaning up commit messages
  sequencer: left-trim lines read from the script
  sequencer: stop releasing the strbuf in write_message()
  sequencer: roll back lock file if write_message() failed
  sequencer: refactor write_message() to take a pointer/length
  sequencer: teach write_message() to append an optional LF
  sequencer: remove overzealous assumption in rebase -i mode
  sequencer: mark action_name() for translation
  sequencer: quote filenames in error messages
  sequencer: start error messages consistently with lower case
  sequencer: mark all error messages for translation

 builtin/commit.c              |   2 +-
 builtin/revert.c              |  46 ++-
 sequencer.c                   | 680 ++++++++++++++++++++++++++++--------------
 sequencer.h                   |  23 +-
 t/t3501-revert-cherry-pick.sh |   2 +-
 5 files changed, 492 insertions(+), 261 deletions(-)


base-commit: 659889482ac63411daea38b2c3d127842ea04e4d
Published-As: https://github.com/dscho/git/releases/tag/prepare-sequencer-v5
Fetch-It-Via: git fetch https://github.com/dscho/git prepare-sequencer-v5

Interdiff vs v4:

 diff --git a/sequencer.c b/sequencer.c
 index 1cf70f7..a61fe76 100644
 --- a/sequencer.c
 +++ b/sequencer.c
 @@ -234,8 +234,8 @@ static void print_advice(int show_hint, struct replay_opts *opts)
  	}
  }
  
 -static int write_with_lock_file(const char *filename,
 -				const void *buf, size_t len, int append_eol)
 +static int write_message(const void *buf, size_t len, const char *filename,
 +			 int append_eol)
  {
  	static struct lock_file msg_file;
  
 @@ -258,16 +258,12 @@ static int write_with_lock_file(const char *filename,
  	return 0;
  }
  
 -static int write_message(struct strbuf *msgbuf, const char *filename)
 -{
 -	int res = write_with_lock_file(filename, msgbuf->buf, msgbuf->len, 0);
 -	strbuf_release(msgbuf);
 -	return res;
 -}
 -
  /*
 - * Reads a file that was presumably written by a shell script, i.e.
 - * with an end-of-line marker that needs to be stripped.
 + * Reads a file that was presumably written by a shell script, i.e. with an
 + * end-of-line marker that needs to be stripped.
 + *
 + * Note that only the last end-of-line marker is stripped, consistent with the
 + * behavior of "$(cat path)" in a shell script.
   *
   * Returns 1 if the file was read, 0 if it could not be read or does not exist.
   */
 @@ -762,12 +758,14 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  					 head, &msgbuf, opts);
  		if (res < 0)
  			return res;
 -		res |= write_message(&msgbuf, git_path_merge_msg());
 +		res |= write_message(msgbuf.buf, msgbuf.len,
 +				     git_path_merge_msg(), 0);
  	} else {
  		struct commit_list *common = NULL;
  		struct commit_list *remotes = NULL;
  
 -		res = write_message(&msgbuf, git_path_merge_msg());
 +		res = write_message(msgbuf.buf, msgbuf.len,
 +				    git_path_merge_msg(), 0);
  
  		commit_list_insert(base, &common);
  		commit_list_insert(next, &remotes);
 @@ -777,6 +775,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		free_commit_list(common);
  		free_commit_list(remotes);
  	}
 +	strbuf_release(&msgbuf);
  
  	/*
  	 * If the merge was clean or if it failed due to conflict, we write
 @@ -808,7 +807,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
  		res = allow;
  		goto leave;
  	}
 -	if (!res && !opts->no_commit)
 +	if (!opts->no_commit)
  		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
  				     opts, allow, opts->edit, 0, 0);
  
 @@ -934,7 +933,7 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
  	for (i = 1; *p; i++, p = next_p) {
  		char *eol = strchrnul(p, '\n');
  
 -		next_p = *eol ? eol + 1 /* strip LF */ : eol;
 +		next_p = *eol ? eol + 1 /* skip LF */ : eol;
  
  		if (p != eol && eol[-1] == '\r')
  			eol--; /* strip Carriage Return */

-- 
2.10.1.583.g721a9e0

[PATCH v5 01/27] sequencer: use static initializers for replay_opts

[Johannes Schindelin]

This change is not completely faithful: instead of initializing all fields
to 0, we choose to initialize command and subcommand to -1 (instead of
defaulting to REPLAY_REVERT and REPLAY_NONE, respectively). Practically,
it makes no difference at all, but future-proofs the code to require
explicit assignments for both fields.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 6 ++----
 sequencer.h      | 1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 4e69380..7365559 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -178,10 +178,9 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	if (isatty(0))
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
@@ -195,10 +194,9 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 
 int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 {
-	struct replay_opts opts;
+	struct replay_opts opts = REPLAY_OPTS_INIT;
 	int res;
 
-	memset(&opts, 0, sizeof(opts));
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
 	parse_args(argc, argv, &opts);
diff --git a/sequencer.h b/sequencer.h
index 5ed5cb1..db425ad 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -47,6 +47,7 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
+#define REPLAY_OPTS_INIT { -1, -1 }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
 
-- 
2.10.1.583.g721a9e0

[PATCH v5 02/27] sequencer: use memoized sequencer directory path

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/commit.c |  2 +-
 sequencer.c      | 11 ++++++-----
 sequencer.h      |  5 +----
 3 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/builtin/commit.c b/builtin/commit.c
index 1cba3b7..9fddb19 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -183,7 +183,7 @@ static void determine_whence(struct wt_status *s)
 		whence = FROM_MERGE;
 	else if (file_exists(git_path_cherry_pick_head())) {
 		whence = FROM_CHERRY_PICK;
-		if (file_exists(git_path(SEQ_DIR)))
+		if (file_exists(git_path_seq_dir()))
 			sequencer_in_use = 1;
 	}
 	else
diff --git a/sequencer.c b/sequencer.c
index eec8a60..cb16cbd 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -21,10 +21,11 @@
 const char sign_off_header[] = "Signed-off-by: ";
 static const char cherry_picked_prefix[] = "(cherry picked from commit ";
 
-static GIT_PATH_FUNC(git_path_todo_file, SEQ_TODO_FILE)
-static GIT_PATH_FUNC(git_path_opts_file, SEQ_OPTS_FILE)
-static GIT_PATH_FUNC(git_path_seq_dir, SEQ_DIR)
-static GIT_PATH_FUNC(git_path_head_file, SEQ_HEAD_FILE)
+GIT_PATH_FUNC(git_path_seq_dir, "sequencer")
+
+static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
+static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
+static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
 static int is_rfc2822_line(const char *buf, int len)
 {
@@ -112,7 +113,7 @@ static void remove_sequencer_state(void)
 {
 	struct strbuf seq_dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path(SEQ_DIR));
+	strbuf_addstr(&seq_dir, git_path_seq_dir());
 	remove_dir_recursively(&seq_dir, 0);
 	strbuf_release(&seq_dir);
 }
diff --git a/sequencer.h b/sequencer.h
index db425ad..dd4d33a 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -1,10 +1,7 @@
 #ifndef SEQUENCER_H
 #define SEQUENCER_H
 
-#define SEQ_DIR		"sequencer"
-#define SEQ_HEAD_FILE	"sequencer/head"
-#define SEQ_TODO_FILE	"sequencer/todo"
-#define SEQ_OPTS_FILE	"sequencer/opts"
+const char *git_path_seq_dir(void);
 
 #define APPEND_SIGNOFF_DEDUP (1u << 0)
 
-- 
2.10.1.583.g721a9e0

[PATCH v5 03/27] sequencer: avoid unnecessary indirection

[Johannes Schindelin]

We really do not need the *pointer to a* pointer to the options in
the read_populate_opts() function.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index cb16cbd..c2fbf6f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -813,7 +813,7 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	return 0;
 }
 
-static int read_populate_opts(struct replay_opts **opts)
+static int read_populate_opts(struct replay_opts *opts)
 {
 	if (!file_exists(git_path_opts_file()))
 		return 0;
@@ -823,7 +823,7 @@ static int read_populate_opts(struct replay_opts **opts)
 	 * about this case, though, because we wrote that file ourselves, so we
 	 * are pretty certain that it is syntactically correct.
 	 */
-	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), *opts) < 0)
+	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
 		return error(_("Malformed options sheet: %s"),
 			git_path_opts_file());
 	return 0;
@@ -1054,7 +1054,7 @@ static int sequencer_continue(struct replay_opts *opts)
 
 	if (!file_exists(git_path_todo_file()))
 		return continue_single_pick();
-	if (read_populate_opts(&opts) ||
+	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
 		return -1;
 
-- 
2.10.1.583.g721a9e0

[PATCH v5 04/27] sequencer: future-proof remove_sequencer_state()

[Johannes Schindelin]

In a couple of commits, we will teach the sequencer to handle the
nitty gritty of the interactive rebase, which keeps its state in a
different directory.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index c2fbf6f..8d272fb 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,11 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+static const char *get_dir(const struct replay_opts *opts)
+{
+	return git_path_seq_dir();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -109,13 +114,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(void)
+static void remove_sequencer_state(const struct replay_opts *opts)
 {
-	struct strbuf seq_dir = STRBUF_INIT;
+	struct strbuf dir = STRBUF_INIT;
 
-	strbuf_addstr(&seq_dir, git_path_seq_dir());
-	remove_dir_recursively(&seq_dir, 0);
-	strbuf_release(&seq_dir);
+	strbuf_addf(&dir, "%s", get_dir(opts));
+	remove_dir_recursively(&dir, 0);
+	strbuf_release(&dir);
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -940,7 +945,7 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	strbuf_release(&buf);
 	return 0;
 fail:
@@ -1034,7 +1039,7 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state();
+	remove_sequencer_state(opts);
 	return 0;
 }
 
@@ -1095,7 +1100,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	 * one that is being continued
 	 */
 	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state();
+		remove_sequencer_state(opts);
 		return 0;
 	}
 	if (opts->subcommand == REPLAY_ROLLBACK)
-- 
2.10.1.583.g721a9e0

[PATCH v5 05/27] sequencer: plug memory leaks for the option values

[Johannes Schindelin]

The sequencer is our attempt to lib-ify cherry-pick. Yet it behaves
like a one-shot command when it reads its configuration: memory is
allocated and released only when the command exits.

This is kind of okay for git-cherry-pick, which *is* a one-shot
command. All the work to make the sequencer its work horse was
done to allow using the functionality as a library function, though,
including proper clean-up after use.

To remedy that, we now take custody of the option values in question,
requiring those values to be malloc()ed or strdup()ed (an alternative
approach, to add a list of pointers to malloc()ed data and to ask the
sequencer to release all of them in the end, was proposed earlier but
rejected during review).

Note: this means that we now have to take care to strdup() the values
passed via the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c |  4 ++++
 sequencer.c      | 26 ++++++++++++++++++++++----
 sequencer.h      |  6 +++---
 3 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index 7365559..ba5a88c 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -174,6 +174,10 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 
 	if (argc > 1)
 		usage_with_options(usage_str, options);
+
+	/* These option values will be free()d */
+	opts->gpg_sign = xstrdup_or_null(opts->gpg_sign);
+	opts->strategy = xstrdup_or_null(opts->strategy);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
diff --git a/sequencer.c b/sequencer.c
index 8d272fb..04c55f2 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -117,6 +117,13 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 static void remove_sequencer_state(const struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
+	int i;
+
+	free(opts->gpg_sign);
+	free(opts->strategy);
+	for (i = 0; i < opts->xopts_nr; i++)
+		free(opts->xopts[i]);
+	free(opts->xopts);
 
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
@@ -280,7 +287,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	struct merge_options o;
 	struct tree *result, *next_tree, *base_tree, *head_tree;
 	int clean;
-	const char **xopt;
+	char **xopt;
 	static struct lock_file index_lock;
 
 	hold_locked_index(&index_lock, 1);
@@ -583,7 +590,8 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 
 		commit_list_insert(base, &common);
 		commit_list_insert(next, &remotes);
-		res |= try_merge_command(opts->strategy, opts->xopts_nr, opts->xopts,
+		res |= try_merge_command(opts->strategy,
+					 opts->xopts_nr, (const char **)opts->xopts,
 					common, sha1_to_hex(head), remotes);
 		free_commit_list(common);
 		free_commit_list(remotes);
@@ -783,6 +791,16 @@ static int read_populate_todo(struct commit_list **todo_list,
 	return 0;
 }
 
+static int git_config_string_dup(char **dest,
+				 const char *var, const char *value)
+{
+	if (!value)
+		return config_error_nonbool(var);
+	free(*dest);
+	*dest = xstrdup(value);
+	return 0;
+}
+
 static int populate_opts_cb(const char *key, const char *value, void *data)
 {
 	struct replay_opts *opts = data;
@@ -803,9 +821,9 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 	else if (!strcmp(key, "options.mainline"))
 		opts->mainline = git_config_int(key, value);
 	else if (!strcmp(key, "options.strategy"))
-		git_config_string(&opts->strategy, key, value);
+		git_config_string_dup(&opts->strategy, key, value);
 	else if (!strcmp(key, "options.gpg-sign"))
-		git_config_string(&opts->gpg_sign, key, value);
+		git_config_string_dup(&opts->gpg_sign, key, value);
 	else if (!strcmp(key, "options.strategy-option")) {
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
 		opts->xopts[opts->xopts_nr++] = xstrdup(value);
diff --git a/sequencer.h b/sequencer.h
index dd4d33a..8453669 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -34,11 +34,11 @@ struct replay_opts {
 
 	int mainline;
 
-	const char *gpg_sign;
+	char *gpg_sign;
 
 	/* Merge strategy */
-	const char *strategy;
-	const char **xopts;
+	char *strategy;
+	char **xopts;
 	size_t xopts_nr, xopts_alloc;
 
 	/* Only used by REPLAY_NONE */
-- 
2.10.1.583.g721a9e0

[PATCH v5 06/27] sequencer: future-proof read_populate_todo()

[Johannes Schindelin]

Over the next commits, we will work on improving the sequencer to the
point where it can process the todo script of an interactive rebase. To
that end, we will need to teach the sequencer to read interactive
rebase's todo file. In preparation, we consolidate all places where
that todo file is needed to call a function that we will later extend.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 04c55f2..fb0b94b 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -32,6 +32,11 @@ static const char *get_dir(const struct replay_opts *opts)
 	return git_path_seq_dir();
 }
 
+static const char *get_todo_path(const struct replay_opts *opts)
+{
+	return git_path_todo_file();
+}
+
 static int is_rfc2822_line(const char *buf, int len)
 {
 	int i;
@@ -769,25 +774,24 @@ static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
 static int read_populate_todo(struct commit_list **todo_list,
 			struct replay_opts *opts)
 {
+	const char *todo_file = get_todo_path(opts);
 	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
-	fd = open(git_path_todo_file(), O_RDONLY);
+	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"),
-				   git_path_todo_file());
+		return error_errno(_("Could not open %s"), todo_file);
 	if (strbuf_read(&buf, fd, 0) < 0) {
 		close(fd);
 		strbuf_release(&buf);
-		return error(_("Could not read %s."), git_path_todo_file());
+		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(buf.buf, todo_list, opts);
 	strbuf_release(&buf);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"),
-			git_path_todo_file());
+		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
@@ -1075,7 +1079,7 @@ static int sequencer_continue(struct replay_opts *opts)
 {
 	struct commit_list *todo_list = NULL;
 
-	if (!file_exists(git_path_todo_file()))
+	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts) ||
 			read_populate_todo(&todo_list, opts))
-- 
2.10.1.583.g721a9e0

[PATCH v5 07/27] sequencer: refactor the code to obtain a short commit name

[Johannes Schindelin]

Not only does this DRY up the code (providing a better documentation what
the code is about, as well as allowing to change the behavior in a single
place), it also makes it substantially shorter to use the same
functionality in functions to be introduced when we teach the sequencer to
process interactive-rebase's git-rebase-todo file.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index fb0b94b..499f5ee 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -147,13 +147,18 @@ struct commit_message {
 	const char *message;
 };
 
+static const char *short_commit_name(struct commit *commit)
+{
+	return find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+}
+
 static int get_message(struct commit *commit, struct commit_message *out)
 {
 	const char *abbrev, *subject;
 	int subject_len;
 
 	out->message = logmsg_reencode(commit, NULL, get_commit_output_encoding());
-	abbrev = find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV);
+	abbrev = short_commit_name(commit);
 
 	subject_len = find_commit_subject(out->message, &subject);
 
@@ -621,8 +626,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		error(opts->action == REPLAY_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
-		      find_unique_abbrev(commit->object.oid.hash, DEFAULT_ABBREV),
-		      msg.subject);
+		      short_commit_name(commit), msg.subject);
 		print_advice(res == 1, opts);
 		rerere(opts->allow_rerere_auto);
 		goto leave;
-- 
2.10.1.583.g721a9e0

[PATCH v5 08/27] sequencer: completely revamp the "todo" script parsing

[Johannes Schindelin]

When we came up with the "sequencer" idea, we really wanted to have
kind of a plumbing equivalent of the interactive rebase. Hence the
choice of words: the "todo" script, a "pick", etc.

However, when it came time to implement the entire shebang, somehow this
idea got lost and the sequencer was used as working horse for
cherry-pick and revert instead. So as not to interfere with the
interactive rebase, it even uses a separate directory to store its
state.

Furthermore, it also is stupidly strict about the "todo" script it
accepts: while it parses commands in a way that was *designed* to be
similar to the interactive rebase, it then goes on to *error out* if the
commands disagree with the overall action (cherry-pick or revert).

Finally, the sequencer code chose to deviate from the interactive rebase
code insofar that when it comes to writing the file with the remaining
commands, it *reformats* the "todo" script instead of just writing the
part of the parsed script that were not yet processed. This is not only
unnecessary churn, but might well lose information that is valuable to
the user (i.e. comments after the commands).

Let's just bite the bullet and rewrite the entire parser; the code now
becomes not only more elegant: it allows us to go on and teach the
sequencer how to parse *true* "todo" scripts as used by the interactive
rebase itself. In a way, the sequencer is about to grow up to do its
older brother's job. Better.

In particular, we choose to maintain the list of commands in an array
instead of a linked list: this is flexible enough to allow us later on to
even implement rebase -i's reordering of fixup!/squash! commits very
easily (and with a very nice speed bonus, at least on Windows).

While at it, do not stop at the first problem, but list *all* of the
problems. This will help the user when the sequencer will do `rebase
-i`'s work by allowing to address all issues in one go rather than going
back and forth until the todo list is valid.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 284 ++++++++++++++++++++++++++++++++++--------------------------
 1 file changed, 163 insertions(+), 121 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 499f5ee..145de78 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -470,7 +470,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
 		return 1;
 }
 
-static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
+enum todo_command {
+	TODO_PICK = 0,
+	TODO_REVERT
+};
+
+static const char *todo_command_strings[] = {
+	"pick",
+	"revert"
+};
+
+static const char *command_to_string(const enum todo_command command)
+{
+	if (command < ARRAY_SIZE(todo_command_strings))
+		return todo_command_strings[command];
+	die("Unknown command: %d", command);
+}
+
+
+static int do_pick_commit(enum todo_command command, struct commit *commit,
+		struct replay_opts *opts)
 {
 	unsigned char head[20];
 	struct commit *base, *next, *parent;
@@ -529,10 +548,11 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		return fast_forward_to(commit->object.oid.hash, head, unborn, opts);
 
 	if (parent && parse_commit(parent) < 0)
-		/* TRANSLATORS: The first %s will be "revert" or
-		   "cherry-pick", the second %s a SHA1 */
+		/* TRANSLATORS: The first %s will be a "todo" command like
+		   "revert" or "pick", the second %s a SHA1. */
 		return error(_("%s: cannot parse parent commit %s"),
-			action_name(opts), oid_to_hex(&parent->object.oid));
+			command_to_string(command),
+			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
 		return error(_("Cannot get commit message for %s"),
@@ -545,7 +565,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * reverse of it if we are revert.
 	 */
 
-	if (opts->action == REPLAY_REVERT) {
+	if (command == TODO_REVERT) {
 		base = commit;
 		base_label = msg.label;
 		next = parent;
@@ -586,7 +606,7 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 		}
 	}
 
-	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || opts->action == REPLAY_REVERT) {
+	if (!opts->strategy || !strcmp(opts->strategy, "recursive") || command == TODO_REVERT) {
 		res = do_recursive_merge(base, next, base_label, next_label,
 					 head, &msgbuf, opts);
 		if (res < 0)
@@ -613,17 +633,17 @@ static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
 	 * However, if the merge did not even start, then we don't want to
 	 * write it at all.
 	 */
-	if (opts->action == REPLAY_PICK && !opts->no_commit && (res == 0 || res == 1) &&
+	if (command == TODO_PICK && !opts->no_commit && (res == 0 || res == 1) &&
 	    update_ref(NULL, "CHERRY_PICK_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
-	if (opts->action == REPLAY_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
+	if (command == TODO_REVERT && ((opts->no_commit && res == 0) || res == 1) &&
 	    update_ref(NULL, "REVERT_HEAD", commit->object.oid.hash, NULL,
 		       REF_NODEREF, UPDATE_REFS_MSG_ON_ERR))
 		res = -1;
 
 	if (res) {
-		error(opts->action == REPLAY_REVERT
+		error(command == TODO_REVERT
 		      ? _("could not revert %s... %s")
 		      : _("could not apply %s... %s"),
 		      short_commit_name(commit), msg.subject);
@@ -684,116 +704,122 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	return 0;
 }
 
-static int format_todo(struct strbuf *buf, struct commit_list *todo_list,
-		struct replay_opts *opts)
+struct todo_item {
+	enum todo_command command;
+	struct commit *commit;
+	size_t offset_in_buf;
+};
+
+struct todo_list {
+	struct strbuf buf;
+	struct todo_item *items;
+	int nr, alloc, current;
+};
+
+#define TODO_LIST_INIT { STRBUF_INIT }
+
+static void todo_list_release(struct todo_list *todo_list)
 {
-	struct commit_list *cur = NULL;
-	const char *sha1_abbrev = NULL;
-	const char *action_str = opts->action == REPLAY_REVERT ? "revert" : "pick";
-	const char *subject;
-	int subject_len;
+	strbuf_release(&todo_list->buf);
+	free(todo_list->items);
+	todo_list->items = NULL;
+	todo_list->nr = todo_list->alloc = 0;
+}
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		const char *commit_buffer = get_commit_buffer(cur->item, NULL);
-		sha1_abbrev = find_unique_abbrev(cur->item->object.oid.hash, DEFAULT_ABBREV);
-		subject_len = find_commit_subject(commit_buffer, &subject);
-		strbuf_addf(buf, "%s %s %.*s\n", action_str, sha1_abbrev,
-			subject_len, subject);
-		unuse_commit_buffer(cur->item, commit_buffer);
-	}
-	return 0;
+static struct todo_item *append_new_todo(struct todo_list *todo_list)
+{
+	ALLOC_GROW(todo_list->items, todo_list->nr + 1, todo_list->alloc);
+	return todo_list->items + todo_list->nr++;
 }
 
-static struct commit *parse_insn_line(char *bol, char *eol, struct replay_opts *opts)
+static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 {
 	unsigned char commit_sha1[20];
-	enum replay_action action;
 	char *end_of_object_name;
-	int saved, status, padding;
-
-	if (starts_with(bol, "pick")) {
-		action = REPLAY_PICK;
-		bol += strlen("pick");
-	} else if (starts_with(bol, "revert")) {
-		action = REPLAY_REVERT;
-		bol += strlen("revert");
-	} else
-		return NULL;
+	int i, saved, status, padding;
+
+	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
+		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
+			item->command = i;
+			break;
+		}
+	if (i >= ARRAY_SIZE(todo_command_strings))
+		return -1;
 
 	/* Eat up extra spaces/ tabs before object name */
 	padding = strspn(bol, " \t");
 	if (!padding)
-		return NULL;
+		return -1;
 	bol += padding;
 
-	end_of_object_name = bol + strcspn(bol, " \t\n");
+	end_of_object_name = (char *) bol + strcspn(bol, " \t\n");
 	saved = *end_of_object_name;
 	*end_of_object_name = '\0';
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
-	/*
-	 * Verify that the action matches up with the one in
-	 * opts; we don't support arbitrary instructions
-	 */
-	if (action != opts->action) {
-		if (action == REPLAY_REVERT)
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot revert during another revert.")
-			    : _("Cannot revert during a cherry-pick."));
-		else
-		      error((opts->action == REPLAY_REVERT)
-			    ? _("Cannot cherry-pick during a revert.")
-			    : _("Cannot cherry-pick during another cherry-pick."));
-		return NULL;
-	}
-
 	if (status < 0)
-		return NULL;
+		return -1;
 
-	return lookup_commit_reference(commit_sha1);
+	item->commit = lookup_commit_reference(commit_sha1);
+	return !item->commit;
 }
 
-static int parse_insn_buffer(char *buf, struct commit_list **todo_list,
-			struct replay_opts *opts)
+static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 {
-	struct commit_list **next = todo_list;
-	struct commit *commit;
-	char *p = buf;
-	int i;
+	struct todo_item *item;
+	char *p = buf, *next_p;
+	int i, res = 0;
 
-	for (i = 1; *p; i++) {
+	for (i = 1; *p; i++, p = next_p) {
 		char *eol = strchrnul(p, '\n');
-		commit = parse_insn_line(p, eol, opts);
-		if (!commit)
-			return error(_("Could not parse line %d."), i);
-		next = commit_list_append(commit, next);
-		p = *eol ? eol + 1 : eol;
+
+		next_p = *eol ? eol + 1 /* skip LF */ : eol;
+
+		item = append_new_todo(todo_list);
+		item->offset_in_buf = p - todo_list->buf.buf;
+		if (parse_insn_line(item, p, eol)) {
+			res = error(_("Invalid line %d: %.*s"),
+				i, (int)(eol - p), p);
+			item->command = -1;
+		}
 	}
-	if (!*todo_list)
+	if (!todo_list->nr)
 		return error(_("No commits parsed."));
-	return 0;
+	return res;
 }
 
-static int read_populate_todo(struct commit_list **todo_list,
+static int read_populate_todo(struct todo_list *todo_list,
 			struct replay_opts *opts)
 {
 	const char *todo_file = get_todo_path(opts);
-	struct strbuf buf = STRBUF_INIT;
 	int fd, res;
 
+	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
 		return error_errno(_("Could not open %s"), todo_file);
-	if (strbuf_read(&buf, fd, 0) < 0) {
+	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		strbuf_release(&buf);
 		return error(_("Could not read %s."), todo_file);
 	}
 	close(fd);
 
-	res = parse_insn_buffer(buf.buf, todo_list, opts);
-	strbuf_release(&buf);
+	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
+	if (!res) {
+		enum todo_command valid =
+			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
+		int i;
+
+		for (i = 0; i < todo_list->nr; i++)
+			if (valid == todo_list->items[i].command)
+				continue;
+			else if (valid == TODO_PICK)
+				return error(_("Cannot cherry-pick during a revert."));
+			else
+				return error(_("Cannot revert during a cherry-pick."));
+	}
+
 	if (res)
 		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
@@ -860,18 +886,31 @@ static int read_populate_opts(struct replay_opts *opts)
 	return 0;
 }
 
-static int walk_revs_populate_todo(struct commit_list **todo_list,
+static int walk_revs_populate_todo(struct todo_list *todo_list,
 				struct replay_opts *opts)
 {
+	enum todo_command command = opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT;
+	const char *command_string = todo_command_strings[command];
 	struct commit *commit;
-	struct commit_list **next;
 
 	if (prepare_revs(opts))
 		return -1;
 
-	next = todo_list;
-	while ((commit = get_revision(opts->revs)))
-		next = commit_list_append(commit, next);
+	while ((commit = get_revision(opts->revs))) {
+		struct todo_item *item = append_new_todo(todo_list);
+		const char *commit_buffer = get_commit_buffer(commit, NULL);
+		const char *subject;
+		int subject_len;
+
+		item->command = command;
+		item->commit = commit;
+		item->offset_in_buf = todo_list->buf.len;
+		subject_len = find_commit_subject(commit_buffer, &subject);
+		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
+			short_commit_name(commit), subject_len, subject);
+		unuse_commit_buffer(commit, commit_buffer);
+	}
 	return 0;
 }
 
@@ -979,30 +1018,22 @@ static int sequencer_rollback(struct replay_opts *opts)
 	return -1;
 }
 
-static int save_todo(struct commit_list *todo_list, struct replay_opts *opts)
+static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 {
 	static struct lock_file todo_lock;
-	struct strbuf buf = STRBUF_INIT;
-	int fd;
+	const char *todo_path = get_todo_path(opts);
+	int next = todo_list->current, offset, fd;
 
-	fd = hold_lock_file_for_update(&todo_lock, git_path_todo_file(), 0);
+	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
-		return error_errno(_("Could not lock '%s'"),
-				   git_path_todo_file());
-	if (format_todo(&buf, todo_list, opts) < 0) {
-		strbuf_release(&buf);
-		return error(_("Could not format %s."), git_path_todo_file());
-	}
-	if (write_in_full(fd, buf.buf, buf.len) < 0) {
-		strbuf_release(&buf);
-		return error_errno(_("Could not write to %s"),
-				   git_path_todo_file());
-	}
-	if (commit_lock_file(&todo_lock) < 0) {
-		strbuf_release(&buf);
-		return error(_("Error wrapping up %s."), git_path_todo_file());
-	}
-	strbuf_release(&buf);
+		return error_errno(_("Could not lock '%s'"), todo_path);
+	offset = next < todo_list->nr ?
+		todo_list->items[next].offset_in_buf : todo_list->buf.len;
+	if (write_in_full(fd, todo_list->buf.buf + offset,
+			todo_list->buf.len - offset) < 0)
+		return error_errno(_("Could not write to '%s'"), todo_path);
+	if (commit_lock_file(&todo_lock) < 0)
+		return error(_("Error wrapping up %s."), todo_path);
 	return 0;
 }
 
@@ -1041,9 +1072,8 @@ static int save_opts(struct replay_opts *opts)
 	return res;
 }
 
-static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
+static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 {
-	struct commit_list *cur;
 	int res;
 
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
@@ -1053,10 +1083,12 @@ static int pick_commits(struct commit_list *todo_list, struct replay_opts *opts)
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	for (cur = todo_list; cur; cur = cur->next) {
-		if (save_todo(cur, opts))
+	while (todo_list->current < todo_list->nr) {
+		struct todo_item *item = todo_list->items + todo_list->current;
+		if (save_todo(todo_list, opts))
 			return -1;
-		res = do_pick_commit(cur->item, opts);
+		res = do_pick_commit(item->command, item->commit, opts);
+		todo_list->current++;
 		if (res)
 			return res;
 	}
@@ -1081,38 +1113,46 @@ static int continue_single_pick(void)
 
 static int sequencer_continue(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
+	int res;
 
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
-	if (read_populate_opts(opts) ||
-			read_populate_todo(&todo_list, opts))
+	if (read_populate_opts(opts))
 		return -1;
+	if ((res = read_populate_todo(&todo_list, opts)))
+		goto release_todo_list;
 
 	/* Verify that the conflict has been resolved */
 	if (file_exists(git_path_cherry_pick_head()) ||
 	    file_exists(git_path_revert_head())) {
-		int ret = continue_single_pick();
-		if (ret)
-			return ret;
+		res = continue_single_pick();
+		if (res)
+			goto release_todo_list;
 	}
-	if (index_differs_from("HEAD", 0))
-		return error_dirty_index(opts);
-	todo_list = todo_list->next;
-	return pick_commits(todo_list, opts);
+	if (index_differs_from("HEAD", 0)) {
+		res = error_dirty_index(opts);
+		goto release_todo_list;
+	}
+	todo_list.current++;
+	res = pick_commits(&todo_list, opts);
+release_todo_list:
+	todo_list_release(&todo_list);
+	return res;
 }
 
 static int single_pick(struct commit *cmit, struct replay_opts *opts)
 {
 	setenv(GIT_REFLOG_ACTION, action_name(opts), 0);
-	return do_pick_commit(cmit, opts);
+	return do_pick_commit(opts->action == REPLAY_PICK ?
+		TODO_PICK : TODO_REVERT, cmit, opts);
 }
 
 int sequencer_pick_revisions(struct replay_opts *opts)
 {
-	struct commit_list *todo_list = NULL;
+	struct todo_list todo_list = TODO_LIST_INIT;
 	unsigned char sha1[20];
-	int i;
+	int i, res;
 
 	if (opts->subcommand == REPLAY_NONE)
 		assert(opts->revs);
@@ -1187,7 +1227,9 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 		return -1;
 	if (save_opts(opts))
 		return -1;
-	return pick_commits(todo_list, opts);
+	res = pick_commits(&todo_list, opts);
+	todo_list_release(&todo_list);
+	return res;
 }
 
 void append_signoff(struct strbuf *msgbuf, int ignore_footer, unsigned flag)
-- 
2.10.1.583.g721a9e0

[PATCH v5 09/27] sequencer: strip CR from the todo script

[Johannes Schindelin]

It is not unheard of that editors on Windows write CR/LF even if the
file originally had only LF. This is particularly awkward for exec lines
of a rebase -i todo sheet. Take for example the insn "exec echo": The
shell script parser splits at the LF and leaves the CR attached to
"echo", which leads to the unknown command "echo\r".

Work around that by stripping CR when reading the todo commands, as we
already do for LF.

This happens to fix t9903.14 and .15 in MSYS1 environments (with the
rebase--helper patches based on this patch series): the todo script
constructed in such a setup contains CR/LF thanks to MSYS1 runtime's
cleverness.

Based on a report and a patch by Johannes Sixt.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 145de78..04fcfd8 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -776,6 +776,9 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 
 		next_p = *eol ? eol + 1 /* skip LF */ : eol;
 
+		if (p != eol && eol[-1] == '\r')
+			eol--; /* strip Carriage Return */
+
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
 		if (parse_insn_line(item, p, eol)) {
-- 
2.10.1.583.g721a9e0

[PATCH v5 10/27] sequencer: avoid completely different messages for different actions

[Johannes Schindelin]

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 04fcfd8..120a8ee 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -229,11 +229,8 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(action_name(opts));
 
-	/* Different translation strings for cherry-pick and revert */
-	if (opts->action == REPLAY_PICK)
-		error(_("Your local changes would be overwritten by cherry-pick."));
-	else
-		error(_("Your local changes would be overwritten by revert."));
+	error(_("Your local changes would be overwritten by %s."),
+		action_name(opts));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
-- 
2.10.1.583.g721a9e0

[PATCH v5 11/27] sequencer: get rid of the subcommand field

[Johannes Schindelin]

The subcommands are used exactly once, at the very beginning of
sequencer_pick_revisions(), to determine what to do. This is an
unnecessary level of indirection: we can simply call the correct
function to begin with. So let's do that.

While at it, ensure that the subcommands return an error code so that
they do not have to die() all over the place (bad practice for library
functions...).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/revert.c | 36 ++++++++++++++++--------------------
 sequencer.c      | 35 +++++++++++------------------------
 sequencer.h      | 13 ++++---------
 3 files changed, 31 insertions(+), 53 deletions(-)

diff --git a/builtin/revert.c b/builtin/revert.c
index ba5a88c..4ca5b51 100644
--- a/builtin/revert.c
+++ b/builtin/revert.c
@@ -71,7 +71,7 @@ static void verify_opt_compatible(const char *me, const char *base_opt, ...)
 		die(_("%s: %s cannot be used with %s"), me, this_opt, base_opt);
 }
 
-static void parse_args(int argc, const char **argv, struct replay_opts *opts)
+static int run_sequencer(int argc, const char **argv, struct replay_opts *opts)
 {
 	const char * const * usage_str = revert_or_cherry_pick_usage(opts);
 	const char *me = action_name(opts);
@@ -115,25 +115,15 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 	if (opts->keep_redundant_commits)
 		opts->allow_empty = 1;
 
-	/* Set the subcommand */
-	if (cmd == 'q')
-		opts->subcommand = REPLAY_REMOVE_STATE;
-	else if (cmd == 'c')
-		opts->subcommand = REPLAY_CONTINUE;
-	else if (cmd == 'a')
-		opts->subcommand = REPLAY_ROLLBACK;
-	else
-		opts->subcommand = REPLAY_NONE;
-
 	/* Check for incompatible command line arguments */
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		char *this_operation;
-		if (opts->subcommand == REPLAY_REMOVE_STATE)
+		if (cmd == 'q')
 			this_operation = "--quit";
-		else if (opts->subcommand == REPLAY_CONTINUE)
+		else if (cmd == 'c')
 			this_operation = "--continue";
 		else {
-			assert(opts->subcommand == REPLAY_ROLLBACK);
+			assert(cmd == 'a');
 			this_operation = "--abort";
 		}
 
@@ -156,7 +146,7 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 				"--edit", opts->edit,
 				NULL);
 
-	if (opts->subcommand != REPLAY_NONE) {
+	if (cmd) {
 		opts->revs = NULL;
 	} else {
 		struct setup_revision_opt s_r_opt;
@@ -178,6 +168,14 @@ static void parse_args(int argc, const char **argv, struct replay_opts *opts)
 	/* These option values will be free()d */
 	opts->gpg_sign = xstrdup_or_null(opts->gpg_sign);
 	opts->strategy = xstrdup_or_null(opts->strategy);
+
+	if (cmd == 'q')
+		return sequencer_remove_state(opts);
+	if (cmd == 'c')
+		return sequencer_continue(opts);
+	if (cmd == 'a')
+		return sequencer_rollback(opts);
+	return sequencer_pick_revisions(opts);
 }
 
 int cmd_revert(int argc, const char **argv, const char *prefix)
@@ -189,8 +187,7 @@ int cmd_revert(int argc, const char **argv, const char *prefix)
 		opts.edit = 1;
 	opts.action = REPLAY_REVERT;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("revert failed"));
 	return res;
@@ -203,8 +200,7 @@ int cmd_cherry_pick(int argc, const char **argv, const char *prefix)
 
 	opts.action = REPLAY_PICK;
 	git_config(git_default_config, NULL);
-	parse_args(argc, argv, &opts);
-	res = sequencer_pick_revisions(&opts);
+	res = run_sequencer(argc, argv, &opts);
 	if (res < 0)
 		die(_("cherry-pick failed"));
 	return res;
diff --git a/sequencer.c b/sequencer.c
index 120a8ee..9f22c5e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -119,7 +119,7 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
-static void remove_sequencer_state(const struct replay_opts *opts)
+int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
 	int i;
@@ -133,6 +133,8 @@ static void remove_sequencer_state(const struct replay_opts *opts)
 	strbuf_addf(&dir, "%s", get_dir(opts));
 	remove_dir_recursively(&dir, 0);
 	strbuf_release(&dir);
+
+	return 0;
 }
 
 static const char *action_name(const struct replay_opts *opts)
@@ -975,7 +977,7 @@ static int rollback_single_pick(void)
 	return reset_for_rollback(head_sha1);
 }
 
-static int sequencer_rollback(struct replay_opts *opts)
+int sequencer_rollback(struct replay_opts *opts)
 {
 	FILE *f;
 	unsigned char sha1[20];
@@ -1010,9 +1012,8 @@ static int sequencer_rollback(struct replay_opts *opts)
 	}
 	if (reset_for_rollback(sha1))
 		goto fail;
-	remove_sequencer_state(opts);
 	strbuf_release(&buf);
-	return 0;
+	return sequencer_remove_state(opts);
 fail:
 	strbuf_release(&buf);
 	return -1;
@@ -1097,8 +1098,7 @@ static int pick_commits(struct todo_list *todo_list, struct replay_opts *opts)
 	 * Sequence of picks finished successfully; cleanup by
 	 * removing the .git/sequencer directory
 	 */
-	remove_sequencer_state(opts);
-	return 0;
+	return sequencer_remove_state(opts);
 }
 
 static int continue_single_pick(void)
@@ -1111,11 +1111,14 @@ static int continue_single_pick(void)
 	return run_command_v_opt(argv, RUN_GIT_CMD);
 }
 
-static int sequencer_continue(struct replay_opts *opts)
+int sequencer_continue(struct replay_opts *opts)
 {
 	struct todo_list todo_list = TODO_LIST_INIT;
 	int res;
 
+	if (read_and_refresh_cache(opts))
+		return -1;
+
 	if (!file_exists(get_todo_path(opts)))
 		return continue_single_pick();
 	if (read_populate_opts(opts))
@@ -1154,26 +1157,10 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 	unsigned char sha1[20];
 	int i, res;
 
-	if (opts->subcommand == REPLAY_NONE)
-		assert(opts->revs);
-
+	assert(opts->revs);
 	if (read_and_refresh_cache(opts))
 		return -1;
 
-	/*
-	 * Decide what to do depending on the arguments; a fresh
-	 * cherry-pick should be handled differently from an existing
-	 * one that is being continued
-	 */
-	if (opts->subcommand == REPLAY_REMOVE_STATE) {
-		remove_sequencer_state(opts);
-		return 0;
-	}
-	if (opts->subcommand == REPLAY_ROLLBACK)
-		return sequencer_rollback(opts);
-	if (opts->subcommand == REPLAY_CONTINUE)
-		return sequencer_continue(opts);
-
 	for (i = 0; i < opts->revs->pending.nr; i++) {
 		unsigned char sha1[20];
 		const char *name = opts->revs->pending.objects[i].name;
diff --git a/sequencer.h b/sequencer.h
index 8453669..7a513c5 100644
--- a/sequencer.h
+++ b/sequencer.h
@@ -10,16 +10,8 @@ enum replay_action {
 	REPLAY_PICK
 };
 
-enum replay_subcommand {
-	REPLAY_NONE,
-	REPLAY_REMOVE_STATE,
-	REPLAY_CONTINUE,
-	REPLAY_ROLLBACK
-};
-
 struct replay_opts {
 	enum replay_action action;
-	enum replay_subcommand subcommand;
 
 	/* Boolean options */
 	int edit;
@@ -44,9 +36,12 @@ struct replay_opts {
 	/* Only used by REPLAY_NONE */
 	struct rev_info *revs;
 };
-#define REPLAY_OPTS_INIT { -1, -1 }
+#define REPLAY_OPTS_INIT { -1 }
 
 int sequencer_pick_revisions(struct replay_opts *opts);
+int sequencer_continue(struct replay_opts *opts);
+int sequencer_rollback(struct replay_opts *opts);
+int sequencer_remove_state(struct replay_opts *opts);
 
 extern const char sign_off_header[];
 
-- 
2.10.1.583.g721a9e0

[PATCH v5 12/27] sequencer: remember the onelines when parsing the todo file

[Johannes Schindelin]

The `git-rebase-todo` file contains a list of commands. Most of those
commands have the form

	<verb> <sha1> <oneline>

The <oneline> is displayed primarily for the user's convenience, as
rebase -i really interprets only the <verb> <sha1> part. However, there
are *some* places in interactive rebase where the <oneline> is used to
display messages, e.g. for reporting at which commit we stopped.

So let's just remember it when parsing the todo file; we keep a copy of
the entire todo file anyway (to write out the new `done` and
`git-rebase-todo` file just before processing each command), so all we
need to do is remember the begin offsets and lengths.

As we will have to parse and remember the command-line for `exec` commands
later, we do not call the field "oneline" but rather "arg" (and will reuse
that for exec's command-line).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 9f22c5e..3d1fdac 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -706,6 +706,8 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 struct todo_item {
 	enum todo_command command;
 	struct commit *commit;
+	const char *arg;
+	int arg_len;
 	size_t offset_in_buf;
 };
 
@@ -757,6 +759,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	status = get_sha1(bol, commit_sha1);
 	*end_of_object_name = saved;
 
+	item->arg = end_of_object_name + strspn(end_of_object_name, " \t");
+	item->arg_len = (int)(eol - item->arg);
+
 	if (status < 0)
 		return -1;
 
@@ -907,6 +912,8 @@ static int walk_revs_populate_todo(struct todo_list *todo_list,
 
 		item->command = command;
 		item->commit = commit;
+		item->arg = NULL;
+		item->arg_len = 0;
 		item->offset_in_buf = todo_list->buf.len;
 		subject_len = find_commit_subject(commit_buffer, &subject);
 		strbuf_addf(&todo_list->buf, "%s %s %.*s\n", command_string,
-- 
2.10.1.583.g721a9e0

[PATCH v5 13/27] sequencer: prepare for rebase -i's commit functionality

[Johannes Schindelin]

In interactive rebases, we commit a little bit differently than the
sequencer did so far: we heed the "author-script", the "message" and the
"amend" files in the .git/rebase-merge/ subdirectory.

Likewise, we may want to edit the commit message *even* when providing a
file containing the suggested commit message. Therefore we change the
code to not even provide a default message when we do not want any, and
to call the editor explicitly.

Also, in "interactive rebase" mode we want to skip reading the options
in the state directory of the cherry-pick/revert commands.

Finally, as interactive rebase's GPG settings are configured differently
from how cherry-pick (and therefore sequencer) handles them, we will
leave support for that to the next commit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 89 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 3d1fdac..6d5fe94 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -27,6 +27,19 @@ static GIT_PATH_FUNC(git_path_todo_file, "sequencer/todo")
 static GIT_PATH_FUNC(git_path_opts_file, "sequencer/opts")
 static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
 
+/*
+ * A script to set the GIT_AUTHOR_NAME, GIT_AUTHOR_EMAIL, and
+ * GIT_AUTHOR_DATE that will be used for the commit that is currently
+ * being rebased.
+ */
+static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+
+/* We will introduce the 'interactive rebase' mode later */
+static inline int is_rebase_i(const struct replay_opts *opts)
+{
+	return 0;
+}
+
 static const char *get_dir(const struct replay_opts *opts)
 {
 	return git_path_seq_dir();
@@ -370,19 +383,79 @@ static int is_index_unchanged(void)
 }
 
 /*
+ * Read the author-script file into an environment block, ready for use in
+ * run_command(), that can be free()d afterwards.
+ */
+static char **read_author_script(void)
+{
+	struct strbuf script = STRBUF_INIT;
+	int i, count = 0;
+	char *p, *p2, **env;
+	size_t env_size;
+
+	if (strbuf_read_file(&script, rebase_path_author_script(), 256) <= 0)
+		return NULL;
+
+	for (p = script.buf; *p; p++)
+		if (skip_prefix(p, "'\\\\''", (const char **)&p2))
+			strbuf_splice(&script, p - script.buf, p2 - p, "'", 1);
+		else if (*p == '\'')
+			strbuf_splice(&script, p-- - script.buf, 1, "", 0);
+		else if (*p == '\n') {
+			*p = '\0';
+			count++;
+		}
+
+	env_size = (count + 1) * sizeof(*env);
+	strbuf_grow(&script, env_size);
+	memmove(script.buf + env_size, script.buf, script.len);
+	p = script.buf + env_size;
+	env = (char **)strbuf_detach(&script, NULL);
+
+	for (i = 0; i < count; i++) {
+		env[i] = p;
+		p += strlen(p) + 1;
+	}
+	env[count] = NULL;
+
+	return env;
+}
+
+/*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
  * author date and name.
+ *
  * If we are revert, or if our cherry-pick results in a hand merge,
  * we had better say that the current user is responsible for that.
+ *
+ * An exception is when run_git_commit() is called during an
+ * interactive rebase: in that case, we will want to retain the
+ * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 			  int allow_empty)
 {
+	char **env = NULL;
 	struct argv_array array;
 	int rc;
 	const char *value;
 
+	if (is_rebase_i(opts)) {
+		env = read_author_script();
+		if (!env)
+			return error("You have staged changes in your working "
+				"tree. If these changes are meant to be\n"
+				"squashed into the previous commit, run:\n\n"
+				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"If they are meant to go into a new commit, "
+				"run:\n\n"
+				"  git commit $gpg_sign_opt_quoted\n\n"
+				"In both cases, once you're done, continue "
+				"with:\n\n"
+				"  git rebase --continue\n");
+	}
+
 	argv_array_init(&array);
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
@@ -391,14 +464,13 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
 		argv_array_push(&array, "-s");
-	if (!opts->edit) {
-		argv_array_push(&array, "-F");
-		argv_array_push(&array, defmsg);
-		if (!opts->signoff &&
-		    !opts->record_origin &&
-		    git_config_get_value("commit.cleanup", &value))
-			argv_array_push(&array, "--cleanup=verbatim");
-	}
+	if (defmsg)
+		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (opts->edit)
+		argv_array_push(&array, "-e");
+	else if (!opts->signoff && !opts->record_origin &&
+		 git_config_get_value("commit.cleanup", &value))
+		argv_array_push(&array, "--cleanup=verbatim");
 
 	if (allow_empty)
 		argv_array_push(&array, "--allow-empty");
@@ -406,8 +478,11 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	if (opts->allow_empty_message)
 		argv_array_push(&array, "--allow-empty-message");
 
-	rc = run_command_v_opt(array.argv, RUN_GIT_CMD);
+	rc = run_command_v_opt_cd_env(array.argv, RUN_GIT_CMD, NULL,
+			(const char *const *)env);
 	argv_array_clear(&array);
+	free(env);
+
 	return rc;
 }
 
@@ -657,7 +732,8 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		goto leave;
 	}
 	if (!opts->no_commit)
-		res = run_git_commit(git_path_merge_msg(), opts, allow);
+		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
+				     opts, allow);
 
 leave:
 	free_message(commit, &msg);
@@ -879,6 +955,9 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
+	if (is_rebase_i(opts))
+		return 0;
+
 	if (!file_exists(git_path_opts_file()))
 		return 0;
 	/*
-- 
2.10.1.583.g721a9e0

[PATCH v5 14/27] sequencer: introduce a helper to read files written by scripts

[Johannes Schindelin]

As we are slowly teaching the sequencer to perform the hard work for
the interactive rebase, we need to read files that were written by
shell scripts.

These files typically contain a single line and are invariably ended
by a line feed (and possibly a carriage return before that). Let's use
a helper to read such files and to remove the line ending.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 6d5fe94..282c4d1 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -234,6 +234,40 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
 	return 0;
 }
 
+/*
+ * Reads a file that was presumably written by a shell script, i.e. with an
+ * end-of-line marker that needs to be stripped.
+ *
+ * Note that only the last end-of-line marker is stripped, consistent with the
+ * behavior of "$(cat path)" in a shell script.
+ *
+ * Returns 1 if the file was read, 0 if it could not be read or does not exist.
+ */
+static int read_oneliner(struct strbuf *buf,
+	const char *path, int skip_if_empty)
+{
+	int orig_len = buf->len;
+
+	if (!file_exists(path))
+		return 0;
+
+	if (strbuf_read_file(buf, path, 0) < 0) {
+		warning_errno(_("could not read '%s'"), path);
+		return 0;
+	}
+
+	if (buf->len > orig_len && buf->buf[buf->len - 1] == '\n') {
+		if (--buf->len > orig_len && buf->buf[buf->len - 1] == '\r')
+			--buf->len;
+		buf->buf[buf->len] = '\0';
+	}
+
+	if (skip_if_empty && buf->len == orig_len)
+		return 0;
+
+	return 1;
+}
+
 static struct tree *empty_tree(void)
 {
 	return lookup_tree(EMPTY_TREE_SHA1_BIN);
-- 
2.10.1.583.g721a9e0

[PATCH v5 15/27] sequencer: allow editing the commit message on a case-by-case basis

[Johannes Schindelin]

In the upcoming commits, we will implement more and more of rebase -i's
functionality inside the sequencer. One particular feature of the
commands to come is that some of them allow editing the commit message
while others don't, i.e. we cannot define in the replay_opts whether the
commit message should be edited or not.

Let's add a new parameter to the run_git_commit() function. Previously,
it was the duty of the caller to ensure that the opts->edit setting
indicates whether to let the user edit the commit message or not,
indicating that it is an "all or nothing" setting, i.e. that the
sequencer wants to let the user edit *all* commit message, or none at
all. In the upcoming rebase -i mode, it will depend on the particular
command that is currently executed, though.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 48 ++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 40 insertions(+), 8 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 282c4d1..c0a0aa0 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -15,6 +15,7 @@
 #include "merge-recursive.h"
 #include "refs.h"
 #include "argv-array.h"
+#include "quote.h"
 
 #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION"
 
@@ -33,6 +34,11 @@ static GIT_PATH_FUNC(git_path_head_file, "sequencer/head")
  * being rebased.
  */
 static GIT_PATH_FUNC(rebase_path_author_script, "rebase-merge/author-script")
+/*
+ * The following files are written by git-rebase just after parsing the
+ * command-line (and are only consumed, not modified, by the sequencer).
+ */
+static GIT_PATH_FUNC(rebase_path_gpg_sign_opt, "rebase-merge/gpg_sign_opt")
 
 /* We will introduce the 'interactive rebase' mode later */
 static inline int is_rebase_i(const struct replay_opts *opts)
@@ -132,6 +138,16 @@ static int has_conforming_footer(struct strbuf *sb, struct strbuf *sob,
 	return 1;
 }
 
+static const char *gpg_sign_opt_quoted(struct replay_opts *opts)
+{
+	static struct strbuf buf = STRBUF_INIT;
+
+	strbuf_reset(&buf);
+	if (opts->gpg_sign)
+		sq_quotef(&buf, "-S%s", opts->gpg_sign);
+	return buf.buf;
+}
+
 int sequencer_remove_state(struct replay_opts *opts)
 {
 	struct strbuf dir = STRBUF_INIT;
@@ -468,7 +484,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty)
+			  int allow_empty, int edit)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -477,17 +493,20 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 
 	if (is_rebase_i(opts)) {
 		env = read_author_script();
-		if (!env)
+		if (!env) {
+			const char *gpg_opt = gpg_sign_opt_quoted(opts);
+
 			return error("You have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend $gpg_sign_opt_quoted\n\n"
+				"  git commit --amend %s\n\n"
 				"If they are meant to go into a new commit, "
 				"run:\n\n"
-				"  git commit $gpg_sign_opt_quoted\n\n"
+				"  git commit %s\n\n"
 				"In both cases, once you're done, continue "
 				"with:\n\n"
-				"  git rebase --continue\n");
+				"  git rebase --continue\n", gpg_opt, gpg_opt);
+		}
 	}
 
 	argv_array_init(&array);
@@ -500,7 +519,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
-	if (opts->edit)
+	if (edit)
 		argv_array_push(&array, "-e");
 	else if (!opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
@@ -767,7 +786,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow);
+				     opts, allow, opts->edit);
 
 leave:
 	free_message(commit, &msg);
@@ -989,8 +1008,21 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 
 static int read_populate_opts(struct replay_opts *opts)
 {
-	if (is_rebase_i(opts))
+	if (is_rebase_i(opts)) {
+		struct strbuf buf = STRBUF_INIT;
+
+		if (read_oneliner(&buf, rebase_path_gpg_sign_opt(), 1)) {
+			if (!starts_with(buf.buf, "-S"))
+				strbuf_reset(&buf);
+			else {
+				free(opts->gpg_sign);
+				opts->gpg_sign = xstrdup(buf.buf + 2);
+			}
+		}
+		strbuf_release(&buf);
+
 		return 0;
+	}
 
 	if (!file_exists(git_path_opts_file()))
 		return 0;
-- 
2.10.1.583.g721a9e0

[PATCH v5 16/27] sequencer: support amending commits

[Johannes Schindelin]

This teaches the run_git_commit() function to take an argument that will
allow us to implement "todo" commands that need to amend the commit
messages ("fixup", "squash" and "reword").

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index c0a0aa0..1ef50a0 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -484,7 +484,7 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit)
+			  int allow_empty, int edit, int amend)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -513,6 +513,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 	argv_array_push(&array, "commit");
 	argv_array_push(&array, "-n");
 
+	if (amend)
+		argv_array_push(&array, "--amend");
 	if (opts->gpg_sign)
 		argv_array_pushf(&array, "-S%s", opts->gpg_sign);
 	if (opts->signoff)
@@ -786,7 +788,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit);
+				     opts, allow, opts->edit, 0);
 
 leave:
 	free_message(commit, &msg);
-- 
2.10.1.583.g721a9e0

[PATCH v5 17/27] sequencer: support cleaning up commit messages

[Johannes Schindelin]

The run_git_commit() function already knows how to amend commits, and
with this new option, it can also clean up commit messages (i.e. strip
out commented lines). This is needed to implement rebase -i's 'fixup'
and 'squash' commands as sequencer commands.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 1ef50a0..8646ca5 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -484,7 +484,8 @@ static char **read_author_script(void)
  * author metadata.
  */
 static int run_git_commit(const char *defmsg, struct replay_opts *opts,
-			  int allow_empty, int edit, int amend)
+			  int allow_empty, int edit, int amend,
+			  int cleanup_commit_message)
 {
 	char **env = NULL;
 	struct argv_array array;
@@ -521,9 +522,12 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		argv_array_push(&array, "-s");
 	if (defmsg)
 		argv_array_pushl(&array, "-F", defmsg, NULL);
+	if (cleanup_commit_message)
+		argv_array_push(&array, "--cleanup=strip");
 	if (edit)
 		argv_array_push(&array, "-e");
-	else if (!opts->signoff && !opts->record_origin &&
+	else if (!cleanup_commit_message &&
+		 !opts->signoff && !opts->record_origin &&
 		 git_config_get_value("commit.cleanup", &value))
 		argv_array_push(&array, "--cleanup=verbatim");
 
@@ -788,7 +792,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 	}
 	if (!opts->no_commit)
 		res = run_git_commit(opts->edit ? NULL : git_path_merge_msg(),
-				     opts, allow, opts->edit, 0);
+				     opts, allow, opts->edit, 0, 0);
 
 leave:
 	free_message(commit, &msg);
-- 
2.10.1.583.g721a9e0

[PATCH v5 18/27] sequencer: left-trim lines read from the script

[Johannes Schindelin]

Interactive rebase's scripts may be indented; we need to handle this
case, too, now that we prepare the sequencer to process interactive
rebases.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/sequencer.c b/sequencer.c
index 8646ca5..d74fdce 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -874,6 +874,9 @@ static int parse_insn_line(struct todo_item *item, const char *bol, char *eol)
 	char *end_of_object_name;
 	int i, saved, status, padding;
 
+	/* left-trim */
+	bol += strspn(bol, " \t");
+
 	for (i = 0; i < ARRAY_SIZE(todo_command_strings); i++)
 		if (skip_prefix(bol, todo_command_strings[i], &bol)) {
 			item->command = i;
-- 
2.10.1.583.g721a9e0

[PATCH v5 19/27] sequencer: stop releasing the strbuf in write_message()

[Johannes Schindelin]

Nothing in the name "write_message()" suggests that the function
releases the strbuf passed to it. So let's release the strbuf in the
caller instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sequencer.c b/sequencer.c
index d74fdce..745c86f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -243,7 +243,6 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
 		return error_errno(_("Could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
 		return error_errno(_("Could not write to %s"), filename);
-	strbuf_release(msgbuf);
 	if (commit_lock_file(&msg_file) < 0)
 		return error(_("Error wrapping up %s."), filename);
 
@@ -759,6 +758,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		free_commit_list(common);
 		free_commit_list(remotes);
 	}
+	strbuf_release(&msgbuf);
 
 	/*
 	 * If the merge was clean or if it failed due to conflict, we write
-- 
2.10.1.583.g721a9e0

[PATCH v5 20/27] sequencer: roll back lock file if write_message() failed

[Johannes Schindelin]

There is no need to wait until the atexit() handler kicks in at the end.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 745c86f..9fced42 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -241,10 +241,14 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
 		return error_errno(_("Could not lock '%s'"), filename);
-	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
-		return error_errno(_("Could not write to %s"), filename);
-	if (commit_lock_file(&msg_file) < 0)
+	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0) {
+		rollback_lock_file(&msg_file);
+		return error_errno(_("Could not write to '%s'"), filename);
+	}
+	if (commit_lock_file(&msg_file) < 0) {
+		rollback_lock_file(&msg_file);
 		return error(_("Error wrapping up %s."), filename);
+	}
 
 	return 0;
 }
-- 
2.10.1.583.g721a9e0

[PATCH v5 21/27] sequencer: refactor write_message() to take a pointer/length

[Johannes Schindelin]

Previously, we required an strbuf. But that limits the use case too much.
In the upcoming patch series (for which the current patch series prepares
the sequencer), we will want to write content to a file for which we have
a pointer and a length, not an strbuf.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 9fced42..300952f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -234,14 +234,14 @@ static void print_advice(int show_hint, struct replay_opts *opts)
 	}
 }
 
-static int write_message(struct strbuf *msgbuf, const char *filename)
+static int write_message(const void *buf, size_t len, const char *filename)
 {
 	static struct lock_file msg_file;
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
 		return error_errno(_("Could not lock '%s'"), filename);
-	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0) {
+	if (write_in_full(msg_fd, buf, len) < 0) {
 		rollback_lock_file(&msg_file);
 		return error_errno(_("Could not write to '%s'"), filename);
 	}
@@ -747,12 +747,14 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 					 head, &msgbuf, opts);
 		if (res < 0)
 			return res;
-		res |= write_message(&msgbuf, git_path_merge_msg());
+		res |= write_message(msgbuf.buf, msgbuf.len,
+				     git_path_merge_msg());
 	} else {
 		struct commit_list *common = NULL;
 		struct commit_list *remotes = NULL;
 
-		res = write_message(&msgbuf, git_path_merge_msg());
+		res = write_message(msgbuf.buf, msgbuf.len,
+				    git_path_merge_msg());
 
 		commit_list_insert(base, &common);
 		commit_list_insert(next, &remotes);
-- 
2.10.1.583.g721a9e0

[PATCH v5 22/27] sequencer: teach write_message() to append an optional LF

[Johannes Schindelin]

This commit prepares for future callers that will have a pointer/length
to some text to be written that lacks an LF, yet an LF is desired.
Instead of requiring the caller to append an LF to the buffer (and
potentially allocate memory to do so), the write_message() function
learns to append an LF at the end of the file.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 300952f..000ce3e 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -234,7 +234,8 @@ static void print_advice(int show_hint, struct replay_opts *opts)
 	}
 }
 
-static int write_message(const void *buf, size_t len, const char *filename)
+static int write_message(const void *buf, size_t len, const char *filename,
+			 int append_eol)
 {
 	static struct lock_file msg_file;
 
@@ -245,6 +246,10 @@ static int write_message(const void *buf, size_t len, const char *filename)
 		rollback_lock_file(&msg_file);
 		return error_errno(_("Could not write to '%s'"), filename);
 	}
+	if (append_eol && write(msg_fd, "\n", 1) < 0) {
+		rollback_lock_file(&msg_file);
+		return error_errno(_("Could not write eol to '%s"), filename);
+	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
 		return error(_("Error wrapping up %s."), filename);
@@ -748,13 +753,13 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		if (res < 0)
 			return res;
 		res |= write_message(msgbuf.buf, msgbuf.len,
-				     git_path_merge_msg());
+				     git_path_merge_msg(), 0);
 	} else {
 		struct commit_list *common = NULL;
 		struct commit_list *remotes = NULL;
 
 		res = write_message(msgbuf.buf, msgbuf.len,
-				    git_path_merge_msg());
+				    git_path_merge_msg(), 0);
 
 		commit_list_insert(base, &common);
 		commit_list_insert(next, &remotes);
-- 
2.10.1.583.g721a9e0

[PATCH v5 23/27] sequencer: remove overzealous assumption in rebase -i mode

[Johannes Schindelin]

The sequencer was introduced to make the cherry-pick and revert
functionality available as library function, with the original idea
being to extend the sequencer to also implement the rebase -i
functionality.

The test to ensure that all of the commands in the script are identical
to the overall operation does not mesh well with that.

Therefore let's disable the test in rebase -i mode.

While at it, error out early if the "instruction sheet" (i.e. the todo
script) could not be parsed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 000ce3e..bd11db4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -962,7 +962,10 @@ static int read_populate_todo(struct todo_list *todo_list,
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
-	if (!res) {
+	if (res)
+		return error(_("Unusable instruction sheet: %s"), todo_file);
+
+	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
 			opts->action == REPLAY_PICK ? TODO_PICK : TODO_REVERT;
 		int i;
@@ -976,8 +979,6 @@ static int read_populate_todo(struct todo_list *todo_list,
 				return error(_("Cannot revert during a cherry-pick."));
 	}
 
-	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
 	return 0;
 }
 
-- 
2.10.1.583.g721a9e0

[PATCH v5 24/27] sequencer: mark action_name() for translation

[Johannes Schindelin]

The definition of this function goes back all the way to 043a449
(sequencer: factor code out of revert builtin, 2012-01-11), long before a
serious effort was made to translate all the error messages.

It is slightly out of the context of the current patch series (whose
purpose it is to re-implement the performance critical parts of the
interactive rebase in C) to make the error messages in the sequencer
translatable, but what the heck. We'll just do it while we're looking at
this part of the code.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index bd11db4..ff76b6f 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -168,7 +168,7 @@ int sequencer_remove_state(struct replay_opts *opts)
 
 static const char *action_name(const struct replay_opts *opts)
 {
-	return opts->action == REPLAY_REVERT ? "revert" : "cherry-pick";
+	return opts->action == REPLAY_REVERT ? N_("revert") : N_("cherry-pick");
 }
 
 struct commit_message {
@@ -300,10 +300,10 @@ static struct tree *empty_tree(void)
 static int error_dirty_index(struct replay_opts *opts)
 {
 	if (read_cache_unmerged())
-		return error_resolve_conflict(action_name(opts));
+		return error_resolve_conflict(_(action_name(opts)));
 
 	error(_("Your local changes would be overwritten by %s."),
-		action_name(opts));
+		_(action_name(opts)));
 
 	if (advice_commit_before_merge)
 		advise(_("Commit your changes or stash them to proceed."));
@@ -321,7 +321,7 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	if (checkout_fast_forward(from, to, 1))
 		return -1; /* the callee should have complained already */
 
-	strbuf_addf(&sb, _("%s: fast-forward"), action_name(opts));
+	strbuf_addf(&sb, _("%s: fast-forward"), _(action_name(opts)));
 
 	transaction = ref_transaction_begin(&err);
 	if (!transaction ||
@@ -397,7 +397,7 @@ static int do_recursive_merge(struct commit *base, struct commit *next,
 	    write_locked_index(&the_index, &index_lock, COMMIT_LOCK))
 		/* TRANSLATORS: %s will be "revert" or "cherry-pick" */
 		return error(_("%s: Unable to write new index file"),
-			action_name(opts));
+			_(action_name(opts)));
 	rollback_lock_file(&index_lock);
 
 	if (opts->signoff)
@@ -835,14 +835,14 @@ static int read_and_refresh_cache(struct replay_opts *opts)
 	if (read_index_preload(&the_index, NULL) < 0) {
 		rollback_lock_file(&index_lock);
 		return error(_("git %s: failed to read the index"),
-			action_name(opts));
+			_(action_name(opts)));
 	}
 	refresh_index(&the_index, REFRESH_QUIET|REFRESH_UNMERGED, NULL, NULL, NULL);
 	if (the_index.cache_changed && index_fd >= 0) {
 		if (write_locked_index(&the_index, &index_lock, COMMIT_LOCK)) {
 			rollback_lock_file(&index_lock);
 			return error(_("git %s: failed to refresh the index"),
-				action_name(opts));
+				_(action_name(opts)));
 		}
 	}
 	rollback_lock_file(&index_lock);
-- 
2.10.1.583.g721a9e0

[PATCH v5 25/27] sequencer: quote filenames in error messages

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3621 bytes --]

This makes the code consistent by fixing quite a couple of error messages.

Suggested by Jakub Narębski.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index ff76b6f..340d0ed 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -252,7 +252,7 @@ static int write_message(const void *buf, size_t len, const char *filename,
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
-		return error(_("Error wrapping up %s."), filename);
+		return error(_("Error wrapping up '%s'."), filename);
 	}
 
 	return 0;
@@ -954,16 +954,16 @@ static int read_populate_todo(struct todo_list *todo_list,
 	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open %s"), todo_file);
+		return error_errno(_("Could not open '%s'"), todo_file);
 	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		return error(_("Could not read %s."), todo_file);
+		return error(_("Could not read '%s'."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
-		return error(_("Unusable instruction sheet: %s"), todo_file);
+		return error(_("Unusable instruction sheet: '%s'"), todo_file);
 
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
@@ -1054,7 +1054,7 @@ static int read_populate_opts(struct replay_opts *opts)
 	 * are pretty certain that it is syntactically correct.
 	 */
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
-		return error(_("Malformed options sheet: %s"),
+		return error(_("Malformed options sheet: '%s'"),
 			git_path_opts_file());
 	return 0;
 }
@@ -1097,7 +1097,7 @@ static int create_seq_dir(void)
 		return -1;
 	}
 	else if (mkdir(git_path_seq_dir(), 0777) < 0)
-		return error_errno(_("Could not create sequencer directory %s"),
+		return error_errno(_("Could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
 }
@@ -1116,12 +1116,12 @@ static int save_head(const char *head)
 	strbuf_addf(&buf, "%s\n", head);
 	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not write to %s"),
+		return error_errno(_("Could not write to '%s'"),
 				   git_path_head_file());
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-		return error(_("Error wrapping up %s."), git_path_head_file());
+		return error(_("Error wrapping up '%s'."), git_path_head_file());
 	}
 	return 0;
 }
@@ -1166,9 +1166,9 @@ int sequencer_rollback(struct replay_opts *opts)
 		return rollback_single_pick();
 	}
 	if (!f)
-		return error_errno(_("cannot open %s"), git_path_head_file());
+		return error_errno(_("cannot open '%s'"), git_path_head_file());
 	if (strbuf_getline_lf(&buf, f)) {
-		error(_("cannot read %s: %s"), git_path_head_file(),
+		error(_("cannot read '%s': %s"), git_path_head_file(),
 		      ferror(f) ?  strerror(errno) : _("unexpected end of file"));
 		fclose(f);
 		goto fail;
@@ -1207,7 +1207,7 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 			todo_list->buf.len - offset) < 0)
 		return error_errno(_("Could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
-		return error(_("Error wrapping up %s."), todo_path);
+		return error(_("Error wrapping up '%s'."), todo_path);
 	return 0;
 }
 
-- 
2.10.1.583.g721a9e0

[PATCH v5 26/27] sequencer: start error messages consistently with lower case

[Johannes Schindelin]

Quite a few error messages touched by this developer during the work to
speed up rebase -i started with an upper case letter, violating our
current conventions. Instead of sneaking in this fix (and forgetting
quite a few error messages), let's just have one wholesale patch fixing
all of the error messages in the sequencer.

While at it, the funny "error: Error wrapping up..." was changed to a
less funny, but more helpful, "error: failed to finalize...".

Pointed out by Junio Hamano.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c                   | 68 +++++++++++++++++++++----------------------
 t/t3501-revert-cherry-pick.sh |  2 +-
 2 files changed, 35 insertions(+), 35 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 340d0ed..4c10c93 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -241,18 +241,18 @@ static int write_message(const void *buf, size_t len, const char *filename,
 
 	int msg_fd = hold_lock_file_for_update(&msg_file, filename, 0);
 	if (msg_fd < 0)
-		return error_errno(_("Could not lock '%s'"), filename);
+		return error_errno(_("could not lock '%s'"), filename);
 	if (write_in_full(msg_fd, buf, len) < 0) {
 		rollback_lock_file(&msg_file);
-		return error_errno(_("Could not write to '%s'"), filename);
+		return error_errno(_("could not write to '%s'"), filename);
 	}
 	if (append_eol && write(msg_fd, "\n", 1) < 0) {
 		rollback_lock_file(&msg_file);
-		return error_errno(_("Could not write eol to '%s"), filename);
+		return error_errno(_("could not write eol to '%s"), filename);
 	}
 	if (commit_lock_file(&msg_file) < 0) {
 		rollback_lock_file(&msg_file);
-		return error(_("Error wrapping up '%s'."), filename);
+		return error(_("failed to finalize '%s'."), filename);
 	}
 
 	return 0;
@@ -302,11 +302,11 @@ static int error_dirty_index(struct replay_opts *opts)
 	if (read_cache_unmerged())
 		return error_resolve_conflict(_(action_name(opts)));
 
-	error(_("Your local changes would be overwritten by %s."),
+	error(_("your local changes would be overwritten by %s."),
 		_(action_name(opts)));
 
 	if (advice_commit_before_merge)
-		advise(_("Commit your changes or stash them to proceed."));
+		advise(_("commit your changes or stash them to proceed."));
 	return -1;
 }
 
@@ -415,7 +415,7 @@ static int is_index_unchanged(void)
 	struct commit *head_commit;
 
 	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
-		return error(_("Could not resolve HEAD commit\n"));
+		return error(_("could not resolve HEAD commit\n"));
 
 	head_commit = lookup_commit(head_sha1);
 
@@ -435,7 +435,7 @@ static int is_index_unchanged(void)
 
 	if (!cache_tree_fully_valid(active_cache_tree))
 		if (cache_tree_update(&the_index, 0))
-			return error(_("Unable to update cache tree\n"));
+			return error(_("unable to update cache tree\n"));
 
 	return !hashcmp(active_cache_tree->sha1, head_commit->tree->object.oid.hash);
 }
@@ -505,7 +505,7 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error("You have staged changes in your working "
+			return error("you have staged changes in your working "
 				"tree. If these changes are meant to be\n"
 				"squashed into the previous commit, run:\n\n"
 				"  git commit --amend %s\n\n"
@@ -558,12 +558,12 @@ static int is_original_commit_empty(struct commit *commit)
 	const unsigned char *ptree_sha1;
 
 	if (parse_commit(commit))
-		return error(_("Could not parse commit %s\n"),
+		return error(_("could not parse commit %s\n"),
 			     oid_to_hex(&commit->object.oid));
 	if (commit->parents) {
 		struct commit *parent = commit->parents->item;
 		if (parse_commit(parent))
-			return error(_("Could not parse parent commit %s\n"),
+			return error(_("could not parse parent commit %s\n"),
 				oid_to_hex(&parent->object.oid));
 		ptree_sha1 = parent->tree->object.oid.hash;
 	} else {
@@ -647,7 +647,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		 * to work on.
 		 */
 		if (write_cache_as_tree(head, 0, NULL))
-			return error(_("Your index file is unmerged."));
+			return error(_("your index file is unmerged."));
 	} else {
 		unborn = get_sha1("HEAD", head);
 		if (unborn)
@@ -666,7 +666,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		struct commit_list *p;
 
 		if (!opts->mainline)
-			return error(_("Commit %s is a merge but no -m option was given."),
+			return error(_("commit %s is a merge but no -m option was given."),
 				oid_to_hex(&commit->object.oid));
 
 		for (cnt = 1, p = commit->parents;
@@ -674,11 +674,11 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 		     cnt++)
 			p = p->next;
 		if (cnt != opts->mainline || !p)
-			return error(_("Commit %s does not have parent %d"),
+			return error(_("commit %s does not have parent %d"),
 				oid_to_hex(&commit->object.oid), opts->mainline);
 		parent = p->item;
 	} else if (0 < opts->mainline)
-		return error(_("Mainline was specified but commit %s is not a merge."),
+		return error(_("mainline was specified but commit %s is not a merge."),
 			oid_to_hex(&commit->object.oid));
 	else
 		parent = commit->parents->item;
@@ -696,7 +696,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
 			oid_to_hex(&parent->object.oid));
 
 	if (get_message(commit, &msg) != 0)
-		return error(_("Cannot get commit message for %s"),
+		return error(_("cannot get commit message for %s"),
 			oid_to_hex(&commit->object.oid));
 
 	/*
@@ -935,13 +935,13 @@ static int parse_insn_buffer(char *buf, struct todo_list *todo_list)
 		item = append_new_todo(todo_list);
 		item->offset_in_buf = p - todo_list->buf.buf;
 		if (parse_insn_line(item, p, eol)) {
-			res = error(_("Invalid line %d: %.*s"),
+			res = error(_("invalid line %d: %.*s"),
 				i, (int)(eol - p), p);
 			item->command = -1;
 		}
 	}
 	if (!todo_list->nr)
-		return error(_("No commits parsed."));
+		return error(_("no commits parsed."));
 	return res;
 }
 
@@ -954,16 +954,16 @@ static int read_populate_todo(struct todo_list *todo_list,
 	strbuf_reset(&todo_list->buf);
 	fd = open(todo_file, O_RDONLY);
 	if (fd < 0)
-		return error_errno(_("Could not open '%s'"), todo_file);
+		return error_errno(_("could not open '%s'"), todo_file);
 	if (strbuf_read(&todo_list->buf, fd, 0) < 0) {
 		close(fd);
-		return error(_("Could not read '%s'."), todo_file);
+		return error(_("could not read '%s'."), todo_file);
 	}
 	close(fd);
 
 	res = parse_insn_buffer(todo_list->buf.buf, todo_list);
 	if (res)
-		return error(_("Unusable instruction sheet: '%s'"), todo_file);
+		return error(_("unusable instruction sheet: '%s'"), todo_file);
 
 	if (!is_rebase_i(opts)) {
 		enum todo_command valid =
@@ -974,9 +974,9 @@ static int read_populate_todo(struct todo_list *todo_list,
 			if (valid == todo_list->items[i].command)
 				continue;
 			else if (valid == TODO_PICK)
-				return error(_("Cannot cherry-pick during a revert."));
+				return error(_("cannot cherry-pick during a revert."));
 			else
-				return error(_("Cannot revert during a cherry-pick."));
+				return error(_("cannot revert during a cherry-pick."));
 	}
 
 	return 0;
@@ -1019,10 +1019,10 @@ static int populate_opts_cb(const char *key, const char *value, void *data)
 		ALLOC_GROW(opts->xopts, opts->xopts_nr + 1, opts->xopts_alloc);
 		opts->xopts[opts->xopts_nr++] = xstrdup(value);
 	} else
-		return error(_("Invalid key: %s"), key);
+		return error(_("invalid key: %s"), key);
 
 	if (!error_flag)
-		return error(_("Invalid value for %s: %s"), key, value);
+		return error(_("invalid value for %s: %s"), key, value);
 
 	return 0;
 }
@@ -1054,7 +1054,7 @@ static int read_populate_opts(struct replay_opts *opts)
 	 * are pretty certain that it is syntactically correct.
 	 */
 	if (git_config_from_file(populate_opts_cb, git_path_opts_file(), opts) < 0)
-		return error(_("Malformed options sheet: '%s'"),
+		return error(_("malformed options sheet: '%s'"),
 			git_path_opts_file());
 	return 0;
 }
@@ -1097,7 +1097,7 @@ static int create_seq_dir(void)
 		return -1;
 	}
 	else if (mkdir(git_path_seq_dir(), 0777) < 0)
-		return error_errno(_("Could not create sequencer directory '%s'"),
+		return error_errno(_("could not create sequencer directory '%s'"),
 				   git_path_seq_dir());
 	return 0;
 }
@@ -1111,17 +1111,17 @@ static int save_head(const char *head)
 	fd = hold_lock_file_for_update(&head_lock, git_path_head_file(), 0);
 	if (fd < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not lock HEAD"));
+		return error_errno(_("could not lock HEAD"));
 	}
 	strbuf_addf(&buf, "%s\n", head);
 	if (write_in_full(fd, buf.buf, buf.len) < 0) {
 		rollback_lock_file(&head_lock);
-		return error_errno(_("Could not write to '%s'"),
+		return error_errno(_("could not write to '%s'"),
 				   git_path_head_file());
 	}
 	if (commit_lock_file(&head_lock) < 0) {
 		rollback_lock_file(&head_lock);
-		return error(_("Error wrapping up '%s'."), git_path_head_file());
+		return error(_("failed to finalize '%s'."), git_path_head_file());
 	}
 	return 0;
 }
@@ -1200,14 +1200,14 @@ static int save_todo(struct todo_list *todo_list, struct replay_opts *opts)
 
 	fd = hold_lock_file_for_update(&todo_lock, todo_path, 0);
 	if (fd < 0)
-		return error_errno(_("Could not lock '%s'"), todo_path);
+		return error_errno(_("could not lock '%s'"), todo_path);
 	offset = next < todo_list->nr ?
 		todo_list->items[next].offset_in_buf : todo_list->buf.len;
 	if (write_in_full(fd, todo_list->buf.buf + offset,
 			todo_list->buf.len - offset) < 0)
-		return error_errno(_("Could not write to '%s'"), todo_path);
+		return error_errno(_("could not write to '%s'"), todo_path);
 	if (commit_lock_file(&todo_lock) < 0)
-		return error(_("Error wrapping up '%s'."), todo_path);
+		return error(_("failed to finalize '%s'."), todo_path);
 	return 0;
 }
 
@@ -1382,7 +1382,7 @@ int sequencer_pick_revisions(struct replay_opts *opts)
 			create_seq_dir() < 0)
 		return -1;
 	if (get_sha1("HEAD", sha1) && (opts->action == REPLAY_REVERT))
-		return error(_("Can't revert as initial commit"));
+		return error(_("can't revert as initial commit"));
 	if (save_head(sha1_to_hex(sha1)))
 		return -1;
 	if (save_opts(opts))
diff --git a/t/t3501-revert-cherry-pick.sh b/t/t3501-revert-cherry-pick.sh
index 51f3bbb..394f000 100755
--- a/t/t3501-revert-cherry-pick.sh
+++ b/t/t3501-revert-cherry-pick.sh
@@ -96,7 +96,7 @@ test_expect_success 'revert forbidden on dirty working tree' '
 	echo content >extra_file &&
 	git add extra_file &&
 	test_must_fail git revert HEAD 2>errors &&
-	test_i18ngrep "Your local changes would be overwritten by " errors
+	test_i18ngrep "your local changes would be overwritten by " errors
 
 '
 
-- 
2.10.1.583.g721a9e0

[PATCH v5 27/27] sequencer: mark all error messages for translation

[Johannes Schindelin]

There was actually only one error message that was not yet marked for
translation.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 sequencer.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/sequencer.c b/sequencer.c
index 4c10c93..a61fe76 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -479,6 +479,20 @@ static char **read_author_script(void)
 	return env;
 }
 
+static const char staged_changes_advice[] =
+N_("you have staged changes in your working tree\n"
+"If these changes are meant to be squashed into the previous commit, run:\n"
+"\n"
+"  git commit --amend %s\n"
+"\n"
+"If they are meant to go into a new commit, run:\n"
+"\n"
+"  git commit %s\n"
+"\n"
+"In both cases, once you're done, continue with:\n"
+"\n"
+"  git rebase --continue\n");
+
 /*
  * If we are cherry-pick, and if the merge did not result in
  * hand-editing, we will hit this commit and inherit the original
@@ -505,16 +519,8 @@ static int run_git_commit(const char *defmsg, struct replay_opts *opts,
 		if (!env) {
 			const char *gpg_opt = gpg_sign_opt_quoted(opts);
 
-			return error("you have staged changes in your working "
-				"tree. If these changes are meant to be\n"
-				"squashed into the previous commit, run:\n\n"
-				"  git commit --amend %s\n\n"
-				"If they are meant to go into a new commit, "
-				"run:\n\n"
-				"  git commit %s\n\n"
-				"In both cases, once you're done, continue "
-				"with:\n\n"
-				"  git rebase --continue\n", gpg_opt, gpg_opt);
+			return error(_(staged_changes_advice),
+				     gpg_opt, gpg_opt);
 		}
 	}
 
-- 
2.10.1.583.g721a9e0

Re: [PATCH v4 20/25] sequencer: refactor write_message()

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> Ah, make that four steps.  The final one is:
>> 
>>     - add append_eol parameter that nobody uses at this step in the
>>       series.
>>
>> This is a new feature to the helper.  While it is OK to have it as a
>> preparatory step in this series, it is easier to understand if it
>> kept as a separate step.  It is even more preferrable if it is made
>> as a preparatory step in a series that adds a caller that passes
>> true to append_eol to this helper...
>
> Done,
> Dscho

Hmm, what has been done exactly?  I still see append_eol in v5 where
nobody uses it yet.  Confused...

Re: [PATCH v5 19/27] sequencer: stop releasing the strbuf in write_message()

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Nothing in the name "write_message()" suggests that the function
> releases the strbuf passed to it. So let's release the strbuf in the
> caller instead.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---

I agree that it makes quite a lot of sense from the point of view of
"taste in the API design".

>  sequencer.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/sequencer.c b/sequencer.c
> index d74fdce..745c86f 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -243,7 +243,6 @@ static int write_message(struct strbuf *msgbuf, const char *filename)
>  		return error_errno(_("Could not lock '%s'"), filename);
>  	if (write_in_full(msg_fd, msgbuf->buf, msgbuf->len) < 0)
>  		return error_errno(_("Could not write to %s"), filename);
> -	strbuf_release(msgbuf);
>  	if (commit_lock_file(&msg_file) < 0)
>  		return error(_("Error wrapping up %s."), filename);
>  
> @@ -759,6 +758,7 @@ static int do_pick_commit(enum todo_command command, struct commit *commit,
>  		free_commit_list(common);
>  		free_commit_list(remotes);
>  	}
> +	strbuf_release(&msgbuf);
>  
>  	/*
>  	 * If the merge was clean or if it failed due to conflict, we write

Re: [PATCH v5 22/27] sequencer: teach write_message() to append an optional LF

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This commit prepares for future callers that will have a pointer/length
> to some text to be written that lacks an LF, yet an LF is desired.
> Instead of requiring the caller to append an LF to the buffer (and
> potentially allocate memory to do so), the write_message() function
> learns to append an LF at the end of the file.

As no existing callers need this, it probably is better left out and
added to the series that actually needs the new feature as a
preparatory step.

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> Changes vs v4:

I still do not understand (note that I am not saying "I do not
accept"--acceptance or rejection happens after an understandable
explanation is given, and "do not understand" means no such
explanation has been given yet) your justification behind adding a
technical debt to reimplement the author-script parser and not
sharing it with "git am" in 13/27.

As I pointed out, I think 22/27 is out of place in this series.

But other than these two points, the changes since v4 look minimum,
and they all looked good to me.

Thanks.

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Junio C Hamano]

Johannes Schindelin <johannes.schindelin@gmx.de> writes:

> This patch series marks the '4' in the countdown to speed up rebase -i
> by implementing large parts in C (read: there will be three more patch
> series after that before the full benefit hits git.git: sequencer-i,
> rebase--helper and rebase-i-extra).
> ...
> It would be *really* nice if we could get this patch series at least into
> `next` soon, as it gets late and later for the rest of the patches to make
> it into `master` in time for v2.11 (and it is not for lack of trying on my
> end...).

This "countdown 4" step can affect cherry-pick and revert, even
though we were careful to review changes to the sequencer.c code.  I
prefer to cook it in 'next' sufficiently long to ensure that we hear
feedbacks from non-Windows users if there is any unexpected breakage.

There isn't enough time to include this topic in the upcoming
release within the current https://tinyurl.com/gitCal calendar,
however, which places the final on Nov 11th.

I am wondering if it makes sense to delay 2.11 by moving the final
by 4 weeks to Dec 9th.

Thoughts?

Speaking of what to and not to include in the upcoming release, we
do want to include Stefan's off-by-one fix to the submodule-helper,
but that is blocked on Windows end due to the test.  I think
everybody agreed that a longer time "right thing to do" fix is to
address the "when base is /path/to/dir/., where is ../sub relative
to it?" issue, but if we are to do so, it would need a longer
gestation period once it hits 'next', as it can affect the current
users and we may even need B/C notes in the release notes for the
change.  Giving ourselves a few more weeks of breathing room would
help us to make sure the fix to relative URL issue is sound, too.

As to "countdown 3" and below steps, I am guessing that some of them
can start cooking in 'next' before 2.11, but even with lengthened
schedule, it is likely that they need to cook there beyond the end
of this cycle, unless they are truly trivial changes that do not
even need any reviews.

Thanks.

Re: [PATCH v4 20/25] sequencer: refactor write_message()

[Johannes Schindelin]

Hi Junio,

On Fri, 21 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> Ah, make that four steps.  The final one is:
> >> 
> >>     - add append_eol parameter that nobody uses at this step in the
> >>       series.
> >>
> >> This is a new feature to the helper.  While it is OK to have it as a
> >> preparatory step in this series, it is easier to understand if it
> >> kept as a separate step.  It is even more preferrable if it is made
> >> as a preparatory step in a series that adds a caller that passes
> >> true to append_eol to this helper...
> >
> > Done,
> > Dscho
> 
> Hmm, what has been done exactly?  I still see append_eol in v5 where
> nobody uses it yet.  Confused...

Your bullet point suggests that this change should be a separate patch. I
did that.

And since this patch series' purpose is to prepare the sequencer for the
upcoming series that teaches it to understand git-rebase-todo scripts,
this patch falls fair and square into the preparatory phase.

Ciao,
Dscho

Re: [PATCH v5 22/27] sequencer: teach write_message() to append an optional LF

[Johannes Schindelin]

Hi Junio,

On Fri, 21 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > This commit prepares for future callers that will have a pointer/length
> > to some text to be written that lacks an LF, yet an LF is desired.
> > Instead of requiring the caller to append an LF to the buffer (and
> > potentially allocate memory to do so), the write_message() function
> > learns to append an LF at the end of the file.
> 
> As no existing callers need this, it probably is better left out and
> added to the series that actually needs the new feature as a
> preparatory step.

Apart from this patch series being semantically the right place
("prepare-sequencer"), there is also the following consideration:
The next patch series is already quite long. Taking this current patch
series into account, which started out as a 22-patch series and needed to
bloat by 25% through four subsequent iterations, it is probably not a wise
idea to move this patch to a patch series that already weighs 34 patches.

So I respectfully, and forcefully, disagree,
Dscho

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

Hi Junio,

On Fri, 21 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> I still do not understand (note that I am not saying "I do not
> accept"--acceptance or rejection happens after an understandable
> explanation is given, and "do not understand" means no such
> explanation has been given yet) your justification behind adding a
> technical debt to reimplement the author-script parser and not
> sharing it with "git am" in 13/27.

At this point, I am most of all reluctant to introduce such a huge change,
which surely would introduce a regression.

This is what happened a couple of times to me, most recently with the
hide-dot-gitdir patch series that worked flawlessly for years, had to be
dramatically changed during review to enter git.git, and introduced the
major regression that `core.hideDotFiles = gitDirOnly` was broken.

The lesson I learned: review should not be valued more than the test of
time. This lesson has been reinforced by all the regressions that have not
been caught by review nor the test suite running on Linux only.

It would be a different matter if I still had the cross-validator in place
(which I did when I sent out v1 of this patch series) and tons of time to
spend on accommodating your wishes, however I may disagree with them. And
in this instance, I thought I made clear that I disagree, and why:
Internally, git-am and git-rebase-i handle the author-script very
differently. That may change at some stage in the future, and it would be
a good time then and there to take care of unifying this code. Currently,
not so much, as the only excuse to use the same parser would be that they
both read the same file, while they have to do very different things with
the parsed output (in fact, your suggestion would ask the parser in the
sequencer to rip apart the information into key/value pairs, only to
re-glue them back together when they are used as the environment variables
as which rebase-i treats the contents of the author-script file).

So no, at this point I am not willing to risk introducing breakages in
code that has been proven to work in practice.

Ciao,
Dscho

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

Hi Junio,

On Sat, 22 Oct 2016, Junio C Hamano wrote:

> Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> 
> > This patch series marks the '4' in the countdown to speed up rebase -i
> > by implementing large parts in C (read: there will be three more patch
> > series after that before the full benefit hits git.git: sequencer-i,
> > rebase--helper and rebase-i-extra).
> > ...
> > It would be *really* nice if we could get this patch series at least
> > into `next` soon, as it gets late and later for the rest of the
> > patches to make it into `master` in time for v2.11 (and it is not for
> > lack of trying on my end...).
> 
> This "countdown 4" step can affect cherry-pick and revert, even
> though we were careful to review changes to the sequencer.c code.

As I pointed out in another mail in this thread: we should not fall into
the trap of overrating review.

In the case of the rebase--helper patches, so far the review mainly
resulted in more work for me (having to change spellings elsewhere, for
example), not in improving the changes I intended to introduce into
git.git's code.

Sure, there has been the occasional improvement, but it certainly feels as
if I spent about 80% of the work after each -v1 iteration on things that
have positively nothing at all to do with accelerating rebase -i.

> I prefer to cook it in 'next' sufficiently long to ensure that we hear
> feedbacks from non-Windows users if there is any unexpected breakage.

FWIW I am using the same patches not only on Windows but also in my Linux
VM.

> There isn't enough time to include this topic in the upcoming
> release within the current https://tinyurl.com/gitCal calendar,
> however, which places the final on Nov 11th.

More is the pity.

Thank you, though, for being upfront with me. I will shift my focus to
tasks that require my attention more urgently, then.

Ciao,
Dscho

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

Hi Junio,

On Sun, 23 Oct 2016, Johannes Schindelin wrote:

> On Sat, 22 Oct 2016, Junio C Hamano wrote:
> 
> > Johannes Schindelin <johannes.schindelin@gmx.de> writes:
> > 
> > > This patch series marks the '4' in the countdown to speed up rebase -i
> > > by implementing large parts in C (read: there will be three more patch
> > > series after that before the full benefit hits git.git: sequencer-i,
> > > rebase--helper and rebase-i-extra).
> > > ...
> > > It would be *really* nice if we could get this patch series at least
> > > into `next` soon, as it gets late and later for the rest of the
> > > patches to make it into `master` in time for v2.11 (and it is not for
> > > lack of trying on my end...).
> > 
> > This "countdown 4" step can affect cherry-pick and revert, even

Oh, I forgot to comment on this tidbit of your mail, sorry.

This *is* the countdown 4, as the remaining 3 patch series depend on each
other in the order I sent them out.

Ciao,
Dscho

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Max Horn]

Hi Dscho,

> On 23 Oct 2016, at 11:54, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> 
> Hi Junio,
> 
> On Sat, 22 Oct 2016, Junio C Hamano wrote:
> 
[...]

>> There isn't enough time to include this topic in the upcoming
>> release within the current https://tinyurl.com/gitCal calendar,
>> however, which places the final on Nov 11th.
> 
> More is the pity.
> 
> Thank you, though, for being upfront with me. I will shift my focus to
> tasks that require my attention more urgently, then.

Junio did go on, though:

>> I am wondering if it makes sense to delay 2.11 by moving the final
>> by 4 weeks to Dec 9th.

I was reading this as an offer to delay things to accommodate the integration your work into 2.11. I.e. "within the current plan, there is no time for this, but we could adjust the plan". But maybe I am misinterpreting?


Cheers,
Max

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Johannes Schindelin]

Hi Max,

On Mon, 24 Oct 2016, Max Horn wrote:

> > On 23 Oct 2016, at 11:54, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> > 
> > On Sat, 22 Oct 2016, Junio C Hamano wrote:
> > 
> [...]
> 
> >> There isn't enough time to include this topic in the upcoming release
> >> within the current https://tinyurl.com/gitCal calendar, however,
> >> which places the final on Nov 11th.
> > 
> > More is the pity.
> > 
> > Thank you, though, for being upfront with me. I will shift my focus to
> > tasks that require my attention more urgently, then.
> 
> Junio did go on, though:
> 
> >> I am wondering if it makes sense to delay 2.11 by moving the final
> >> by 4 weeks to Dec 9th.
> 
> I was reading this as an offer to delay things to accommodate the
> integration your work into 2.11. I.e. "within the current plan, there is
> no time for this, but we could adjust the plan". But maybe I am
> misinterpreting?

There is no indication that the rebase--helper patches would make it into
2.11 even with four more weeks.

I will now focus on other things that I postponed in favor of the
interactive rebase patches. In fact, I *have* to focus on some quite
pressing tasks that I neglected over those patches.

It's not like the process would magically improve just because a release
date is pushed. To the contrary, pushing the release date to allow for the
rebase--helper to be included may very well have the counterintuitive
effect of delaying things beyond even that pushed date "because there is
now so much time left" (until there isn't). It's a variation of
[Parkinson's Law](https://en.wikipedia.org/wiki/Parkinson%27s_law) ;-)

Anyway, back to work,
Dscho

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Stefan Beller]

On Sat, Oct 22, 2016 at 10:11 AM, Junio C Hamano <gitster@pobox.com> wrote:

>
> There isn't enough time to include this topic in the upcoming
> release within the current https://tinyurl.com/gitCal calendar,
> however, which places the final on Nov 11th.
>
> I am wondering if it makes sense to delay 2.11 by moving the final
> by 4 weeks to Dec 9th.
>
> Thoughts?
>
> Speaking of what to and not to include in the upcoming release, we
> do want to include Stefan's off-by-one fix to the submodule-helper,
> but that is blocked on Windows end due to the test.

I'd be happy either way, i.e. we could revert that fix and make a release?
AFAICT, Windows only has broken tests, not broken functionality with that
submodule bug fix.

> I think
> everybody agreed that a longer time "right thing to do" fix is to
> address the "when base is /path/to/dir/., where is ../sub relative
> to it?" issue, but if we are to do so, it would need a longer
> gestation period once it hits 'next', as it can affect the current
> users and we may even need B/C notes in the release notes for the
> change.  Giving ourselves a few more weeks of breathing room would
> help us to make sure the fix to relative URL issue is sound, too.

If we want a longer gestation period, we'd ideally merge it to master
just after a release, such that we "cook" it in master without having
it in any release (we had a similar discussion for the diff heuristics IIRC).

So please don't let the release schedule depend on my ability to deliver a
proper patch for the submodule path issue.

Thanks,
Stefan

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I still do not understand (note that I am not saying "I do not
>> accept"--acceptance or rejection happens after an understandable
>> explanation is given, and "do not understand" means no such
>> explanation has been given yet) your justification behind adding a
>> technical debt to reimplement the author-script parser and not
>> sharing it with "git am" in 13/27.
>
> At this point, I am most of all reluctant to introduce such a huge change,
> which surely would introduce a regression.

That is a perfectly sensible and honest answer that I can
understand, accept and even support.

You've been working on the series for several months, running with
these dozen or so lines of code, and replacing it with another dozen
or so lines of code would require you to make sure the result is
actually doing the same thing for the remainder of your series.  And
I agree that is an unnecessary risk in order to ship a working code.
The code being battle tested counts.

I cared on this point mostly because I wanted to make sure that
people later can find out why there are two functions that ought to
be doing the same thing.  

If there were a technical reason why these two must stay to be
different implementations that are potentially doing different
things, I want to see that reason described, so that those who come
later and want to refactor the helper functions into one later will
know why they are separate in the first place.

If on the other hand there isn't any technical reason why they must
stay to be different, and they are different mostly because you
happened to have written a different one in 13/27 and have been
running with it, that is also a good thing for them to know.

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I prefer to cook it in 'next' sufficiently long to ensure that we hear
>> feedbacks from non-Windows users if there is any unexpected breakage.
>
> FWIW I am using the same patches not only on Windows but also in my Linux
> VM.

Thanks for a datapoint, but when I said "non-Windows users", I was
not referring you as "the" Windows user.  I am expecting that you
would hear from Windows users who got exposure to your series by its
inclusion in Git for Windows.  They are the ones that I had in mind
as "Windows users"---and not hearing breakage reported by them would
be a good sign.

The primary reason why we want to cook a new topic in 'next' is to
expose it to people with different workflows using it on different
things, and that is especially more important for a change that
affects features that are flexible and can be used in different
ways---the set of options and commands used by the original author
of the series are often different from other people's.

Any change, when it hits 'next', is expected to be sufficiently
tested by the original author [*1*], but that is only true in the
context of the original author's daily use.  Both reviews and
author's tests are not sufficient to find bugs [*2*].

Topics that touch parts of the system that are more important to
users' daily Git life deserve extra time to find any unexpected
breakage in them.  Windows users are participating in that test by
inclusion of the topic in the released version of Git for Windows.
I want to see the the test for the rest of the world done by early
adopters who run 'next' (as 'pu' is too scary for daily use).


[Footnote]

*1* And me, as topics geting ready to be in 'next' are first merged
    to my private edition branch that is slightly ahead of 'next' to
    be used in my everyday use, but just like the original author is
    merely one user, I am also merely one user with a specific set
    of workflows that is different from others'.

*2* Bug finding is not the primary purpose of the review in the
    first place.  It is to find design mistakes both at the external
    and internal level, and bug finding "here you have off-by-one"
    is merely a side effect.  End user tests may expose the former
    (e.g. the design based on a wrong assumption may not accomodate
    certain workflow the original author and the reviewers failed to
    consider while writing and reviewing), but no amount of test
    will uncover the latter (e.g. internal API that is misdesigned
    will make future enhancement unnecessarily harder).

    I think it was one of the achievements of the review cycle of
    this particular series that we got rid of the _entrust() thing,
    for example.  That had no visible external effect that would
    have been caught by cooking on 'next' or releasing it to the
    public, but was the kind of thing the code review was expected
    to find and fix.

Re: [PATCH v5 00/27] Prepare the sequencer for the upcoming rebase -i patches

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

>> Speaking of what to and not to include in the upcoming release, we
>> do want to include Stefan's off-by-one fix to the submodule-helper,
>> but that is blocked on Windows end due to the test.
>
> I'd be happy either way, i.e. we could revert that fix and make a release?
> AFAICT, Windows only has broken tests, not broken functionality with that
> submodule bug fix.

If you are referring the "trailing /. should not make difference
when resolving ../relative/path" change with "rever that fix", I
think that may be a reasonable way to proceed.  Even though that
change is a bugfix (at least from the point of view by me and j6t in
the recent discussion), it is a behaviour change that we would want
to see feedback from existing submodule users and deserves a longer
gestation period.  And that part is not yet in 'next' yet ;-)

> If we want a longer gestation period, we'd ideally merge it to master
> just after a release, such that we "cook" it in master without having
> it in any release (we had a similar discussion for the diff heuristics IIRC).

Yes.  

It would mean that we would need a separate patch that adds the
!MINGW prerequisite to some tests to what is on 'next', as the early
patches on sb/submodule-ignore-trailing-slash~ that fixes off-by-one
is the right thing to do either way.  It of course needs help from
Windows folks to validate the results.

Re: [PATCH v5 08/27] sequencer: completely revamp the "todo" script parsing

[Lars Schneider]

> On 21 Oct 2016, at 14:24, Johannes Schindelin <johannes.schindelin@gmx.de> wrote:
> 
> When we came up with the "sequencer" idea, we really wanted to have
> kind of a plumbing equivalent of the interactive rebase. Hence the
> choice of words: the "todo" script, a "pick", etc.
> 
> ...
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
> sequencer.c | 284 ++++++++++++++++++++++++++++++++++--------------------------
> 1 file changed, 163 insertions(+), 121 deletions(-)
> 
> diff --git a/sequencer.c b/sequencer.c
> index 499f5ee..145de78 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -470,7 +470,26 @@ static int allow_empty(struct replay_opts *opts, struct commit *commit)
> 		return 1;
> }
> 
> -static int do_pick_commit(struct commit *commit, struct replay_opts *opts)
> +enum todo_command {
> +	TODO_PICK = 0,
> +	TODO_REVERT
> +};
> +
> +static const char *todo_command_strings[] = {
> +	"pick",
> +	"revert"
> +};
> +
> +static const char *command_to_string(const enum todo_command command)
> +{
> +	if (command < ARRAY_SIZE(todo_command_strings))

With DEVELOPER=1 I get this error on macOS when I compile current git/next (b27dc33) using clang:

sequencer.c:632:14: error: comparison of constant 2 with expression of type 'const enum todo_command' is always true [-Werror,-Wtautological-constant-out-of-range-compare]
        if (command < ARRAY_SIZE(todo_command_strings))

Torsten discovered this problem already in v3 and Peff suggested
a working solution [1]. Is there any reason not to use Peff's suggestion?


Cheers,
Lars

[1] http://public-inbox.org/git/d9f4f658-94fb-cb9e-7da8-3a2fac120a9e@web.de/