[PATCH 00/19] Cleanup merge API

[PATCH 00/19] Cleanup merge API

[Elijah Newren]

Before writing a replacement merge strategy for recursive, I decided
to first cleanup the merge API -- streamlining merge-recursive.h and
making it more readable.  It includes some fixes I noticed along the
way, and the last two patches were some forgotten changes of mine I
rediscovered that had minor textual conflicts before I rebased them on
this series.

    While there are minor textual and semantic dependencies between
    these patches (preventing me from splitting up this series), they
    are logically separate and can be reviewed independently.

Stuff I'd most welcome review on:
  * Is cache-tree.c the right place for write_tree_from_memory()?
    [see patch 7]  Should there be docs on how it differs from
    write_index_as_tree(), already found in cache-tree?  What does
    the latter even do?

Some notes:
  * Applies on master, merges cleanly to next & pu
  * Only patches 3, 5-7 touch anything outside of merge-recursive
  * Patches 1 & 3 are bugfixes, 2 & 4 arguably are too, but all are
    quite minor and do not need to be part of the impending release
    candidates or 2.23.0.
  * I'm going to be out next week (July 29-Aug 3), so I can only
    respond to feedback for the next couple days or it'll have to
    wait until the 5th.

Stuff I didn't address:
  * merge_recursive() empties the commit_list of merge_bases it is
    passed, so the caller is expected to NOT use it afterwards.  Seems
    suboptimal to place such an expectation on the caller.
  * All current callers (3 of them?) of merge_recursive() always pass
    it a specially created reversed-list for the merge_bases.  Some
    history spelunking provides no details on any of these about why;
    it appears that the 2nd and 3rd callers reversed the list because
    the first did, and I'm guessing the first did in an attempt to
    exactly match the git-merge-recursive.py scripts' behavior.  But
    if the API needs them in a reverse order from what people would
    normally expect to pass them in, shouldn't it reverse them itself
    instead of making all callers do it?  Also, the order shouldn't
    matter when there are no conflicts, and when there are conflicts
    it'd only change which side of the conflict markers the made-up
    virtual merge base would list things in.  However, we do have
    tests with recursive virtual merge bases and which test the
    output, and I didn't want to try to clean those all up.  Besides,
    the current order shows nicely when commits are named things like
    "L1", "L2", "R1", "R2" -- it's nice having a well defined left and
    right side.  Wasn't sure what to do yet, so I just punted for now;
    this series is already long enough...


Elijah Newren (19):
  merge-recursive: fix minor memory leak in error condition
  merge-recursive: remove another implicit dependency on the_repository
  Ensure index matches head before invoking merge machinery, round N
  merge-recursive: exit early if index != head
  merge-recursive: don't force external callers to do our logging
  Change call signature of write_tree_from_memory()
  Move write_tree_from_memory() from merge-recursive to cache-tree
  merge-recursive: fix some overly long lines
  merge-recursive: use common name for ancestors/common/base_list
  merge-recursive: rename 'mrtree' to 'result_tree', for clarity
  merge-recursive: rename merge_options argument to opt in header
  merge-recursive: move some definitions around to clean up the header
  merge-recursive: consolidate unnecessary fields in merge_options
  merge-recursive: comment and reorder the merge_options fields
  merge-recursive: split internal fields into a separate struct
  merge-recursive.c: alphabetize include list
  merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
  merge-recursive: be consistent with assert
  merge-recursive: provide a better label for diff3 common ancestor

 builtin/checkout.c                |   2 +-
 builtin/merge-recursive.c         |   4 +
 builtin/stash.c                   |   2 +
 cache-tree.c                      |  30 ++
 cache-tree.h                      |   2 +
 merge-recursive.c                 | 517 +++++++++++++++++-------------
 merge-recursive.h                 | 119 +++----
 sequencer.c                       |   1 -
 t/t3030-merge-recursive.sh        |   9 +-
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 191 +++++++++++
 11 files changed, 589 insertions(+), 296 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

-- 
2.22.0.564.gb52f8ae349

[PATCH 01/19] merge-recursive: fix minor memory leak in error condition

[Elijah Newren]

Returning before freeing the allocated buffer is suboptimal; as with
elsewhere in the same function, make sure buf gets free'd.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 12300131fc..1163508811 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -934,9 +934,11 @@ static int update_file_flags(struct merge_options *opt,
 		}
 
 		buf = read_object_file(&contents->oid, &type, &size);
-		if (!buf)
-			return err(opt, _("cannot read object %s '%s'"),
-				   oid_to_hex(&contents->oid), path);
+		if (!buf) {
+			ret = err(opt, _("cannot read object %s '%s'"),
+				  oid_to_hex(&contents->oid), path);
+			goto free_buf;
+		}
 		if (type != OBJ_BLOB) {
 			ret = err(opt, _("blob expected for %s '%s'"),
 				  oid_to_hex(&contents->oid), path);
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 02/19] merge-recursive: remove another implicit dependency on the_repository

[Elijah Newren]

Commit d7cf3a96e9a0 ("merge-recursive.c: remove implicit dependency on
the_repository", 2019-01-12) and follow-ups like commit 34e7771bc644
("Use the right 'struct repository' instead of the_repository",
2019-06-27), removed most implicit uses of the_repository.  Convert
calls to get_commit_tree() to instead use repo_get_commit_tree() to get
rid of another.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1163508811..37bb94fb4d 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3571,8 +3571,11 @@ int merge_recursive(struct merge_options *opt,
 		repo_read_index(opt->repo);
 
 	opt->ancestor = "merged common ancestors";
-	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
-			    get_commit_tree(merged_common_ancestors),
+	clean = merge_trees(opt,
+			    repo_get_commit_tree(opt->repo, h1),
+			    repo_get_commit_tree(opt->repo, h2),
+			    repo_get_commit_tree(opt->repo,
+						 merged_common_ancestors),
 			    &mrtree);
 	if (clean < 0) {
 		flush_output(opt);
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 03/19] Ensure index matches head before invoking merge machinery, round N

[Elijah Newren]

This is the bug that just won't die; there always seems to be another
form of it somewhere.  See the commit message of 55f39cf7551b ("merge:
fix misleading pre-merge check documentation", 2018-06-30) for a more
detailed explanation), but in short:

<quick summary>

builtin/merge.c contains this important requirement for merge
strategies:

    ...the index must be in sync with the head commit.  The strategies are
    responsible to ensure this.

This condition is important to enforce because there are two likely
failure cases when the index isn't in sync with the head commit:

  * we silently throw away changes the user had staged before the merge

  * we accidentally (and silently) include changes in the merge that
    were not part of either of the branches/trees being merged

Discarding users' work and mis-merging are both bad outcomes, especially
when done silently, so naturally this rule was stated sternly -- but,
unfortunately totally ignored in practice unless and until actual bugs
were found.  But, fear not: the bugs from this were fixed in commit
  ee6566e8d70d ("[PATCH] Rewrite read-tree", 2005-09-05)
through a rewrite of read-tree (again, commit 55f39cf7551b has a more
detailed explanation of how this affected merge).  And it was fixed
again in commit
  160252f81626 ("git-merge-ours: make sure our index matches HEAD", 2005-11-03)
...and it was fixed again in commit
  3ec62ad9ffba ("merge-octopus: abort if index does not match HEAD", 2016-04-09)
...and again in commit
  65170c07d466 ("merge-recursive: avoid incorporating uncommitted changes in a merge", 2017-12-21)
...and again in commit
  eddd1a411d93 ("merge-recursive: enforce rule that index matches head before merging", 2018-06-30)

...with multiple testcases added to the testsuite that could be
enumerated in even more commits.

Then, finally, in a patch in the same series as the last fix above, the
documentation about this requirement was fixed in commit 55f39cf7551b
("merge: fix misleading pre-merge check documentation", 2018-06-30), and
we all lived happily ever after...

</quick summary>

Unfortunately, "ever after" apparently denotes a limited time and it
expired today.  The merge-recursive rule to enforce that index matches
head was at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially HUGE amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Trying to enforce that all of merge_trees(), merge_recursive(), and
merge_recursive_generic() checked the index == head condition earlier
resulted in a bunch of broken tests.  It turns out that
merge_recursive() has code to drop and reload the cache while recursing
to create intermediate virtual merge bases, but unfortunately that code
runs even when no recursion is necessary.  This unconditional dropping
and reloading of the cache masked a few bugs:

  * builtin/merge-recursive.c: didn't even bother loading the index.

  * builtin/stash.c: feels like a fake 'builtin' because it repeatedly
    invokes git subprocesses all over the place, mixed with other
    operations.  In particular, invoking "git reset" will reset the
    index on disk, but the parent process that invoked it won't
    automatically have its in-memory index updated.

  * t3030-merge-recursive.h: this test has always been broken in that it
    didn't make sure to make index match head before running.  But, it
    didn't care about the index or even the merge result, just the
    verbose output while running.  While commit eddd1a411d93
    ("merge-recursive: enforce rule that index matches head before
    merging", 2018-06-30) should have uncovered this broken test, it
    used a test_must_fail wrapper around the merge-recursive call
    because it was known that the merge resulted in a rename/rename
    conflict.  Thus, that fix only made this test fail for a different
    reason, and since the index == head check didn't happen until after
    coming all the way back out of the recursion, the testcase had
    enough information to pass the one check that it did perform.

So, load the index in builtin/merge-recursive.c, reload the in-memory
index in builtin/stash.c, and modify the t3030 testcase to correctly
setup the index and make sure that the test fails in the expected way
(meaning it reports a rename/rename conflict).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-recursive.c  | 4 ++++
 builtin/stash.c            | 2 ++
 t/t3030-merge-recursive.sh | 9 ++++++++-
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
index 5b910e351e..a4bfd8fc51 100644
--- a/builtin/merge-recursive.c
+++ b/builtin/merge-recursive.c
@@ -1,3 +1,4 @@
+#include "cache.h"
 #include "builtin.h"
 #include "commit.h"
 #include "tag.h"
@@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
 	if (argc - i != 3) /* "--" "<head>" "<remote>" */
 		die(_("not handling anything other than two heads merge."));
 
+	if (repo_read_index_unmerged(the_repository))
+		die_resolve_conflict("merge");
+
 	o.branch1 = argv[++i];
 	o.branch2 = argv[++i];
 
diff --git a/builtin/stash.c b/builtin/stash.c
index fde6397caa..bec011c1bb 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -427,6 +427,8 @@ static int do_apply_stash(const char *prefix, struct stash_info *info,
 				return error(_("could not save index tree"));
 
 			reset_head();
+			discard_cache();
+			read_cache();
 		}
 	}
 
diff --git a/t/t3030-merge-recursive.sh b/t/t3030-merge-recursive.sh
index ff641b348a..a37bcc58a0 100755
--- a/t/t3030-merge-recursive.sh
+++ b/t/t3030-merge-recursive.sh
@@ -667,15 +667,22 @@ test_expect_success 'merging with triple rename across D/F conflict' '
 test_expect_success 'merge-recursive remembers the names of all base trees' '
 	git reset --hard HEAD &&
 
+	# make the index match $c1 so that merge-recursive below does not
+	# fail early
+	git diff --binary HEAD $c1 -- | git apply --cached &&
+
 	# more trees than static slots used by oid_to_hex()
 	for commit in $c0 $c2 $c4 $c5 $c6 $c7
 	do
 		git rev-parse "$commit^{tree}"
 	done >trees &&
 
-	# ignore the return code -- it only fails because the input is weird
+	# ignore the return code; it only fails because the input is weird...
 	test_must_fail git -c merge.verbosity=5 merge-recursive $(cat trees) -- $c1 $c3 >out &&
 
+	# ...but make sure it fails in the expected way
+	test_i18ngrep CONFLICT.*rename/rename out &&
+
 	# merge-recursive prints in reverse order, but we do not care
 	sort <trees >expect &&
 	sed -n "s/^virtual //p" out | sort >actual &&
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 04/19] merge-recursive: exit early if index != head

[Elijah Newren]

We had a rule to enforce that the index matches head, but it was found
at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially huge amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Further, not enforcing this requirement earlier allowed other bugs (such
as an unintentional unconditional dropping and reloading of the index in
merge_recursive() even when no recursion was necessary), to mask bugs in
other callers (which were fixed in the commit prior to this one).

Make sure we do the index == head check at the beginning of the merge,
and error out immediately if it fails.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 93 +++++++++++++++++++++++++++++++++++------------
 1 file changed, 69 insertions(+), 24 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 37bb94fb4d..b762ecd7bd 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3381,21 +3381,14 @@ static int process_entry(struct merge_options *opt,
 	return clean_merge;
 }
 
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *common,
-		struct tree **result)
+static int merge_trees_internal(struct merge_options *opt,
+				struct tree *head,
+				struct tree *merge,
+				struct tree *common,
+				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
-	struct strbuf sb = STRBUF_INIT;
-
-	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
-		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
-		    sb.buf);
-		return -1;
-	}
 
 	if (opt->subtree_shift) {
 		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
@@ -3499,11 +3492,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
  * Merge the commits h1 and h2, return the resulting virtual
  * commit object and a flag indicating the cleanness of the merge.
  */
-int merge_recursive(struct merge_options *opt,
-		    struct commit *h1,
-		    struct commit *h2,
-		    struct commit_list *ca,
-		    struct commit **result)
+static int merge_recursive_internal(struct merge_options *opt,
+				    struct commit *h1,
+				    struct commit *h2,
+				    struct commit_list *ca,
+				    struct commit **result)
 {
 	struct commit_list *iter;
 	struct commit *merged_common_ancestors;
@@ -3555,7 +3548,7 @@ int merge_recursive(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive(opt, merged_common_ancestors, iter->item,
+		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
 				    NULL, &merged_common_ancestors) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
@@ -3571,12 +3564,12 @@ int merge_recursive(struct merge_options *opt,
 		repo_read_index(opt->repo);
 
 	opt->ancestor = "merged common ancestors";
-	clean = merge_trees(opt,
-			    repo_get_commit_tree(opt->repo, h1),
-			    repo_get_commit_tree(opt->repo, h2),
-			    repo_get_commit_tree(opt->repo,
-						 merged_common_ancestors),
-			    &mrtree);
+	clean = merge_trees_internal(opt,
+				     repo_get_commit_tree(opt->repo, h1),
+				     repo_get_commit_tree(opt->repo, h2),
+				     repo_get_commit_tree(opt->repo,
+							  merged_common_ancestors),
+				     &mrtree);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
@@ -3596,6 +3589,58 @@ int merge_recursive(struct merge_options *opt,
 	return clean;
 }
 
+static int merge_start(struct merge_options *opt, struct tree *head)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	assert(opt->branch1 && opt->branch2);
+
+	if (repo_index_has_changes(opt->repo, head, &sb)) {
+		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
+		    sb.buf);
+		return -1;
+	}
+
+	return 0;
+}
+
+static void merge_finalize(struct merge_options *opt)
+{
+	/* Common code for wrapping up merges will be added here later */
+}
+
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *common,
+		struct tree **result)
+{
+	int ret;
+
+	if (merge_start(opt, head))
+		return -1;
+	ret = merge_trees_internal(opt, head, merge, common, result);
+	merge_finalize(opt);
+
+	return ret;
+}
+
+int merge_recursive(struct merge_options *opt,
+		    struct commit *h1,
+		    struct commit *h2,
+		    struct commit_list *ca,
+		    struct commit **result)
+{
+	int ret;
+
+	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
+		return -1;
+	ret = merge_recursive_internal(opt, h1, h2, ca, result);
+	merge_finalize(opt);
+
+	return ret;
+}
+
 static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
 			      const char *name)
 {
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 05/19] merge-recursive: don't force external callers to do our logging

[Elijah Newren]

Alternatively, you can view this as "make the merge functions behave
more similarly."  merge-recursive has three different entry points:
merge_trees(), merge_recursive(), and merge_recursive_generic().  Two of
these would call diff_warn_rename_limit(), but merge_trees() didn't.
This lead to callers of merge_trees() needing to manually call
diff_warn_rename_limit() themselves.  Move this to the new
merge_finalize() function to make sure that all three entry points run
this function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++----
 sequencer.c       | 1 -
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index b762ecd7bd..7f56cb0ed1 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3583,9 +3583,6 @@ static int merge_recursive_internal(struct merge_options *opt,
 	flush_output(opt);
 	if (!opt->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
-	if (show(opt, 2))
-		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
 	return clean;
 }
 
@@ -3606,7 +3603,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 
 static void merge_finalize(struct merge_options *opt)
 {
-	/* Common code for wrapping up merges will be added here later */
+	if (show(opt, 2))
+		diff_warn_rename_limit("merge.renamelimit",
+				       opt->needed_rename_limit, 0);
 }
 
 int merge_trees(struct merge_options *opt,
diff --git a/sequencer.c b/sequencer.c
index 34ebf8ed94..c85e594704 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -617,7 +617,6 @@ static int do_recursive_merge(struct repository *r,
 	if (is_rebase_i(opts) && clean <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
-	diff_warn_rename_limit("merge.renamelimit", o.needed_rename_limit, 0);
 	if (clean < 0) {
 		rollback_lock_file(&index_lock);
 		return clean;
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 06/19] Change call signature of write_tree_from_memory()

[Elijah Newren]

write_tree_from_memory() is more of a cache-tree thing than a
merge-recursive thing (especially since it is called from checkout.c in
a context before doing anything with merging), but in particular there
is no need for it to take a merge_options struct when it only really
needs a repository struct.

One small wrinkle in this is that there is a call to err(), which takes
a merge_options.  However, this did not used to be there.  In commits
6003303a1e50 ("merge-recursive: switch to returning errors instead of
dying", 2016-07-26) and 033abf97fcbc ("Replace all die("BUG: ...") calls
by BUG() ones", 2018-05-02), all the calls to die() were switched over
to either err() or BUG() -- and this particular case was converted
incorrectly; it should have been a BUG().

So, convert write_tree_from_memory()'s current call to err() to instead
call BUG(), and then make it take a struct repository instead of a
struct merge_options.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c |  2 +-
 merge-recursive.c  | 11 +++++------
 merge-recursive.h  |  2 +-
 3 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 91f8509f85..ec13116354 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -760,7 +760,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 */
 			init_merge_options(&o, the_repository);
 			o.verbosity = 0;
-			work = write_tree_from_memory(&o);
+			work = write_tree_from_memory(the_repository);
 
 			ret = reset_tree(new_tree,
 					 opts, 1,
diff --git a/merge-recursive.c b/merge-recursive.c
index 7f56cb0ed1..1a3c6ab7f3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -412,10 +412,10 @@ static void unpack_trees_finish(struct merge_options *opt)
 	clear_unpack_trees_porcelain(&opt->unpack_opts);
 }
 
-struct tree *write_tree_from_memory(struct merge_options *opt)
+struct tree *write_tree_from_memory(struct repository *repo)
 {
 	struct tree *result = NULL;
-	struct index_state *istate = opt->repo->index;
+	struct index_state *istate = repo->index;
 
 	if (unmerged_index(istate)) {
 		int i;
@@ -434,11 +434,10 @@ struct tree *write_tree_from_memory(struct merge_options *opt)
 
 	if (!cache_tree_fully_valid(istate->cache_tree) &&
 	    cache_tree_update(istate, 0) < 0) {
-		err(opt, _("error building trees"));
-		return NULL;
+		BUG("error building trees");
 	}
 
-	result = lookup_tree(opt->repo, &istate->cache_tree->oid);
+	result = lookup_tree(repo, &istate->cache_tree->oid);
 
 	return result;
 }
@@ -3471,7 +3470,7 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth && !(*result = write_tree_from_memory(opt)))
+	if (opt->call_depth && !(*result = write_tree_from_memory(opt->repo)))
 		return -1;
 
 	return clean;
diff --git a/merge-recursive.h b/merge-recursive.h
index c2b7bb65c6..33f3d53c09 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -97,7 +97,7 @@ int merge_recursive_generic(struct merge_options *o,
 
 void init_merge_options(struct merge_options *o,
 			struct repository *repo);
-struct tree *write_tree_from_memory(struct merge_options *o);
+struct tree *write_tree_from_memory(struct repository *repo);
 
 int parse_merge_opt(struct merge_options *out, const char *s);
 
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 07/19] Move write_tree_from_memory() from merge-recursive to cache-tree

[Elijah Newren]

No code changes, just moving the function to what seems like a more
logical place.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 cache-tree.c      | 30 ++++++++++++++++++++++++++++++
 cache-tree.h      |  2 ++
 merge-recursive.c | 30 ------------------------------
 merge-recursive.h |  1 -
 4 files changed, 32 insertions(+), 31 deletions(-)

diff --git a/cache-tree.c b/cache-tree.c
index 706ffcf188..d2baf6baf5 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -659,6 +659,36 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 	return ret;
 }
 
+struct tree *write_tree_from_memory(struct repository *repo)
+{
+	struct tree *result = NULL;
+	struct index_state *istate = repo->index;
+
+	if (unmerged_index(istate)) {
+		int i;
+		fprintf(stderr, "BUG: There are unmerged index entries:\n");
+		for (i = 0; i < istate->cache_nr; i++) {
+			const struct cache_entry *ce = istate->cache[i];
+			if (ce_stage(ce))
+				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
+					(int)ce_namelen(ce), ce->name);
+		}
+		BUG("unmerged index entries in merge-recursive.c");
+	}
+
+	if (!istate->cache_tree)
+		istate->cache_tree = cache_tree();
+
+	if (!cache_tree_fully_valid(istate->cache_tree) &&
+	    cache_tree_update(istate, 0) < 0) {
+		BUG("error building trees");
+	}
+
+	result = lookup_tree(repo, &istate->cache_tree->oid);
+
+	return result;
+}
+
 static void prime_cache_tree_rec(struct repository *r,
 				 struct cache_tree *it,
 				 struct tree *tree)
diff --git a/cache-tree.h b/cache-tree.h
index 757bbc48bc..4d67c237ba 100644
--- a/cache-tree.h
+++ b/cache-tree.h
@@ -47,6 +47,8 @@ void cache_tree_verify(struct repository *, struct index_state *);
 #define WRITE_TREE_PREFIX_ERROR (-3)
 
 int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix);
+struct tree *write_tree_from_memory(struct repository *repo);
+
 void prime_cache_tree(struct repository *, struct index_state *, struct tree *);
 
 int cache_tree_matches_traversal(struct cache_tree *, struct name_entry *ent, struct traverse_info *info);
diff --git a/merge-recursive.c b/merge-recursive.c
index 1a3c6ab7f3..937a816288 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -412,36 +412,6 @@ static void unpack_trees_finish(struct merge_options *opt)
 	clear_unpack_trees_porcelain(&opt->unpack_opts);
 }
 
-struct tree *write_tree_from_memory(struct repository *repo)
-{
-	struct tree *result = NULL;
-	struct index_state *istate = repo->index;
-
-	if (unmerged_index(istate)) {
-		int i;
-		fprintf(stderr, "BUG: There are unmerged index entries:\n");
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
-			if (ce_stage(ce))
-				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
-					(int)ce_namelen(ce), ce->name);
-		}
-		BUG("unmerged index entries in merge-recursive.c");
-	}
-
-	if (!istate->cache_tree)
-		istate->cache_tree = cache_tree();
-
-	if (!cache_tree_fully_valid(istate->cache_tree) &&
-	    cache_tree_update(istate, 0) < 0) {
-		BUG("error building trees");
-	}
-
-	result = lookup_tree(repo, &istate->cache_tree->oid);
-
-	return result;
-}
-
 static int save_files_dirs(const struct object_id *oid,
 			   struct strbuf *base, const char *path,
 			   unsigned int mode, int stage, void *context)
diff --git a/merge-recursive.h b/merge-recursive.h
index 33f3d53c09..973f2729a8 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -97,7 +97,6 @@ int merge_recursive_generic(struct merge_options *o,
 
 void init_merge_options(struct merge_options *o,
 			struct repository *repo);
-struct tree *write_tree_from_memory(struct repository *repo);
 
 int parse_merge_opt(struct merge_options *out, const char *s);
 
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 08/19] merge-recursive: fix some overly long lines

[Elijah Newren]

No substantive code change, just add some line breaks to fix lines that
have grown in length due to various refactorings.  Most remaining lines
of excessive length in merge-recursive include error messages and it's
not clear that splitting those improves things.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 937a816288..104c70b787 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -681,7 +681,9 @@ static void add_flattened_path(struct strbuf *out, const char *s)
 			out->buf[i] = '_';
 }
 
-static char *unique_path(struct merge_options *opt, const char *path, const char *branch)
+static char *unique_path(struct merge_options *opt,
+			 const char *path,
+			 const char *branch)
 {
 	struct path_hashmap_entry *entry;
 	struct strbuf newpath = STRBUF_INIT;
@@ -915,7 +917,8 @@ static int update_file_flags(struct merge_options *opt,
 		}
 		if (S_ISREG(contents->mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
-			if (convert_to_working_tree(opt->repo->index, path, buf, size, &strbuf)) {
+			if (convert_to_working_tree(opt->repo->index,
+						    path, buf, size, &strbuf)) {
 				free(buf);
 				size = strbuf.len;
 				buf = strbuf_detach(&strbuf, NULL);
@@ -3392,7 +3395,8 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp, NULL, 512);
+		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
 
@@ -3498,7 +3502,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo, tree, "ancestor");
+		merged_common_ancestors = make_virtual_commit(opt->repo,
+							      tree, "ancestor");
 	}
 
 	for (iter = ca; iter; iter = iter->next) {
@@ -3609,7 +3614,8 @@ int merge_recursive(struct merge_options *opt,
 	return ret;
 }
 
-static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
+static struct commit *get_ref(struct repository *repo,
+			      const struct object_id *oid,
 			      const char *name)
 {
 	struct object *object;
@@ -3644,7 +3650,8 @@ int merge_recursive_generic(struct merge_options *opt,
 		int i;
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i], oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, base_list[i],
+					     oid_to_hex(base_list[i]))))
 				return err(opt, _("Could not parse object '%s'"),
 					   oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 09/19] merge-recursive: use common name for ancestors/common/base_list

[Elijah Newren]

merge_trees(), merge_recursive(), and merge_recursive_generic() in
their function headers used four different names for the merge base or
list of merge bases they were passed:
  * 'common'
  * 'ancestors'
  * 'ca'
  * 'base_list'
They were able to refer to it four different ways instead of only three
by using a different name in the signature for the .c file than the .h
file.  Change all of these to 'merge_base' or 'merge_bases'.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 68 ++++++++++++++++++++++++-----------------------
 merge-recursive.h |  8 +++---
 2 files changed, 39 insertions(+), 37 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 104c70b787..0a90546824 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3356,24 +3356,26 @@ static int process_entry(struct merge_options *opt,
 static int merge_trees_internal(struct merge_options *opt,
 				struct tree *head,
 				struct tree *merge,
-				struct tree *common,
+				struct tree *merge_base,
 				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
 
 	if (opt->subtree_shift) {
-		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
-		common = shift_tree_object(opt->repo, head, common, opt->subtree_shift);
+		merge = shift_tree_object(opt->repo, head, merge,
+					  opt->subtree_shift);
+		merge_base = shift_tree_object(opt->repo, head, merge_base,
+					       opt->subtree_shift);
 	}
 
-	if (oid_eq(&common->object.oid, &merge->object.oid)) {
+	if (oid_eq(&merge_base->object.oid, &merge->object.oid)) {
 		output(opt, 0, _("Already up to date!"));
 		*result = head;
 		return 1;
 	}
 
-	code = unpack_trees_start(opt, common, head, merge);
+	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
 		if (show(opt, 4) || opt->call_depth)
@@ -3401,7 +3403,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		get_files_dirs(opt, merge);
 
 		entries = get_unmerged(opt->repo->index);
-		clean = detect_and_process_renames(opt, common, head, merge,
+		clean = detect_and_process_renames(opt, merge_base, head, merge,
 						   entries, &re_info);
 		record_df_conflict_files(opt, entries);
 		if (clean < 0)
@@ -3468,11 +3470,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
 static int merge_recursive_internal(struct merge_options *opt,
 				    struct commit *h1,
 				    struct commit *h2,
-				    struct commit_list *ca,
+				    struct commit_list *merge_bases,
 				    struct commit **result)
 {
 	struct commit_list *iter;
-	struct commit *merged_common_ancestors;
+	struct commit *merged_merge_bases;
 	struct tree *mrtree;
 	int clean;
 
@@ -3482,31 +3484,31 @@ static int merge_recursive_internal(struct merge_options *opt,
 		output_commit_title(opt, h2);
 	}
 
-	if (!ca) {
-		ca = get_merge_bases(h1, h2);
-		ca = reverse_commit_list(ca);
+	if (!merge_bases) {
+		merge_bases = get_merge_bases(h1, h2);
+		merge_bases = reverse_commit_list(merge_bases);
 	}
 
 	if (show(opt, 5)) {
-		unsigned cnt = commit_list_count(ca);
+		unsigned cnt = commit_list_count(merge_bases);
 
 		output(opt, 5, Q_("found %u common ancestor:",
 				"found %u common ancestors:", cnt), cnt);
-		for (iter = ca; iter; iter = iter->next)
+		for (iter = merge_bases; iter; iter = iter->next)
 			output_commit_title(opt, iter->item);
 	}
 
-	merged_common_ancestors = pop_commit(&ca);
-	if (merged_common_ancestors == NULL) {
+	merged_merge_bases = pop_commit(&merge_bases);
+	if (merged_merge_bases == NULL) {
 		/* if there is no common ancestor, use an empty tree */
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo,
-							      tree, "ancestor");
+		merged_merge_bases = make_virtual_commit(opt->repo, tree,
+							 "ancestor");
 	}
 
-	for (iter = ca; iter; iter = iter->next) {
+	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
 		opt->call_depth++;
 		/*
@@ -3522,14 +3524,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
-				    NULL, &merged_common_ancestors) < 0)
+		if (merge_recursive_internal(opt, merged_merge_bases, iter->item,
+					     NULL, &merged_merge_bases) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
 		opt->call_depth--;
 
-		if (!merged_common_ancestors)
+		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
@@ -3542,7 +3544,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h1),
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
-							  merged_common_ancestors),
+							  merged_merge_bases),
 				     &mrtree);
 	if (clean < 0) {
 		flush_output(opt);
@@ -3585,14 +3587,14 @@ static void merge_finalize(struct merge_options *opt)
 int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
+		struct tree *merge_base,
 		struct tree **result)
 {
 	int ret;
 
 	if (merge_start(opt, head))
 		return -1;
-	ret = merge_trees_internal(opt, head, merge, common, result);
+	ret = merge_trees_internal(opt, head, merge, merge_base, result);
 	merge_finalize(opt);
 
 	return ret;
@@ -3601,14 +3603,14 @@ int merge_trees(struct merge_options *opt,
 int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ca,
+		    struct commit_list *merge_bases,
 		    struct commit **result)
 {
 	int ret;
 
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
-	ret = merge_recursive_internal(opt, h1, h2, ca, result);
+	ret = merge_recursive_internal(opt, h1, h2, merge_bases, result);
 	merge_finalize(opt);
 
 	return ret;
@@ -3636,8 +3638,8 @@ static struct commit *get_ref(struct repository *repo,
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_base_list,
-			    const struct object_id **base_list,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result)
 {
 	int clean;
@@ -3646,14 +3648,14 @@ int merge_recursive_generic(struct merge_options *opt,
 	struct commit *next_commit = get_ref(opt->repo, merge, opt->branch2);
 	struct commit_list *ca = NULL;
 
-	if (base_list) {
+	if (merge_bases) {
 		int i;
-		for (i = 0; i < num_base_list; ++i) {
+		for (i = 0; i < num_merge_bases; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i],
-					     oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, merge_bases[i],
+					     oid_to_hex(merge_bases[i]))))
 				return err(opt, _("Could not parse object '%s'"),
-					   oid_to_hex(base_list[i]));
+					   oid_to_hex(merge_bases[i]));
 			commit_list_insert(base, &ca);
 		}
 	}
diff --git a/merge-recursive.h b/merge-recursive.h
index 973f2729a8..a96c8ad50f 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -74,14 +74,14 @@ static inline int merge_detect_rename(struct merge_options *o)
 int merge_recursive(struct merge_options *o,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ancestors,
+		    struct commit_list *merge_bases,
 		    struct commit **result);
 
 /* rename-detecting three-way merge, no recursion */
 int merge_trees(struct merge_options *o,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
+		struct tree *merge_base,
 		struct tree **result);
 
 /*
@@ -91,8 +91,8 @@ int merge_trees(struct merge_options *o,
 int merge_recursive_generic(struct merge_options *o,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_ca,
-			    const struct object_id **ca,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result);
 
 void init_merge_options(struct merge_options *o,
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 10/19] merge-recursive: rename 'mrtree' to 'result_tree', for clarity

[Elijah Newren]

It is not at all clear what 'mr' was supposed to stand for, at least not
to me.  Pick a clearer name for this variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0a90546824..61faa26c4f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3475,7 +3475,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 {
 	struct commit_list *iter;
 	struct commit *merged_merge_bases;
-	struct tree *mrtree;
+	struct tree *result_tree;
 	int clean;
 
 	if (show(opt, 4)) {
@@ -3545,14 +3545,15 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
 							  merged_merge_bases),
-				     &mrtree);
+				     &result_tree);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
 	}
 
 	if (opt->call_depth) {
-		*result = make_virtual_commit(opt->repo, mrtree, "merged tree");
+		*result = make_virtual_commit(opt->repo, result_tree,
+					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 11/19] merge-recursive: rename merge_options argument to opt in header

[Elijah Newren]

In commit 259ccb6cc324 ("merge-recursive: rename merge_options argument
from 'o' to 'opt'", 2019-04-05), I renamed a bunch of function
arguments in merge-recursive.c, but forgot to make that same change to
merge-recursive.h.  Make the two match.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.h | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/merge-recursive.h b/merge-recursive.h
index a96c8ad50f..e27c2a05bb 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -64,21 +64,21 @@ struct collision_entry {
 	unsigned reported_already:1;
 };
 
-static inline int merge_detect_rename(struct merge_options *o)
+static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return o->merge_detect_rename >= 0 ? o->merge_detect_rename :
-		o->diff_detect_rename >= 0 ? o->diff_detect_rename : 1;
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
 }
 
 /* merge_trees() but with recursive ancestor consolidation */
-int merge_recursive(struct merge_options *o,
+int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
 		    struct commit_list *merge_bases,
 		    struct commit **result);
 
 /* rename-detecting three-way merge, no recursion */
-int merge_trees(struct merge_options *o,
+int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
 		struct tree *merge_base,
@@ -88,16 +88,16 @@ int merge_trees(struct merge_options *o,
  * "git-merge-recursive" can be fed trees; wrap them into
  * virtual commits and call merge_recursive() proper.
  */
-int merge_recursive_generic(struct merge_options *o,
+int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
 			    int num_merge_bases,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *o,
+void init_merge_options(struct merge_options *opt,
 			struct repository *repo);
 
-int parse_merge_opt(struct merge_options *out, const char *s);
+int parse_merge_opt(struct merge_options *opt, const char *s);
 
 #endif
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 12/19] merge-recursive: move some definitions around to clean up the header

[Elijah Newren]

No substantive code changes (view this with diff --color-moved), but
a few small code cleanups:
  * Move structs and an inline function only used by merge-recursive.c
    into merge-recursive.c
  * Re-order function declarations to be more logical
  * Add or fix some explanatory comments

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 31 +++++++++++++++++++++++++++
 merge-recursive.h | 53 ++++++++++-------------------------------------
 2 files changed, 42 insertions(+), 42 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 61faa26c4f..0a944f51ae 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -54,6 +54,24 @@ static unsigned int path_hash(const char *path)
 	return ignore_case ? strihash(path) : strhash(path);
 }
 
+/*
+ * For dir_rename_entry, directory names are stored as a full path from the
+ * toplevel of the repository and do not include a trailing '/'.  Also:
+ *
+ *   dir:                original name of directory being renamed
+ *   non_unique_new_dir: if true, could not determine new_dir
+ *   new_dir:            final name of directory being renamed
+ *   possible_new_dirs:  temporary used to help determine new_dir; see comments
+ *                       in get_directory_renames() for details
+ */
+struct dir_rename_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *dir;
+	unsigned non_unique_new_dir:1;
+	struct strbuf new_dir;
+	struct string_list possible_new_dirs;
+};
+
 static struct dir_rename_entry *dir_rename_find_entry(struct hashmap *hashmap,
 						      char *dir)
 {
@@ -92,6 +110,13 @@ static void dir_rename_entry_init(struct dir_rename_entry *entry,
 	string_list_init(&entry->possible_new_dirs, 0);
 }
 
+struct collision_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *target_file;
+	struct string_list source_files;
+	unsigned reported_already:1;
+};
+
 static struct collision_entry *collision_find_entry(struct hashmap *hashmap,
 						    char *target_file)
 {
@@ -358,6 +383,12 @@ static int add_cacheinfo(struct merge_options *opt,
 	return ret;
 }
 
+static inline int merge_detect_rename(struct merge_options *opt)
+{
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+}
+
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 {
 	parse_tree(tree);
diff --git a/merge-recursive.h b/merge-recursive.h
index e27c2a05bb..1454772528 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -39,36 +39,17 @@ struct merge_options {
 	struct repository *repo;
 };
 
-/*
- * For dir_rename_entry, directory names are stored as a full path from the
- * toplevel of the repository and do not include a trailing '/'.  Also:
- *
- *   dir:                original name of directory being renamed
- *   non_unique_new_dir: if true, could not determine new_dir
- *   new_dir:            final name of directory being renamed
- *   possible_new_dirs:  temporary used to help determine new_dir; see comments
- *                       in get_directory_renames() for details
- */
-struct dir_rename_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *dir;
-	unsigned non_unique_new_dir:1;
-	struct strbuf new_dir;
-	struct string_list possible_new_dirs;
-};
+void init_merge_options(struct merge_options *opt, struct repository *repo);
 
-struct collision_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *target_file;
-	struct string_list source_files;
-	unsigned reported_already:1;
-};
+/* parse the option in s and update the relevant field of opt */
+int parse_merge_opt(struct merge_options *opt, const char *s);
 
-static inline int merge_detect_rename(struct merge_options *opt)
-{
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
-}
+/* rename-detecting three-way merge, no recursion */
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *merge_base,
+		struct tree **result);
 
 /* merge_trees() but with recursive ancestor consolidation */
 int merge_recursive(struct merge_options *opt,
@@ -77,16 +58,9 @@ int merge_recursive(struct merge_options *opt,
 		    struct commit_list *merge_bases,
 		    struct commit **result);
 
-/* rename-detecting three-way merge, no recursion */
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *merge_base,
-		struct tree **result);
-
 /*
- * "git-merge-recursive" can be fed trees; wrap them into
- * virtual commits and call merge_recursive() proper.
+ * merge_recursive_generic can operate on trees instead of commits, by
+ * wrapping the trees into virtual commits, and calling merge_recursive().
  */
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
@@ -95,9 +69,4 @@ int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *opt,
-			struct repository *repo);
-
-int parse_merge_opt(struct merge_options *opt, const char *s);
-
 #endif
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 13/19] merge-recursive: consolidate unnecessary fields in merge_options

[Elijah Newren]

We provided users with the ability to state whether they wanted rename
detection, and to put a limit on how much CPU would be spent.  Both of
these fields had multiple configuration parameters for setting them,
with one being a fallback and the other being an override.  However,
instead of implementing the logic for how to combine the multiple
source locations into the appropriate setting at config loading time,
we loaded and tracked both values and then made the code combine them
every time it wanted to check the overall value.  This had a few
minor drawbacks:
  * it seems more complicated than necessary
  * it runs the risk of people using the independent settings in the
    future and breaking the intent of how the options are used
    together
  * it makes merge_options more complicated than necessary for other
    potential users of the API

Fix these problems by moving the logic for combining the pairs of
options into a single value; make it apply at time-of-config-loading
instead of each-time-of-use.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 27 +++++++++++----------------
 merge-recursive.h |  6 ++----
 2 files changed, 13 insertions(+), 20 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0a944f51ae..0f8d451f2e 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -385,8 +385,7 @@ static int add_cacheinfo(struct merge_options *opt,
 
 static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+	return (opt->detect_renames != -1) ? opt->detect_renames : 1;
 }
 
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
@@ -1885,9 +1884,7 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	 */
 	if (opts.detect_rename > DIFF_DETECT_RENAME)
 		opts.detect_rename = DIFF_DETECT_RENAME;
-	opts.rename_limit = opt->merge_rename_limit >= 0 ? opt->merge_rename_limit :
-			    opt->diff_rename_limit >= 0 ? opt->diff_rename_limit :
-			    1000;
+	opts.rename_limit = (opt->rename_limit != -1) ? opt->rename_limit : 1000;
 	opts.rename_score = opt->rename_score;
 	opts.show_rename_progress = opt->show_rename_progress;
 	opts.output_format = DIFF_FORMAT_NO_OUTPUT;
@@ -3711,14 +3708,14 @@ static void merge_recursive_config(struct merge_options *opt)
 {
 	char *value = NULL;
 	git_config_get_int("merge.verbosity", &opt->verbosity);
-	git_config_get_int("diff.renamelimit", &opt->diff_rename_limit);
-	git_config_get_int("merge.renamelimit", &opt->merge_rename_limit);
+	git_config_get_int("diff.renamelimit", &opt->rename_limit);
+	git_config_get_int("merge.renamelimit", &opt->rename_limit);
 	if (!git_config_get_string("diff.renames", &value)) {
-		opt->diff_detect_rename = git_config_rename("diff.renames", value);
+		opt->detect_renames = git_config_rename("diff.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.renames", &value)) {
-		opt->merge_detect_rename = git_config_rename("merge.renames", value);
+		opt->detect_renames = git_config_rename("merge.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.directoryrenames", &value)) {
@@ -3741,11 +3738,9 @@ void init_merge_options(struct merge_options *opt,
 	opt->repo = repo;
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->diff_rename_limit = -1;
-	opt->merge_rename_limit = -1;
+	opt->rename_limit = -1;
 	opt->renormalize = 0;
-	opt->diff_detect_rename = -1;
-	opt->merge_detect_rename = -1;
+	opt->detect_renames = -1;
 	opt->detect_directory_renames = 1;
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
@@ -3797,16 +3792,16 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	else if (!strcmp(s, "no-renormalize"))
 		opt->renormalize = 0;
 	else if (!strcmp(s, "no-renames"))
-		opt->merge_detect_rename = 0;
+		opt->detect_renames = 0;
 	else if (!strcmp(s, "find-renames")) {
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 		opt->rename_score = 0;
 	}
 	else if (skip_prefix(s, "find-renames=", &arg) ||
 		 skip_prefix(s, "rename-threshold=", &arg)) {
 		if ((opt->rename_score = parse_rename_score(&arg)) == -1 || *arg != 0)
 			return -1;
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 	}
 	/*
 	 * Please update $__git_merge_strategy_options in
diff --git a/merge-recursive.h b/merge-recursive.h
index 1454772528..e63483b8db 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -23,10 +23,8 @@ struct merge_options {
 	long xdl_opts;
 	int verbosity;
 	int detect_directory_renames;
-	int diff_detect_rename;
-	int merge_detect_rename;
-	int diff_rename_limit;
-	int merge_rename_limit;
+	int detect_renames;
+	int rename_limit;
 	int rename_score;
 	int needed_rename_limit;
 	int show_rename_progress;
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 14/19] merge-recursive: comment and reorder the merge_options fields

[Elijah Newren]

The merge_options struct had lots of fields, making it a little
imposing, but the options naturally fall into multiple different groups.
Grouping similar options and adding a comment or two makes it easier to
read, easier for new folks to figure out which options are related, and
thus easier for them to find the options they need.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 16 +++++++++++-----
 merge-recursive.h | 34 +++++++++++++++++++++++-----------
 2 files changed, 34 insertions(+), 16 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0f8d451f2e..a5049b06a3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3735,21 +3735,27 @@ void init_merge_options(struct merge_options *opt,
 {
 	const char *merge_verbosity;
 	memset(opt, 0, sizeof(struct merge_options));
+
 	opt->repo = repo;
+
+	opt->detect_renames = -1;
+	opt->detect_directory_renames = 1;
+	opt->rename_limit = -1;
+
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->rename_limit = -1;
+	strbuf_init(&opt->obuf, 0);
+
 	opt->renormalize = 0;
-	opt->detect_renames = -1;
-	opt->detect_directory_renames = 1;
+
+	string_list_init(&opt->df_conflict_file_set, 1);
+
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
 		opt->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (opt->verbosity >= 5)
 		opt->buffer_output = 0;
-	strbuf_init(&opt->obuf, 0);
-	string_list_init(&opt->df_conflict_file_set, 1);
 }
 
 int parse_merge_opt(struct merge_options *opt, const char *s)
diff --git a/merge-recursive.h b/merge-recursive.h
index e63483b8db..d57fce0daa 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -9,32 +9,44 @@ struct commit;
 struct repository;
 
 struct merge_options {
+	struct repository *repo;
+
+	/* ref names used in console messages and conflict markers */
 	const char *ancestor;
 	const char *branch1;
 	const char *branch2;
+
+	/* rename related options */
+	int detect_renames;
+	int detect_directory_renames;
+	int rename_limit;
+	int rename_score;
+	int show_rename_progress;
+
+	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
+	long xdl_opts;
 	enum {
 		MERGE_RECURSIVE_NORMAL = 0,
 		MERGE_RECURSIVE_OURS,
 		MERGE_RECURSIVE_THEIRS
 	} recursive_variant;
-	const char *subtree_shift;
+
+	/* console output related options */
+	int verbosity;
 	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
+	struct strbuf obuf;     /* output buffer */
+
+	/* miscellaneous control options */
+	const char *subtree_shift;
 	unsigned renormalize : 1;
-	long xdl_opts;
-	int verbosity;
-	int detect_directory_renames;
-	int detect_renames;
-	int rename_limit;
-	int rename_score;
-	int needed_rename_limit;
-	int show_rename_progress;
+
+	/* internal fields used by the implementation (do NOT set these) */
 	int call_depth;
-	struct strbuf obuf;
+	int needed_rename_limit;
 	struct hashmap current_file_dir_set;
 	struct string_list df_conflict_file_set;
 	struct unpack_trees_options unpack_opts;
 	struct index_state orig_index;
-	struct repository *repo;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 15/19] merge-recursive: split internal fields into a separate struct

[Elijah Newren]

merge_options has several internal fields that should not be set or read
by external callers.  This just complicates the API.  Move them into an
opaque merge_options_internal struct that is defined only in
merge-recursive.c and keep these out of merge-recursive.h.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 185 +++++++++++++++++++++++++---------------------
 merge-recursive.h |  17 ++---
 2 files changed, 106 insertions(+), 96 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index a5049b06a3..c572d37b21 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -29,6 +29,15 @@
 #include "revision.h"
 #include "commit-reach.h"
 
+struct merge_options_internal {
+	int call_depth;
+	int needed_rename_limit;
+	struct hashmap current_file_dir_set;
+	struct string_list df_conflict_file_set;
+	struct unpack_trees_options unpack_opts;
+	struct index_state orig_index;
+};
+
 struct path_hashmap_entry {
 	struct hashmap_entry e;
 	char path[FLEX_ARRAY];
@@ -309,7 +318,8 @@ static inline void setup_rename_conflict_info(enum rename_type rename_type,
 
 static int show(struct merge_options *opt, int v)
 {
-	return (!opt->call_depth && opt->verbosity >= v) || opt->verbosity >= 5;
+	return (!opt->priv->call_depth && opt->verbosity >= v) ||
+		opt->verbosity >= 5;
 }
 
 __attribute__((format (printf, 3, 4)))
@@ -320,7 +330,7 @@ static void output(struct merge_options *opt, int v, const char *fmt, ...)
 	if (!show(opt, v))
 		return;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 
 	va_start(ap, fmt);
 	strbuf_vaddf(&opt->obuf, fmt, ap);
@@ -335,7 +345,7 @@ static void output_commit_title(struct merge_options *opt, struct commit *commit
 {
 	struct merge_remote_desc *desc;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 	desc = merge_remote_util(commit);
 	if (desc)
 		strbuf_addf(&opt->obuf, "virtual %s\n", desc->name);
@@ -403,43 +413,43 @@ static int unpack_trees_start(struct merge_options *opt,
 	struct tree_desc t[3];
 	struct index_state tmp_index = { NULL };
 
-	memset(&opt->unpack_opts, 0, sizeof(opt->unpack_opts));
-	if (opt->call_depth)
-		opt->unpack_opts.index_only = 1;
+	memset(&opt->priv->unpack_opts, 0, sizeof(opt->priv->unpack_opts));
+	if (opt->priv->call_depth)
+		opt->priv->unpack_opts.index_only = 1;
 	else
-		opt->unpack_opts.update = 1;
-	opt->unpack_opts.merge = 1;
-	opt->unpack_opts.head_idx = 2;
-	opt->unpack_opts.fn = threeway_merge;
-	opt->unpack_opts.src_index = opt->repo->index;
-	opt->unpack_opts.dst_index = &tmp_index;
-	opt->unpack_opts.aggressive = !merge_detect_rename(opt);
-	setup_unpack_trees_porcelain(&opt->unpack_opts, "merge");
+		opt->priv->unpack_opts.update = 1;
+	opt->priv->unpack_opts.merge = 1;
+	opt->priv->unpack_opts.head_idx = 2;
+	opt->priv->unpack_opts.fn = threeway_merge;
+	opt->priv->unpack_opts.src_index = opt->repo->index;
+	opt->priv->unpack_opts.dst_index = &tmp_index;
+	opt->priv->unpack_opts.aggressive = !merge_detect_rename(opt);
+	setup_unpack_trees_porcelain(&opt->priv->unpack_opts, "merge");
 
 	init_tree_desc_from_tree(t+0, common);
 	init_tree_desc_from_tree(t+1, head);
 	init_tree_desc_from_tree(t+2, merge);
 
-	rc = unpack_trees(3, t, &opt->unpack_opts);
+	rc = unpack_trees(3, t, &opt->priv->unpack_opts);
 	cache_tree_free(&opt->repo->index->cache_tree);
 
 	/*
-	 * Update opt->repo->index to match the new results, AFTER saving a copy
-	 * in opt->orig_index.  Update src_index to point to the saved copy.
-	 * (verify_uptodate() checks src_index, and the original index is
-	 * the one that had the necessary modification timestamps.)
+	 * Update opt->repo->index to match the new results, AFTER saving a
+	 * copy in opt->priv->orig_index.  Update src_index to point to the
+	 * saved copy.  (verify_uptodate() checks src_index, and the original
+	 * index is the one that had the necessary modification timestamps.)
 	 */
-	opt->orig_index = *opt->repo->index;
+	opt->priv->orig_index = *opt->repo->index;
 	*opt->repo->index = tmp_index;
-	opt->unpack_opts.src_index = &opt->orig_index;
+	opt->priv->unpack_opts.src_index = &opt->priv->orig_index;
 
 	return rc;
 }
 
 static void unpack_trees_finish(struct merge_options *opt)
 {
-	discard_index(&opt->orig_index);
-	clear_unpack_trees_porcelain(&opt->unpack_opts);
+	discard_index(&opt->priv->orig_index);
+	clear_unpack_trees_porcelain(&opt->priv->unpack_opts);
 }
 
 static int save_files_dirs(const struct object_id *oid,
@@ -454,7 +464,7 @@ static int save_files_dirs(const struct object_id *oid,
 
 	FLEX_ALLOC_MEM(entry, path, base->buf, base->len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 
 	strbuf_setlen(base, baselen);
 	return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);
@@ -585,7 +595,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	 * If we're merging merge-bases, we don't want to bother with
 	 * any working directory changes.
 	 */
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return;
 
 	/* Ensure D/F conflicts are adjacent in the entries list. */
@@ -597,7 +607,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	df_sorted_entries.cmp = string_list_df_name_compare;
 	string_list_sort(&df_sorted_entries);
 
-	string_list_clear(&opt->df_conflict_file_set, 1);
+	string_list_clear(&opt->priv->df_conflict_file_set, 1);
 	for (i = 0; i < df_sorted_entries.nr; i++) {
 		const char *path = df_sorted_entries.items[i].string;
 		int len = strlen(path);
@@ -613,7 +623,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 		    len > last_len &&
 		    memcmp(path, last_file, last_len) == 0 &&
 		    path[last_len] == '/') {
-			string_list_insert(&opt->df_conflict_file_set, last_file);
+			string_list_insert(&opt->priv->df_conflict_file_set, last_file);
 		}
 
 		/*
@@ -680,8 +690,8 @@ static void update_entry(struct stage_data *entry,
 static int remove_file(struct merge_options *opt, int clean,
 		       const char *path, int no_wd)
 {
-	int update_cache = opt->call_depth || clean;
-	int update_working_directory = !opt->call_depth && !no_wd;
+	int update_cache = opt->priv->call_depth || clean;
+	int update_working_directory = !opt->priv->call_depth && !no_wd;
 
 	if (update_cache) {
 		if (remove_file_from_index(opt->repo->index, path))
@@ -724,16 +734,16 @@ static char *unique_path(struct merge_options *opt,
 	add_flattened_path(&newpath, branch);
 
 	base_len = newpath.len;
-	while (hashmap_get_from_hash(&opt->current_file_dir_set,
+	while (hashmap_get_from_hash(&opt->priv->current_file_dir_set,
 				     path_hash(newpath.buf), newpath.buf) ||
-	       (!opt->call_depth && file_exists(newpath.buf))) {
+	       (!opt->priv->call_depth && file_exists(newpath.buf))) {
 		strbuf_setlen(&newpath, base_len);
 		strbuf_addf(&newpath, "_%d", suffix++);
 	}
 
 	FLEX_ALLOC_MEM(entry, path, newpath.buf, newpath.len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 	return strbuf_detach(&newpath, NULL);
 }
 
@@ -775,7 +785,7 @@ static int dir_in_way(struct index_state *istate, const char *path,
 static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 				   const struct diff_filespec *blob)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 	struct cache_entry *ce;
 
 	if (0 > pos)
@@ -783,7 +793,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 		return 0;
 
 	/* See if the file we were tracking before matches */
-	ce = opt->orig_index.cache[pos];
+	ce = opt->priv->orig_index.cache[pos];
 	return (oid_eq(&ce->oid, &blob->oid) && ce->ce_mode == blob->mode);
 }
 
@@ -792,7 +802,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
  */
 static int was_tracked(struct merge_options *opt, const char *path)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 
 	if (0 <= pos)
 		/* we were tracking this path before the merge */
@@ -849,12 +859,12 @@ static int was_dirty(struct merge_options *opt, const char *path)
 	struct cache_entry *ce;
 	int dirty = 1;
 
-	if (opt->call_depth || !was_tracked(opt, path))
+	if (opt->priv->call_depth || !was_tracked(opt, path))
 		return !dirty;
 
-	ce = index_file_exists(opt->unpack_opts.src_index,
+	ce = index_file_exists(opt->priv->unpack_opts.src_index,
 			       path, strlen(path), ignore_case);
-	dirty = verify_uptodate(ce, &opt->unpack_opts) != 0;
+	dirty = verify_uptodate(ce, &opt->priv->unpack_opts) != 0;
 	return dirty;
 }
 
@@ -864,8 +874,8 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 	const char *msg = _("failed to create path '%s'%s");
 
 	/* Unlink any D/F conflict files that are in the way */
-	for (i = 0; i < opt->df_conflict_file_set.nr; i++) {
-		const char *df_path = opt->df_conflict_file_set.items[i].string;
+	for (i = 0; i < opt->priv->df_conflict_file_set.nr; i++) {
+		const char *df_path = opt->priv->df_conflict_file_set.items[i].string;
 		size_t pathlen = strlen(path);
 		size_t df_pathlen = strlen(df_path);
 		if (df_pathlen < pathlen &&
@@ -875,7 +885,7 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 			       _("Removing %s to make room for subdirectory\n"),
 			       df_path);
 			unlink(df_path);
-			unsorted_string_list_delete_item(&opt->df_conflict_file_set,
+			unsorted_string_list_delete_item(&opt->priv->df_conflict_file_set,
 							 i, 0);
 			break;
 		}
@@ -916,7 +926,7 @@ static int update_file_flags(struct merge_options *opt,
 {
 	int ret = 0;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		update_wd = 0;
 
 	if (update_wd) {
@@ -1001,7 +1011,7 @@ static int update_file(struct merge_options *opt,
 		       const char *path)
 {
 	return update_file_flags(opt, contents, path,
-				 opt->call_depth || clean, !opt->call_depth);
+				 opt->priv->call_depth || clean, !opt->priv->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1030,7 +1040,7 @@ static int merge_3way(struct merge_options *opt,
 	ll_opts.extra_marker_size = extra_marker_size;
 	ll_opts.xdl_opts = opt->xdl_opts;
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		ll_opts.virtual_ancestor = 1;
 		ll_opts.variant = 0;
 	} else {
@@ -1164,7 +1174,7 @@ static int merge_submodule(struct merge_options *opt,
 	struct object_array merges;
 
 	int i;
-	int search = !opt->call_depth;
+	int search = !opt->priv->call_depth;
 
 	/* store a in result in case we fail */
 	oidcpy(result, a);
@@ -1385,7 +1395,7 @@ static int handle_rename_via_dir(struct merge_options *opt,
 	int mark_conflicted = (opt->detect_directory_renames == 1);
 	assert(ren->dir_rename_original_dest);
 
-	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
+	if (!opt->priv->call_depth && would_lose_untracked(opt, dest->path)) {
 		mark_conflicted = 1;
 		file_path = unique_path(opt, dest->path, ren->branch);
 		output(opt, 1, _("Error: Refusing to lose untracked file at %s; "
@@ -1428,12 +1438,12 @@ static int handle_change_delete(struct merge_options *opt,
 	const char *update_path = path;
 	int ret = 0;
 
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0) ||
-	    (!opt->call_depth && would_lose_untracked(opt, path))) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0) ||
+	    (!opt->priv->call_depth && would_lose_untracked(opt, path))) {
 		update_path = alt_path = unique_path(opt, path, change_branch);
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * We cannot arbitrarily accept either a_sha or b_sha as
 		 * correct; since there is no true "middle point" between
@@ -1508,14 +1518,14 @@ static int handle_rename_delete(struct merge_options *opt,
 				     opt->branch2 : opt->branch1);
 
 	if (handle_change_delete(opt,
-				 opt->call_depth ? orig->path : dest->path,
-				 opt->call_depth ? NULL : orig->path,
+				 opt->priv->call_depth ? orig->path : dest->path,
+				 opt->priv->call_depth ? NULL : orig->path,
 				 orig, dest,
 				 rename_branch, delete_branch,
 				 _("rename"), _("renamed")))
 		return -1;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return remove_file_from_index(opt->repo->index, dest->path);
 	else
 		return update_stages(opt, dest->path, NULL,
@@ -1552,7 +1562,7 @@ static int handle_file_collision(struct merge_options *opt,
 	/*
 	 * In the recursive case, we just opt to undo renames
 	 */
-	if (opt->call_depth && (prev_path1 || prev_path2)) {
+	if (opt->priv->call_depth && (prev_path1 || prev_path2)) {
 		/* Put first file (a->oid, a->mode) in its original spot */
 		if (prev_path1) {
 			if (update_file(opt, 1, a, prev_path1))
@@ -1581,10 +1591,10 @@ static int handle_file_collision(struct merge_options *opt,
 	/* Remove rename sources if rename/add or rename/rename(2to1) */
 	if (prev_path1)
 		remove_file(opt, 1, prev_path1,
-			    opt->call_depth || would_lose_untracked(opt, prev_path1));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path1));
 	if (prev_path2)
 		remove_file(opt, 1, prev_path2,
-			    opt->call_depth || would_lose_untracked(opt, prev_path2));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path2));
 
 	/*
 	 * Remove the collision path, if it wouldn't cause dirty contents
@@ -1626,12 +1636,12 @@ static int handle_file_collision(struct merge_options *opt,
 	null.mode = 0;
 
 	if (merge_mode_and_contents(opt, &null, a, b, collide_path,
-				    branch1, branch2, opt->call_depth * 2, &mfi))
+				    branch1, branch2, opt->priv->call_depth * 2, &mfi))
 		return -1;
 	mfi.clean &= !alt_path;
 	if (update_file(opt, mfi.clean, &mfi.blob, update_path))
 		return -1;
-	if (!mfi.clean && !opt->call_depth &&
+	if (!mfi.clean && !opt->priv->call_depth &&
 	    update_stages(opt, collide_path, NULL, a, b))
 		return -1;
 	free(alt_path);
@@ -1671,7 +1681,7 @@ static int handle_rename_add(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[other_stage],
 				    prev_path_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi))
+				    1 + opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(prev_path_desc);
 
@@ -1689,7 +1699,7 @@ static char *find_path_for_conflict(struct merge_options *opt,
 				    const char *branch2)
 {
 	char *new_path = NULL;
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0)) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0)) {
 		new_path = unique_path(opt, path, branch1);
 		output(opt, 1, _("%s is a directory in %s adding "
 			       "as %s instead"),
@@ -1720,17 +1730,17 @@ static int handle_rename_rename_1to2(struct merge_options *opt,
 	       "rename \"%s\"->\"%s\" in \"%s\"%s"),
 	       o->path, a->path, ci->ren1->branch,
 	       o->path, b->path, ci->ren2->branch,
-	       opt->call_depth ? _(" (left unresolved)") : "");
+	       opt->priv->call_depth ? _(" (left unresolved)") : "");
 
 	path_desc = xstrfmt("%s and %s, both renamed from %s",
 			    a->path, b->path, o->path);
 	if (merge_mode_and_contents(opt, o, a, b, path_desc,
 				    ci->ren1->branch, ci->ren2->branch,
-				    opt->call_depth * 2, &mfi))
+				    opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(path_desc);
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1845,12 +1855,12 @@ static int handle_rename_rename_2to1(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[ostage1],
 				    path_side_1_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c1) ||
+				    1 + opt->priv->call_depth * 2, &mfi_c1) ||
 	    merge_mode_and_contents(opt, b,
 				    &ci->ren2->src_entry->stages[ostage2],
 				    c2, path_side_2_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c2))
+				    1 + opt->priv->call_depth * 2, &mfi_c2))
 		return -1;
 	free(path_side_1_desc);
 	free(path_side_2_desc);
@@ -1891,8 +1901,8 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	diff_setup_done(&opts);
 	diff_tree_oid(&o_tree->object.oid, &tree->object.oid, "", &opts);
 	diffcore_std(&opts);
-	if (opts.needed_rename_limit > opt->needed_rename_limit)
-		opt->needed_rename_limit = opts.needed_rename_limit;
+	if (opts.needed_rename_limit > opt->priv->needed_rename_limit)
+		opt->priv->needed_rename_limit = opts.needed_rename_limit;
 
 	ret = xmalloc(sizeof(*ret));
 	*ret = diff_queued_diff;
@@ -3022,13 +3032,13 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		reason = _("add/add");
 
 	assert(o->path && a->path && b->path);
-	if (ci && dir_in_way(opt->repo->index, path, !opt->call_depth,
+	if (ci && dir_in_way(opt->repo->index, path, !opt->priv->call_depth,
 			     S_ISGITLINK(ci->ren1->pair->two->mode)))
 		df_conflict_remains = 1;
 
 	if (merge_mode_and_contents(opt, o, a, b, path,
 				    opt->branch1, opt->branch2,
-				    opt->call_depth * 2, mfi))
+				    opt->priv->call_depth * 2, mfi))
 		return -1;
 
 	/*
@@ -3044,7 +3054,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 		output(opt, 3, _("Skipped %s (merged same as existing)"), path);
 		if (add_cacheinfo(opt, &mfi->blob, path,
-				  0, (!opt->call_depth && !is_dirty), 0))
+				  0, (!opt->priv->call_depth && !is_dirty), 0))
 			return -1;
 		/*
 		 * However, add_cacheinfo() will delete the old cache entry
@@ -3052,8 +3062,8 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		 * flag to avoid making the file appear as if it were
 		 * deleted by the user.
 		 */
-		pos = index_name_pos(&opt->orig_index, path, strlen(path));
-		ce = opt->orig_index.cache[pos];
+		pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
+		ce = opt->priv->orig_index.cache[pos];
 		if (ce_skip_worktree(ce)) {
 			pos = index_name_pos(opt->repo->index, path, strlen(path));
 			ce = opt->repo->index->cache[pos];
@@ -3074,7 +3084,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 	if (df_conflict_remains || is_dirty) {
 		char *new_path;
-		if (opt->call_depth) {
+		if (opt->priv->call_depth) {
 			remove_file_from_index(opt->repo->index, path);
 		} else {
 			if (!mfi->clean) {
@@ -3332,7 +3342,7 @@ static int process_entry(struct merge_options *opt,
 			conf = _("directory/file");
 		}
 		if (dir_in_way(opt->repo->index, path,
-			       !opt->call_depth && !S_ISGITLINK(a->mode),
+			       !opt->priv->call_depth && !S_ISGITLINK(a->mode),
 			       0)) {
 			char *new_path = unique_path(opt, path, add_branch);
 			clean_merge = 0;
@@ -3341,7 +3351,7 @@ static int process_entry(struct merge_options *opt,
 			       conf, path, other_branch, path, new_path);
 			if (update_file(opt, 0, contents, new_path))
 				clean_merge = -1;
-			else if (opt->call_depth)
+			else if (opt->priv->call_depth)
 				remove_file_from_index(opt->repo->index, path);
 			free(new_path);
 		} else {
@@ -3406,7 +3416,7 @@ static int merge_trees_internal(struct merge_options *opt,
 	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
-		if (show(opt, 4) || opt->call_depth)
+		if (show(opt, 4) || opt->priv->call_depth)
 			err(opt, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
@@ -3425,7 +3435,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+		hashmap_init(&opt->priv->current_file_dir_set, path_hashmap_cmp,
 			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
@@ -3462,7 +3472,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		string_list_clear(entries, 1);
 		free(entries);
 
-		hashmap_free(&opt->current_file_dir_set, 1);
+		hashmap_free(&opt->priv->current_file_dir_set, 1);
 
 		if (clean < 0) {
 			unpack_trees_finish(opt);
@@ -3474,7 +3484,8 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth && !(*result = write_tree_from_memory(opt->repo)))
+	if (opt->priv->call_depth &&
+	    !(*result = write_tree_from_memory(opt->repo)))
 		return -1;
 
 	return clean;
@@ -3538,7 +3549,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 
 	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
-		opt->call_depth++;
+		opt->priv->call_depth++;
 		/*
 		 * When the merge fails, the result contains files
 		 * with conflict markers. The cleanness flag is
@@ -3557,14 +3568,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
-		opt->call_depth--;
+		opt->priv->call_depth--;
 
 		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
 	discard_index(opt->repo->index);
-	if (!opt->call_depth)
+	if (!opt->priv->call_depth)
 		repo_read_index(opt->repo);
 
 	opt->ancestor = "merged common ancestors";
@@ -3579,14 +3590,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 		return clean;
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		*result = make_virtual_commit(opt->repo, result_tree,
 					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
 	flush_output(opt);
-	if (!opt->call_depth && opt->buffer_output < 2)
+	if (!opt->priv->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
 	return clean;
 }
@@ -3603,6 +3614,8 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 		return -1;
 	}
 
+	opt->priv = xcalloc(1, sizeof(*opt->priv));
+	string_list_init(&opt->priv->df_conflict_file_set, 1);
 	return 0;
 }
 
@@ -3610,7 +3623,9 @@ static void merge_finalize(struct merge_options *opt)
 {
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0);
+	free(opt->priv);
+	opt->priv = NULL;
 }
 
 int merge_trees(struct merge_options *opt,
@@ -3748,8 +3763,6 @@ void init_merge_options(struct merge_options *opt,
 
 	opt->renormalize = 0;
 
-	string_list_init(&opt->df_conflict_file_set, 1);
-
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
diff --git a/merge-recursive.h b/merge-recursive.h
index d57fce0daa..a1bb29dc33 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -1,13 +1,15 @@
 #ifndef MERGE_RECURSIVE_H
 #define MERGE_RECURSIVE_H
 
-#include "string-list.h"
-#include "unpack-trees.h"
+#include "strbuf.h"
 
 struct commit;
-
+struct commit_list;
+struct object_id;
 struct repository;
+struct tree;
 
+struct merge_options_internal;
 struct merge_options {
 	struct repository *repo;
 
@@ -40,13 +42,8 @@ struct merge_options {
 	const char *subtree_shift;
 	unsigned renormalize : 1;
 
-	/* internal fields used by the implementation (do NOT set these) */
-	int call_depth;
-	int needed_rename_limit;
-	struct hashmap current_file_dir_set;
-	struct string_list df_conflict_file_set;
-	struct unpack_trees_options unpack_opts;
-	struct index_state orig_index;
+	/* internal fields used by the implementation */
+	struct merge_options_internal *priv;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 16/19] merge-recursive: alphabetize include list

[Elijah Newren]

Other than cache.h which needs to appear first, and merge-recursive.h
which I want to be second so that we are more likely to notice if
merge-recursive.h has any missing includes, the rest of the list is
long and easier to look through if it's alphabetical.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index c572d37b21..32889a2f0d 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -4,30 +4,31 @@
  * The thieves were Alex Riesen and Johannes Schindelin, in June/July 2006
  */
 #include "cache.h"
-#include "config.h"
+#include "merge-recursive.h"
+
 #include "advice.h"
-#include "lockfile.h"
-#include "cache-tree.h"
-#include "object-store.h"
-#include "repository.h"
-#include "commit.h"
+#include "alloc.h"
+#include "attr.h"
 #include "blob.h"
 #include "builtin.h"
-#include "tree-walk.h"
+#include "cache-tree.h"
+#include "commit.h"
+#include "commit-reach.h"
+#include "config.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "dir.h"
+#include "ll-merge.h"
+#include "lockfile.h"
+#include "object-store.h"
+#include "repository.h"
+#include "revision.h"
+#include "string-list.h"
+#include "submodule.h"
 #include "tag.h"
-#include "alloc.h"
+#include "tree-walk.h"
 #include "unpack-trees.h"
-#include "string-list.h"
 #include "xdiff-interface.h"
-#include "ll-merge.h"
-#include "attr.h"
-#include "merge-recursive.h"
-#include "dir.h"
-#include "submodule.h"
-#include "revision.h"
-#include "commit-reach.h"
 
 struct merge_options_internal {
 	int call_depth;
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 17/19] merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*

[Elijah Newren]

I want to implement the same outward facing API as found within
merge-recursive.h in a different merge strategy.  However, that makes
names like MERGE_RECURSIVE_{NORMAL,OURS,THEIRS} look a little funny;
rename to MERGE_VARIANT_{NORMAL,OURS,THEIRS}.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 14 +++++++-------
 merge-recursive.h |  6 +++---
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 32889a2f0d..0f689095d9 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1046,10 +1046,10 @@ static int merge_3way(struct merge_options *opt,
 		ll_opts.variant = 0;
 	} else {
 		switch (opt->recursive_variant) {
-		case MERGE_RECURSIVE_OURS:
+		case MERGE_VARIANT_OURS:
 			ll_opts.variant = XDL_MERGE_FAVOR_OURS;
 			break;
-		case MERGE_RECURSIVE_THEIRS:
+		case MERGE_VARIANT_THEIRS:
 			ll_opts.variant = XDL_MERGE_FAVOR_THEIRS;
 			break;
 		default:
@@ -1359,15 +1359,15 @@ static int merge_mode_and_contents(struct merge_options *opt,
 							&b->oid);
 		} else if (S_ISLNK(a->mode)) {
 			switch (opt->recursive_variant) {
-			case MERGE_RECURSIVE_NORMAL:
+			case MERGE_VARIANT_NORMAL:
 				oidcpy(&result->blob.oid, &a->oid);
 				if (!oid_eq(&a->oid, &b->oid))
 					result->clean = 0;
 				break;
-			case MERGE_RECURSIVE_OURS:
+			case MERGE_VARIANT_OURS:
 				oidcpy(&result->blob.oid, &a->oid);
 				break;
-			case MERGE_RECURSIVE_THEIRS:
+			case MERGE_VARIANT_THEIRS:
 				oidcpy(&result->blob.oid, &b->oid);
 				break;
 			}
@@ -3779,9 +3779,9 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	if (!s || !*s)
 		return -1;
 	if (!strcmp(s, "ours"))
-		opt->recursive_variant = MERGE_RECURSIVE_OURS;
+		opt->recursive_variant = MERGE_VARIANT_OURS;
 	else if (!strcmp(s, "theirs"))
-		opt->recursive_variant = MERGE_RECURSIVE_THEIRS;
+		opt->recursive_variant = MERGE_VARIANT_THEIRS;
 	else if (!strcmp(s, "subtree"))
 		opt->subtree_shift = "";
 	else if (skip_prefix(s, "subtree=", &arg))
diff --git a/merge-recursive.h b/merge-recursive.h
index a1bb29dc33..c4e6fb85f1 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -28,9 +28,9 @@ struct merge_options {
 	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
 	long xdl_opts;
 	enum {
-		MERGE_RECURSIVE_NORMAL = 0,
-		MERGE_RECURSIVE_OURS,
-		MERGE_RECURSIVE_THEIRS
+		MERGE_VARIANT_NORMAL = 0,
+		MERGE_VARIANT_OURS,
+		MERGE_VARIANT_THEIRS
 	} recursive_variant;
 
 	/* console output related options */
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 18/19] merge-recursive: be consistent with assert

[Elijah Newren]

In commit 8daec1df03de ("merge-recursive: switch from (oid,mode) pairs
to a diff_filespec", 2019-04-05), an assertion on a->path && b->path
was added for code readability to document that these both needed to be
non-NULL at this point in the code.  However, the subsequent lines also
read o->path, so it should be included in the assert.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0f689095d9..4cd6599296 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1058,7 +1058,7 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path);
+	assert(a->path && b->path && o->path);
 	if (strcmp(a->path, b->path) ||
 	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
 		base_name = opt->ancestor == NULL ? NULL :
-- 
2.22.0.559.g28a8880890.dirty

[PATCH 19/19] merge-recursive: provide a better label for diff3 common ancestor

[Elijah Newren]

In commit 7ca56aa07619 ("merge-recursive: add a label for ancestor",
2010-03-20), a label was added for the '||||||' line to make it have
the more informative heading '|||||| merged common ancestors', with
the statement:

    It would be nicer to use a more informative label.  Perhaps someone
    will provide one some day.

This chosen label was perfectly reasonable when recursiveness kicks in,
i.e. when there are multiple merge bases.  (I can't think of a better
label in such cases.)  But it is actually somewhat misleading when there
is a unique merge base or no merge base.  Change this based on the
number of merge bases:
    >=2: "merged common ancestors"
    1:   <abbreviated commit hash>
    0:   "<empty tree>"

Also, the code in merge_3way making use of opt->ancestor was overly
complex because it tried to handle the case of opt->ancestor being NULL.
We always set it first, though, so just add an assert that opt->ancestor
is not NULL and simplify the surrounding code.

Tests have also been added to check that we get the right ancestor name
for each of the three cases.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |  35 ++++--
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 191 ++++++++++++++++++++++++++++++
 3 files changed, 220 insertions(+), 14 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index 4cd6599296..8ac53cacdf 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1034,7 +1034,7 @@ static int merge_3way(struct merge_options *opt,
 {
 	mmfile_t orig, src1, src2;
 	struct ll_merge_options ll_opts = {0};
-	char *base_name, *name1, *name2;
+	char *base, *name1, *name2;
 	int merge_status;
 
 	ll_opts.renormalize = opt->renormalize;
@@ -1058,16 +1058,13 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path && o->path);
-	if (strcmp(a->path, b->path) ||
-	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s:%s", opt->ancestor, o->path);
+	assert(a->path && b->path && o->path && opt->ancestor);
+	if (strcmp(a->path, b->path) || strcmp(a->path, o->path) != 0) {
+		base  = mkpathdup("%s:%s", opt->ancestor, o->path);
 		name1 = mkpathdup("%s:%s", branch1, a->path);
 		name2 = mkpathdup("%s:%s", branch2, b->path);
 	} else {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s", opt->ancestor);
+		base  = mkpathdup("%s", opt->ancestor);
 		name1 = mkpathdup("%s", branch1);
 		name2 = mkpathdup("%s", branch2);
 	}
@@ -1076,11 +1073,11 @@ static int merge_3way(struct merge_options *opt,
 	read_mmblob(&src1, &a->oid);
 	read_mmblob(&src2, &b->oid);
 
-	merge_status = ll_merge(result_buf, a->path, &orig, base_name,
+	merge_status = ll_merge(result_buf, a->path, &orig, base,
 				&src1, name1, &src2, name2,
 				opt->repo->index, &ll_opts);
 
-	free(base_name);
+	free(base);
 	free(name1);
 	free(name2);
 	free(orig.ptr);
@@ -3517,6 +3514,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 	struct commit *merged_merge_bases;
 	struct tree *result_tree;
 	int clean;
+	int num_merge_bases;
+	struct strbuf commit_name = STRBUF_INIT;
 
 	if (show(opt, 4)) {
 		output(opt, 4, _("Merging:"));
@@ -3538,6 +3537,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 			output_commit_title(opt, iter->item);
 	}
 
+	num_merge_bases = commit_list_count(merge_bases);
 	merged_merge_bases = pop_commit(&merge_bases);
 	if (merged_merge_bases == NULL) {
 		/* if there is no common ancestor, use an empty tree */
@@ -3579,13 +3579,26 @@ static int merge_recursive_internal(struct merge_options *opt,
 	if (!opt->priv->call_depth)
 		repo_read_index(opt->repo);
 
-	opt->ancestor = "merged common ancestors";
+	switch (num_merge_bases) {
+	case 0:
+		opt->ancestor = "<empty tree>";
+		break;
+	case 1:
+		strbuf_add_unique_abbrev(&commit_name,
+					 &merged_merge_bases->object.oid,
+					 DEFAULT_ABBREV);
+		opt->ancestor = commit_name.buf;
+		break;
+	default:
+		opt->ancestor = "merged common ancestors";
+	}
 	clean = merge_trees_internal(opt,
 				     repo_get_commit_tree(opt->repo, h1),
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
 							  merged_merge_bases),
 				     &result_tree);
+	strbuf_release(&commit_name);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index d23b948f27..7fddcc8c73 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -1562,6 +1562,7 @@ test_expect_success 'check nested conflicts' '
 		cd nested_conflicts &&
 
 		git clean -f &&
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L2^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1582,7 +1583,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:a >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1594,7 +1595,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:b >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1732,6 +1733,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 	(
 		cd virtual_merge_base_has_nested_conflicts &&
 
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L3^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1760,7 +1762,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 		cp left merged-once &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			merged-once \
 			base        \
diff --git a/t/t6047-diff3-conflict-markers.sh b/t/t6047-diff3-conflict-markers.sh
new file mode 100755
index 0000000000..08cb1668a5
--- /dev/null
+++ b/t/t6047-diff3-conflict-markers.sh
@@ -0,0 +1,191 @@
+#!/bin/sh
+
+test_description='recursive merge diff3 style conflict markers'
+
+. ./test-lib.sh
+
+# Setup:
+#          L1
+#            \
+#             ?
+#            /
+#          R1
+#
+# Where:
+#   L1 and R1 both have a file named 'content' but have no common history
+#
+
+test_expect_success 'setup no merge base' '
+	test_create_repo no_merge_base &&
+	(
+		cd no_merge_base &&
+
+		git checkout --orphan L &&
+		test_seq 1 9 >content &&
+		echo "A" >>content &&
+		git add content &&
+		test_tick &&
+		git commit -m "version L1 of content" &&
+
+		# Create R
+		git checkout --orphan R &&
+		test_seq 1 9 >content &&
+		echo "10" >>content &&
+		git add content &&
+		test_tick &&
+		git commit -m "version R1 of content"
+	)
+'
+
+test_expect_success 'check no merge base' '
+	(
+		cd no_merge_base &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge --allow-unrelated-histories -s recursive R^0 &&
+
+		grep "|||||| <empty tree>" content
+	)
+'
+
+# Setup:
+#          L1
+#         /  \
+#   master    ?
+#         \  /
+#          R1
+#
+# Where:
+#   L1 and R1 have modified the same file ('content') in conflicting ways
+#
+
+test_expect_success 'setup unique merge base' '
+	test_create_repo unique_merge_base &&
+	(
+		cd unique_merge_base &&
+
+		test_seq 1 9 >content &&
+		git add content &&
+		test_tick &&
+		git commit -m initial &&
+
+		git branch L &&
+		git branch R &&
+
+		# Create L1
+		git checkout L &&
+		test_seq 0 9 >content &&
+		echo "A" >>content &&
+		git add content &&
+		test_tick &&
+		git commit -m "version L1 of content" &&
+
+		# Create R1
+		git checkout R &&
+		test_seq 0 9 >content &&
+		echo "ten" >>content &&
+		git add content &&
+		git mv content renamed &&
+		test_tick &&
+		git commit -m "version R1 of content"
+	)
+'
+
+test_expect_success 'check unique merge base' '
+	(
+		cd unique_merge_base &&
+
+		git checkout L^0 &&
+		MASTER=$(git rev-parse --short master) &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| $MASTER:content" renamed
+	)
+'
+
+# Setup:
+#          L1---L2--L3
+#         /  \ /      \
+#   master    X1       ?
+#         \  / \      /
+#          R1---R2--R3
+#
+# Where:
+#   commits L1 and R1 have modified the same file in non-conflicting ways
+#   X1 is an auto-generated merge-base used when merging L1 and R1
+#   commits L2 and R2 are merges of R1 and L1 into L1 and R1, respectively
+#   commits L3 and R3 both modify 'content' in conflicting ways
+#
+
+test_expect_success 'setup multiple merge bases' '
+	test_create_repo multiple_merge_bases &&
+	(
+		cd multiple_merge_bases &&
+
+		# Create some related files now
+		test_seq 1 9 >content &&
+		git add content &&
+		test_tick &&
+		git commit -m initial &&
+
+		git branch L &&
+		git branch R &&
+
+		# Create L1
+		git checkout L &&
+		test_seq 0 9 >content &&
+		git add content &&
+		test_tick &&
+		git commit -m "version L1 of content" &&
+		git tag L1 &&
+
+		# Create R1
+		git checkout R &&
+		test_seq 1 10 >content &&
+		git add content &&
+		test_tick &&
+		git commit -m "verson R1 of content" &&
+		git tag R1 &&
+
+		# Create L2
+		git checkout L &&
+		git merge R1 &&
+
+		# Create R2
+		git checkout R &&
+		git merge L1 &&
+
+		# Create L3
+		git checkout L &&
+		test_seq 0 9 >content &&
+		echo "A" >>content &&
+		git add content &&
+		test_tick &&
+		git commit -m "version L3 of content" &&
+
+		# Create R3
+		git checkout R &&
+		test_seq 0 9 >content &&
+		echo "ten" >>content &&
+		git add content &&
+		git mv content renamed &&
+		test_tick &&
+		git commit -m "version R3 of content"
+	)
+'
+
+test_expect_success 'check multiple merge bases' '
+	(
+		cd multiple_merge_bases &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| merged common ancestors:content" renamed
+	)
+'
+
+test_done
-- 
2.22.0.559.g28a8880890.dirty

Re: [PATCH 00/19] Cleanup merge API

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Before writing a replacement merge strategy for recursive, I decided
> to first cleanup the merge API -- streamlining merge-recursive.h and
> making it more readable.  It includes some fixes I noticed along the
> way, and the last two patches were some forgotten changes of mine I
> rediscovered that had minor textual conflicts before I rebased them on
> this series.
>
>     While there are minor textual and semantic dependencies between
>     these patches (preventing me from splitting up this series), they
>     are logically separate and can be reviewed independently.

Nice.

> Stuff I'd most welcome review on:
>   * Is cache-tree.c the right place for write_tree_from_memory()?
>     [see patch 7]  Should there be docs on how it differs from
>     write_index_as_tree(), already found in cache-tree?  What does
>     the latter even do?

write_index_as_tree() is supposed to write the contents of an index
state as a tree object, and return the object ID for the resulting
tree.  It is the primary interface designed to be used by
write-tree.

> Stuff I didn't address:
>   * All current callers (3 of them?) of merge_recursive() always pass
>     ... I'm guessing the first did in an attempt to
>     exactly match the git-merge-recursive.py scripts' behavior.

That matches my recollection.  

Re: [PATCH 00/19] Cleanup merge API

[Elijah Newren]

On Thu, Jul 25, 2019 at 11:12 AM Junio C Hamano <gitster@pobox.com> wrote:

> > Stuff I'd most welcome review on:
> >   * Is cache-tree.c the right place for write_tree_from_memory()?
> >     [see patch 7]  Should there be docs on how it differs from
> >     write_index_as_tree(), already found in cache-tree?  What does
> >     the latter even do?
>
> write_index_as_tree() is supposed to write the contents of an index
> state as a tree object, and return the object ID for the resulting
> tree.  It is the primary interface designed to be used by
> write-tree.

Other than the last sentence, that also sounds like the description of
write_index_as_tree() -- at least as best I understood it.  But I
don't understand enough of the cache-tree and locking code to
determine if these are accidental duplicates of the same functionality
(meaning we could delete one, or rewrite one in terms of the other),
or whether there's a useful distinction between the two.  I guess it
doesn't hurt for now to have both, but I was curious if perhaps one
was more robust, and wanted to at least flag it for someone to take a
look.

Re: [PATCH 00/19] Cleanup merge API

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

>   * All current callers (3 of them?) of merge_recursive() always pass
>     it a specially created reversed-list for the merge_bases.  Some
>     history spelunking provides no details on any of these about why;
>     it appears that the 2nd and 3rd callers reversed the list because
>     the first did, and I'm guessing the first did in an attempt to
>     exactly match the git-merge-recursive.py scripts' behavior.

That is part of the reason.

After I worked on converting that Python script with Alex Riesen, I
tested it on all merge commits I could find at that point in linux.git,
to compare the outcomes between scripted and non-scripted versions. IIRC
I even reported those findings back to the mailing list.

*clickety-click*

Ah, I did the extensive test on git.git first [*1*]:
https://public-inbox.org/git/Pine.LNX.4.63.0608092232000.13885@wbgn013.biozentrum.uni-wuerzburg.de/

As you can see, it seemed that the reverse order of merge bases (trying
to merge the oldest two merge bases first, then merging with the
3rd-oldest, etc) avoided more merge conflicts than the original order.

>     But if the API needs them in a reverse order from what people
>     would normally expect to pass them in, shouldn't it reverse them
>     itself instead of making all callers do it?

I would not, as the order the merge bases are passed in defines the
order in which they are handled.

>     Also, the order shouldn't matter when there are no conflicts, and
>     when there are conflicts it'd only change which side of the
>     conflict markers the made-up virtual merge base would list things
>     in.

That's what I thought, right up until I re-created those merge commits.

I really forgot most of the details, but I seem to remember that there
was a puzzling one where the reverse order caused no merge conflicts,
and the original order caused a double merge conflict.

>     However, we do have tests with recursive virtual merge bases
>     and which test the output, and I didn't want to try to clean those
>     all up.  Besides, the current order shows nicely when commits are
>     named things like "L1", "L2", "R1", "R2" -- it's nice having a
>     well defined left and right side.  Wasn't sure what to do yet, so
>     I just punted for now; this series is already long enough...

Good. I think you would have entered an unpleasant space if you had run
down that rabbit hole ;-)

Ciao,
Dscho

Footnote *1*: I did remember correctly, though, that I re-created all
merge commits of then-current linux.git:
https://public-inbox.org/git/Pine.LNX.4.63.0608131550290.10541@wbgn013.biozentrum.uni-wuerzburg.de/

Re: [PATCH 03/19] Ensure index matches head before invoking merge machinery, round N

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> This is the bug that just won't die; there always seems to be another
> form of it somewhere.  See the commit message of 55f39cf7551b ("merge:
> fix misleading pre-merge check documentation", 2018-06-30) for a more
> detailed explanation), but in short:
>
> <quick summary>
>
> builtin/merge.c contains this important requirement for merge
> strategies:
>
>     ...the index must be in sync with the head commit.  The strategies are
>     responsible to ensure this.
>
> This condition is important to enforce because there are two likely
> failure cases when the index isn't in sync with the head commit:
>
>   * we silently throw away changes the user had staged before the merge
>
>   * we accidentally (and silently) include changes in the merge that
>     were not part of either of the branches/trees being merged
>
> Discarding users' work and mis-merging are both bad outcomes, especially
> when done silently, so naturally this rule was stated sternly -- but,
> unfortunately totally ignored in practice unless and until actual bugs
> were found.  But, fear not: the bugs from this were fixed in commit
>   ee6566e8d70d ("[PATCH] Rewrite read-tree", 2005-09-05)
> through a rewrite of read-tree (again, commit 55f39cf7551b has a more
> detailed explanation of how this affected merge).  And it was fixed
> again in commit
>   160252f81626 ("git-merge-ours: make sure our index matches HEAD", 2005-11-03)
> ...and it was fixed again in commit
>   3ec62ad9ffba ("merge-octopus: abort if index does not match HEAD", 2016-04-09)
> ...and again in commit
>   65170c07d466 ("merge-recursive: avoid incorporating uncommitted changes in a merge", 2017-12-21)
> ...and again in commit
>   eddd1a411d93 ("merge-recursive: enforce rule that index matches head before merging", 2018-06-30)
>
> ...with multiple testcases added to the testsuite that could be
> enumerated in even more commits.
>
> Then, finally, in a patch in the same series as the last fix above, the
> documentation about this requirement was fixed in commit 55f39cf7551b
> ("merge: fix misleading pre-merge check documentation", 2018-06-30), and
> we all lived happily ever after...
>
> </quick summary>

Whoa. What a story.

> Unfortunately, "ever after" apparently denotes a limited time and it
> expired today.  The merge-recursive rule to enforce that index matches
> head was at the beginning of merge_trees() and would only trigger when
> opt->call_depth was 0.  Since merge_recursive() doesn't call
> merge_trees() until after returning from recursing, this meant that the
> check wasn't triggered by merge_recursive() until it had first finished
> all the intermediate merges to create virtual merge bases.  That is a
> potentially HUGE amount of computation (and writing of intermediate
> merge results into the .git/objects directory) before it errors out and
> says, in effect, "Sorry, I can't do any merging because you have some
> local changes that would be overwritten."
>
> Trying to enforce that all of merge_trees(), merge_recursive(), and
> merge_recursive_generic() checked the index == head condition earlier
> resulted in a bunch of broken tests.  It turns out that
> merge_recursive() has code to drop and reload the cache while recursing
> to create intermediate virtual merge bases, but unfortunately that code
> runs even when no recursion is necessary.  This unconditional dropping
> and reloading of the cache masked a few bugs:
>
>   * builtin/merge-recursive.c: didn't even bother loading the index.
>
>   * builtin/stash.c: feels like a fake 'builtin' because it repeatedly
>     invokes git subprocesses all over the place, mixed with other
>     operations.  In particular, invoking "git reset" will reset the
>     index on disk, but the parent process that invoked it won't
>     automatically have its in-memory index updated.

Yep, the idea was to move fast to a built-in, and then continue by
converting all those process spawns to proper calls to libgit "API"
functions.

Sadly, that did not happen yet.

And you're absolutely right that failing to re-read the index after
spawning a `reset --hard` causes problems. IIRC I fixed them all,
though, see
https://public-inbox.org/git/nycvar.QRO.7.76.6.1902191127420.41@tvgsbejvaqbjf.bet/

> So, load the index in builtin/merge-recursive.c, reload the in-memory
> index in builtin/stash.c, and modify the t3030 testcase to correctly
> setup the index and make sure that the test fails in the expected way
> (meaning it reports a rename/rename conflict).

Makes sense to me.

> diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
> index 5b910e351e..a4bfd8fc51 100644
> --- a/builtin/merge-recursive.c
> +++ b/builtin/merge-recursive.c
> @@ -1,3 +1,4 @@
> +#include "cache.h"
>  #include "builtin.h"
>  #include "commit.h"
>  #include "tag.h"
> @@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
>  	if (argc - i != 3) /* "--" "<head>" "<remote>" */
>  		die(_("not handling anything other than two heads merge."));
>
> +	if (repo_read_index_unmerged(the_repository))
> +		die_resolve_conflict("merge");

For a moment I was unsure whether `_unmerged()` is the right thing to do
here, as it specifically allows to read the index even when there are
conflict stages. But I guess it does not matter too much here. I
probably would have opted for `repo_read_index()` instead, though.

> +
>  	o.branch1 = argv[++i];
>  	o.branch2 = argv[++i];
>
> diff --git a/builtin/stash.c b/builtin/stash.c
> index fde6397caa..bec011c1bb 100644
> --- a/builtin/stash.c
> +++ b/builtin/stash.c
> @@ -427,6 +427,8 @@ static int do_apply_stash(const char *prefix, struct stash_info *info,
>  				return error(_("could not save index tree"));
>
>  			reset_head();
> +			discard_cache();
> +			read_cache();

I was honestly puzzled why this is necessary, at first. The preceding
context expands to this:

                        discard_cache();
                        read_cache();
                        if (write_cache_as_tree(&index_tree, 0, NULL))
                                return error(_("could not save index tree"));

So basically, we already discard the index, read it again, then write it
as a tree, then reset and then we have to discard the index again?

But of course, if there are uncommitted changes, this would write a tree
different from HEAD, then reset the index to match HEAD, so indeed, this
discard/read dance is necessary.

So this hunk is good.

>  		}
>  	}
>
> diff --git a/t/t3030-merge-recursive.sh b/t/t3030-merge-recursive.sh
> index ff641b348a..a37bcc58a0 100755
> --- a/t/t3030-merge-recursive.sh
> +++ b/t/t3030-merge-recursive.sh
> @@ -667,15 +667,22 @@ test_expect_success 'merging with triple rename across D/F conflict' '
>  test_expect_success 'merge-recursive remembers the names of all base trees' '
>  	git reset --hard HEAD &&
>
> +	# make the index match $c1 so that merge-recursive below does not
> +	# fail early
> +	git diff --binary HEAD $c1 -- | git apply --cached &&
> +
>  	# more trees than static slots used by oid_to_hex()
>  	for commit in $c0 $c2 $c4 $c5 $c6 $c7
>  	do
>  		git rev-parse "$commit^{tree}"
>  	done >trees &&
>
> -	# ignore the return code -- it only fails because the input is weird
> +	# ignore the return code; it only fails because the input is weird...
>  	test_must_fail git -c merge.verbosity=5 merge-recursive $(cat trees) -- $c1 $c3 >out &&
>
> +	# ...but make sure it fails in the expected way
> +	test_i18ngrep CONFLICT.*rename/rename out &&
> +

This is obviously a good change: it strengthens the test case by fixing
a subtle bug.

Thanks,
Dscho

>  	# merge-recursive prints in reverse order, but we do not care
>  	sort <trees >expect &&
>  	sed -n "s/^virtual //p" out | sort >actual &&
> --
> 2.22.0.559.g28a8880890.dirty
>
>

Re: [PATCH 04/19] merge-recursive: exit early if index != head

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> We had a rule to enforce that the index matches head, but it was found
> at the beginning of merge_trees() and would only trigger when
> opt->call_depth was 0.  Since merge_recursive() doesn't call
> merge_trees() until after returning from recursing, this meant that the
> check wasn't triggered by merge_recursive() until it had first finished
> all the intermediate merges to create virtual merge bases.  That is a
> potentially huge amount of computation (and writing of intermediate
> merge results into the .git/objects directory) before it errors out and
> says, in effect, "Sorry, I can't do any merging because you have some
> local changes that would be overwritten."
>
> Further, not enforcing this requirement earlier allowed other bugs (such
> as an unintentional unconditional dropping and reloading of the index in
> merge_recursive() even when no recursion was necessary), to mask bugs in
> other callers (which were fixed in the commit prior to this one).
>
> Make sure we do the index == head check at the beginning of the merge,
> and error out immediately if it fails.

Very clear commit message.

> diff --git a/merge-recursive.c b/merge-recursive.c
> index 37bb94fb4d..b762ecd7bd 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3381,21 +3381,14 @@ static int process_entry(struct merge_options *opt,
>  	return clean_merge;
>  }
>
> -int merge_trees(struct merge_options *opt,
> -		struct tree *head,
> -		struct tree *merge,
> -		struct tree *common,
> -		struct tree **result)
> +static int merge_trees_internal(struct merge_options *opt,

In other, similar cases, we seem to use the suffix `_1`. Not sure
whether you want to change that here.

> +				struct tree *head,
> +				struct tree *merge,
> +				struct tree *common,
> +				struct tree **result)
>  {
>  	struct index_state *istate = opt->repo->index;
>  	int code, clean;
> -	struct strbuf sb = STRBUF_INIT;
> -
> -	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
> -		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
> -		    sb.buf);
> -		return -1;
> -	}
>
>  	if (opt->subtree_shift) {
>  		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
> @@ -3499,11 +3492,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
>   * Merge the commits h1 and h2, return the resulting virtual
>   * commit object and a flag indicating the cleanness of the merge.
>   */
> -int merge_recursive(struct merge_options *opt,
> -		    struct commit *h1,
> -		    struct commit *h2,
> -		    struct commit_list *ca,
> -		    struct commit **result)
> +static int merge_recursive_internal(struct merge_options *opt,

Same here.

> +				    struct commit *h1,
> +				    struct commit *h2,
> +				    struct commit_list *ca,
> +				    struct commit **result)
>  {
>  	struct commit_list *iter;
>  	struct commit *merged_common_ancestors;
> [...]
> @@ -3596,6 +3589,58 @@ int merge_recursive(struct merge_options *opt,
>  	return clean;
>  }
>
> +static int merge_start(struct merge_options *opt, struct tree *head)

I would prefer to call this something like `check_invariants()` or
something similar.

> +{
> +	struct strbuf sb = STRBUF_INIT;
> +
> +	assert(opt->branch1 && opt->branch2);
> +
> +	if (repo_index_has_changes(opt->repo, head, &sb)) {
> +		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
> +		    sb.buf);

I know that you did not introduce this leak, but maybe we could slip an
`strbuf_release(&sb);` in at this point?

> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
> +static void merge_finalize(struct merge_options *opt)
> +{
> +	/* Common code for wrapping up merges will be added here later */
> +}

I was about to comment that this complicates this here diff and that we
should do this when we need it, but I just peeked into the next patch,
and it uses it, so I leave this here paragraph only to show that I
actually reviewed this part, too.

And of course, if we have a `merge_finalize()`, then a `merge_start()`
does not sound all that bad, either.

The rest looks good to me.

Thanks,
Dscho

> +
> +int merge_trees(struct merge_options *opt,
> +		struct tree *head,
> +		struct tree *merge,
> +		struct tree *common,
> +		struct tree **result)
> +{
> +	int ret;
> +
> +	if (merge_start(opt, head))
> +		return -1;
> +	ret = merge_trees_internal(opt, head, merge, common, result);
> +	merge_finalize(opt);
> +
> +	return ret;
> +}
> +
> +int merge_recursive(struct merge_options *opt,
> +		    struct commit *h1,
> +		    struct commit *h2,
> +		    struct commit_list *ca,
> +		    struct commit **result)
> +{
> +	int ret;
> +
> +	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
> +		return -1;
> +	ret = merge_recursive_internal(opt, h1, h2, ca, result);
> +	merge_finalize(opt);
> +
> +	return ret;
> +}
> +
>  static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
>  			      const char *name)
>  {
> --
> 2.22.0.559.g28a8880890.dirty
>
>

Re: [PATCH 06/19] Change call signature of write_tree_from_memory()

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> diff --git a/merge-recursive.c b/merge-recursive.c
> index 7f56cb0ed1..1a3c6ab7f3 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> [...]
> @@ -434,11 +434,10 @@ struct tree *write_tree_from_memory(struct merge_options *opt)
>
>  	if (!cache_tree_fully_valid(istate->cache_tree) &&
>  	    cache_tree_update(istate, 0) < 0) {
> -		err(opt, _("error building trees"));
> -		return NULL;
> +		BUG("error building trees");

Hmm. Is it possible that something else than a bug in Git causes this to
fail?

I wonder, for example, whether a full disk can cause
`cache_tree_update()` to return a negative value.

Ciao,
Dscho

Re: [PATCH 03/19] Ensure index matches head before invoking merge machinery, round N

[Elijah Newren]

Hi Dscho,

On Thu, Jul 25, 2019 at 12:41 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Thu, 25 Jul 2019, Elijah Newren wrote:
>
<snip>
> > ...And it was fixed again in commit
> >   160252f81626 ("git-merge-ours: make sure our index matches HEAD", 2005-11-03)
> > ...and it was fixed again in commit
> >   3ec62ad9ffba ("merge-octopus: abort if index does not match HEAD", 2016-04-09)
> > ...and again in commit
> >   65170c07d466 ("merge-recursive: avoid incorporating uncommitted changes in a merge", 2017-12-21)
> > ...and again in commit
> >   eddd1a411d93 ("merge-recursive: enforce rule that index matches head before merging", 2018-06-30)
> >
> > ...with multiple testcases added to the testsuite that could be
> > enumerated in even more commits.
> >
> > Then, finally, in a patch in the same series as the last fix above, the
> > documentation about this requirement was fixed in commit 55f39cf7551b
> > ("merge: fix misleading pre-merge check documentation", 2018-06-30), and
> > we all lived happily ever after...
> >
> > </quick summary>
>
> Whoa. What a story.

I know, right?

> > diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
> > index 5b910e351e..a4bfd8fc51 100644
> > --- a/builtin/merge-recursive.c
> > +++ b/builtin/merge-recursive.c
> > @@ -1,3 +1,4 @@
> > +#include "cache.h"
> >  #include "builtin.h"
> >  #include "commit.h"
> >  #include "tag.h"
> > @@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
> >       if (argc - i != 3) /* "--" "<head>" "<remote>" */
> >               die(_("not handling anything other than two heads merge."));
> >
> > +     if (repo_read_index_unmerged(the_repository))
> > +             die_resolve_conflict("merge");
>
> For a moment I was unsure whether `_unmerged()` is the right thing to do
> here, as it specifically allows to read the index even when there are
> conflict stages. But I guess it does not matter too much here. I
> probably would have opted for `repo_read_index()` instead, though.

The names repo_read_index() and repo_read_index_unmerged() actually
seem slightly misleading to me; they seem to do the opposite of what
you'd think they do.

repo_read_index() reads in an index and allows unmerged entries and
returns istate->cache_nr.

repo_read_index_unmerged() calls repo_read_index(), then checks to see
if any of the entries are unmerged and returns whether or not any
unmerged entries were found.

So, the way to disallow conflict stages isn't to use
repo_read_index(), but to use repo_read_index_unmerged(), as I did.
Counter-intuitive, I know.

<snip>

> But of course, if there are uncommitted changes, this would write a tree
> different from HEAD, then reset the index to match HEAD, so indeed, this
> discard/read dance is necessary.
>
> So this hunk is good.
>
<snip>
>
> This is obviously a good change: it strengthens the test case by fixing
> a subtle bug.
>
> Thanks,
> Dscho


Thanks for taking a look!

Re: [PATCH 10/19] merge-recursive: rename 'mrtree' to 'result_tree', for clarity

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> It is not at all clear what 'mr' was supposed to stand for, at least not
> to me.  Pick a clearer name for this variable.

I got curious, and it looks as if I introduced this in 3af244caa82
(Cumulative update of merge-recursive in C, 2006-07-27), where I moved
the variable of type `struct tree` out of a `struct merge_tree_result`
(probably because I found the latter a bit pointless).

The best backsplanation I have for the "mr" is therefore "merge result".

I like `result_tree` better.

Thanks,
Dscho

>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 0a90546824..61faa26c4f 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3475,7 +3475,7 @@ static int merge_recursive_internal(struct merge_options *opt,
>  {
>  	struct commit_list *iter;
>  	struct commit *merged_merge_bases;
> -	struct tree *mrtree;
> +	struct tree *result_tree;
>  	int clean;
>
>  	if (show(opt, 4)) {
> @@ -3545,14 +3545,15 @@ static int merge_recursive_internal(struct merge_options *opt,
>  				     repo_get_commit_tree(opt->repo, h2),
>  				     repo_get_commit_tree(opt->repo,
>  							  merged_merge_bases),
> -				     &mrtree);
> +				     &result_tree);
>  	if (clean < 0) {
>  		flush_output(opt);
>  		return clean;
>  	}
>
>  	if (opt->call_depth) {
> -		*result = make_virtual_commit(opt->repo, mrtree, "merged tree");
> +		*result = make_virtual_commit(opt->repo, result_tree,
> +					      "merged tree");
>  		commit_list_insert(h1, &(*result)->parents);
>  		commit_list_insert(h2, &(*result)->parents->next);
>  	}
> --
> 2.22.0.559.g28a8880890.dirty
>
>

Re: [PATCH 04/19] merge-recursive: exit early if index != head

[Elijah Newren]

On Thu, Jul 25, 2019 at 12:51 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Thu, 25 Jul 2019, Elijah Newren wrote:
>
> > We had a rule to enforce that the index matches head, but it was found
> > at the beginning of merge_trees() and would only trigger when
> > opt->call_depth was 0.  Since merge_recursive() doesn't call
> > merge_trees() until after returning from recursing, this meant that the
> > check wasn't triggered by merge_recursive() until it had first finished
> > all the intermediate merges to create virtual merge bases.  That is a
> > potentially huge amount of computation (and writing of intermediate
> > merge results into the .git/objects directory) before it errors out and
> > says, in effect, "Sorry, I can't do any merging because you have some
> > local changes that would be overwritten."
> >
> > Further, not enforcing this requirement earlier allowed other bugs (such
> > as an unintentional unconditional dropping and reloading of the index in
> > merge_recursive() even when no recursion was necessary), to mask bugs in
> > other callers (which were fixed in the commit prior to this one).
> >
> > Make sure we do the index == head check at the beginning of the merge,
> > and error out immediately if it fails.
>
> Very clear commit message.
>
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index 37bb94fb4d..b762ecd7bd 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -3381,21 +3381,14 @@ static int process_entry(struct merge_options *opt,
> >       return clean_merge;
> >  }
> >
> > -int merge_trees(struct merge_options *opt,
> > -             struct tree *head,
> > -             struct tree *merge,
> > -             struct tree *common,
> > -             struct tree **result)
> > +static int merge_trees_internal(struct merge_options *opt,
>
> In other, similar cases, we seem to use the suffix `_1`. Not sure
> whether you want to change that here.

Hmm, we do seem to use `_1` about 2.5 times as frequently as
`_internal`, but both do seem to be in use to me (e.g.
init_tree_desc_internal, convert_to_working_tree_internal,
repo_parse_commit_internal).  `_1` does have the advantage of being
shorter, which makes lines fit in 80 columns better, but `_internal`
seems like a clearer description to me.

So...I'm not sure what's best here.  I don't have a strong opinion,
but I'm inclined towards laziness and leaving it alone unless someone
else has strong opinions.

> > +{
> > +     struct strbuf sb = STRBUF_INIT;
> > +
> > +     assert(opt->branch1 && opt->branch2);
> > +
> > +     if (repo_index_has_changes(opt->repo, head, &sb)) {
> > +             err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
> > +                 sb.buf);
>
> I know that you did not introduce this leak, but maybe we could slip an
> `strbuf_release(&sb);` in at this point?

Ooh, good point.  I'll fix that up for round 2.

> > +             return -1;
> > +     }
> > +
> > +     return 0;
> > +}
> > +
> > +static void merge_finalize(struct merge_options *opt)
> > +{
> > +     /* Common code for wrapping up merges will be added here later */
> > +}
>
> I was about to comment that this complicates this here diff and that we
> should do this when we need it, but I just peeked into the next patch,
> and it uses it, so I leave this here paragraph only to show that I
> actually reviewed this part, too.
>
> And of course, if we have a `merge_finalize()`, then a `merge_start()`
> does not sound all that bad, either.
>
> The rest looks good to me.

Thanks!

Re: [PATCH 15/19] merge-recursive: split internal fields into a separate struct

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> merge_options has several internal fields that should not be set or read
> by external callers.  This just complicates the API.  Move them into an
> opaque merge_options_internal struct that is defined only in
> merge-recursive.c and keep these out of merge-recursive.h.

This method requires an extra `malloc()`/`free()` pair, leaving room for
leaks.

I'd rather keep the definition of that struct in `merge-recursive.h`,
and make the `priv` field a full struct instead of a pointer. That would
save on that extra allocation.

Ciao,
Dscho

>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c | 185 +++++++++++++++++++++++++---------------------
>  merge-recursive.h |  17 ++---
>  2 files changed, 106 insertions(+), 96 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index a5049b06a3..c572d37b21 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -29,6 +29,15 @@
>  #include "revision.h"
>  #include "commit-reach.h"
>
> +struct merge_options_internal {
> +	int call_depth;
> +	int needed_rename_limit;
> +	struct hashmap current_file_dir_set;
> +	struct string_list df_conflict_file_set;
> +	struct unpack_trees_options unpack_opts;
> +	struct index_state orig_index;
> +};
> +
>  struct path_hashmap_entry {
>  	struct hashmap_entry e;
>  	char path[FLEX_ARRAY];
> @@ -309,7 +318,8 @@ static inline void setup_rename_conflict_info(enum rename_type rename_type,
>
>  static int show(struct merge_options *opt, int v)
>  {
> -	return (!opt->call_depth && opt->verbosity >= v) || opt->verbosity >= 5;
> +	return (!opt->priv->call_depth && opt->verbosity >= v) ||
> +		opt->verbosity >= 5;
>  }
>
>  __attribute__((format (printf, 3, 4)))
> @@ -320,7 +330,7 @@ static void output(struct merge_options *opt, int v, const char *fmt, ...)
>  	if (!show(opt, v))
>  		return;
>
> -	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
> +	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
>
>  	va_start(ap, fmt);
>  	strbuf_vaddf(&opt->obuf, fmt, ap);
> @@ -335,7 +345,7 @@ static void output_commit_title(struct merge_options *opt, struct commit *commit
>  {
>  	struct merge_remote_desc *desc;
>
> -	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
> +	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
>  	desc = merge_remote_util(commit);
>  	if (desc)
>  		strbuf_addf(&opt->obuf, "virtual %s\n", desc->name);
> @@ -403,43 +413,43 @@ static int unpack_trees_start(struct merge_options *opt,
>  	struct tree_desc t[3];
>  	struct index_state tmp_index = { NULL };
>
> -	memset(&opt->unpack_opts, 0, sizeof(opt->unpack_opts));
> -	if (opt->call_depth)
> -		opt->unpack_opts.index_only = 1;
> +	memset(&opt->priv->unpack_opts, 0, sizeof(opt->priv->unpack_opts));
> +	if (opt->priv->call_depth)
> +		opt->priv->unpack_opts.index_only = 1;
>  	else
> -		opt->unpack_opts.update = 1;
> -	opt->unpack_opts.merge = 1;
> -	opt->unpack_opts.head_idx = 2;
> -	opt->unpack_opts.fn = threeway_merge;
> -	opt->unpack_opts.src_index = opt->repo->index;
> -	opt->unpack_opts.dst_index = &tmp_index;
> -	opt->unpack_opts.aggressive = !merge_detect_rename(opt);
> -	setup_unpack_trees_porcelain(&opt->unpack_opts, "merge");
> +		opt->priv->unpack_opts.update = 1;
> +	opt->priv->unpack_opts.merge = 1;
> +	opt->priv->unpack_opts.head_idx = 2;
> +	opt->priv->unpack_opts.fn = threeway_merge;
> +	opt->priv->unpack_opts.src_index = opt->repo->index;
> +	opt->priv->unpack_opts.dst_index = &tmp_index;
> +	opt->priv->unpack_opts.aggressive = !merge_detect_rename(opt);
> +	setup_unpack_trees_porcelain(&opt->priv->unpack_opts, "merge");
>
>  	init_tree_desc_from_tree(t+0, common);
>  	init_tree_desc_from_tree(t+1, head);
>  	init_tree_desc_from_tree(t+2, merge);
>
> -	rc = unpack_trees(3, t, &opt->unpack_opts);
> +	rc = unpack_trees(3, t, &opt->priv->unpack_opts);
>  	cache_tree_free(&opt->repo->index->cache_tree);
>
>  	/*
> -	 * Update opt->repo->index to match the new results, AFTER saving a copy
> -	 * in opt->orig_index.  Update src_index to point to the saved copy.
> -	 * (verify_uptodate() checks src_index, and the original index is
> -	 * the one that had the necessary modification timestamps.)
> +	 * Update opt->repo->index to match the new results, AFTER saving a
> +	 * copy in opt->priv->orig_index.  Update src_index to point to the
> +	 * saved copy.  (verify_uptodate() checks src_index, and the original
> +	 * index is the one that had the necessary modification timestamps.)
>  	 */
> -	opt->orig_index = *opt->repo->index;
> +	opt->priv->orig_index = *opt->repo->index;
>  	*opt->repo->index = tmp_index;
> -	opt->unpack_opts.src_index = &opt->orig_index;
> +	opt->priv->unpack_opts.src_index = &opt->priv->orig_index;
>
>  	return rc;
>  }
>
>  static void unpack_trees_finish(struct merge_options *opt)
>  {
> -	discard_index(&opt->orig_index);
> -	clear_unpack_trees_porcelain(&opt->unpack_opts);
> +	discard_index(&opt->priv->orig_index);
> +	clear_unpack_trees_porcelain(&opt->priv->unpack_opts);
>  }
>
>  static int save_files_dirs(const struct object_id *oid,
> @@ -454,7 +464,7 @@ static int save_files_dirs(const struct object_id *oid,
>
>  	FLEX_ALLOC_MEM(entry, path, base->buf, base->len);
>  	hashmap_entry_init(entry, path_hash(entry->path));
> -	hashmap_add(&opt->current_file_dir_set, entry);
> +	hashmap_add(&opt->priv->current_file_dir_set, entry);
>
>  	strbuf_setlen(base, baselen);
>  	return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);
> @@ -585,7 +595,7 @@ static void record_df_conflict_files(struct merge_options *opt,
>  	 * If we're merging merge-bases, we don't want to bother with
>  	 * any working directory changes.
>  	 */
> -	if (opt->call_depth)
> +	if (opt->priv->call_depth)
>  		return;
>
>  	/* Ensure D/F conflicts are adjacent in the entries list. */
> @@ -597,7 +607,7 @@ static void record_df_conflict_files(struct merge_options *opt,
>  	df_sorted_entries.cmp = string_list_df_name_compare;
>  	string_list_sort(&df_sorted_entries);
>
> -	string_list_clear(&opt->df_conflict_file_set, 1);
> +	string_list_clear(&opt->priv->df_conflict_file_set, 1);
>  	for (i = 0; i < df_sorted_entries.nr; i++) {
>  		const char *path = df_sorted_entries.items[i].string;
>  		int len = strlen(path);
> @@ -613,7 +623,7 @@ static void record_df_conflict_files(struct merge_options *opt,
>  		    len > last_len &&
>  		    memcmp(path, last_file, last_len) == 0 &&
>  		    path[last_len] == '/') {
> -			string_list_insert(&opt->df_conflict_file_set, last_file);
> +			string_list_insert(&opt->priv->df_conflict_file_set, last_file);
>  		}
>
>  		/*
> @@ -680,8 +690,8 @@ static void update_entry(struct stage_data *entry,
>  static int remove_file(struct merge_options *opt, int clean,
>  		       const char *path, int no_wd)
>  {
> -	int update_cache = opt->call_depth || clean;
> -	int update_working_directory = !opt->call_depth && !no_wd;
> +	int update_cache = opt->priv->call_depth || clean;
> +	int update_working_directory = !opt->priv->call_depth && !no_wd;
>
>  	if (update_cache) {
>  		if (remove_file_from_index(opt->repo->index, path))
> @@ -724,16 +734,16 @@ static char *unique_path(struct merge_options *opt,
>  	add_flattened_path(&newpath, branch);
>
>  	base_len = newpath.len;
> -	while (hashmap_get_from_hash(&opt->current_file_dir_set,
> +	while (hashmap_get_from_hash(&opt->priv->current_file_dir_set,
>  				     path_hash(newpath.buf), newpath.buf) ||
> -	       (!opt->call_depth && file_exists(newpath.buf))) {
> +	       (!opt->priv->call_depth && file_exists(newpath.buf))) {
>  		strbuf_setlen(&newpath, base_len);
>  		strbuf_addf(&newpath, "_%d", suffix++);
>  	}
>
>  	FLEX_ALLOC_MEM(entry, path, newpath.buf, newpath.len);
>  	hashmap_entry_init(entry, path_hash(entry->path));
> -	hashmap_add(&opt->current_file_dir_set, entry);
> +	hashmap_add(&opt->priv->current_file_dir_set, entry);
>  	return strbuf_detach(&newpath, NULL);
>  }
>
> @@ -775,7 +785,7 @@ static int dir_in_way(struct index_state *istate, const char *path,
>  static int was_tracked_and_matches(struct merge_options *opt, const char *path,
>  				   const struct diff_filespec *blob)
>  {
> -	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
> +	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
>  	struct cache_entry *ce;
>
>  	if (0 > pos)
> @@ -783,7 +793,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
>  		return 0;
>
>  	/* See if the file we were tracking before matches */
> -	ce = opt->orig_index.cache[pos];
> +	ce = opt->priv->orig_index.cache[pos];
>  	return (oid_eq(&ce->oid, &blob->oid) && ce->ce_mode == blob->mode);
>  }
>
> @@ -792,7 +802,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
>   */
>  static int was_tracked(struct merge_options *opt, const char *path)
>  {
> -	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
> +	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
>
>  	if (0 <= pos)
>  		/* we were tracking this path before the merge */
> @@ -849,12 +859,12 @@ static int was_dirty(struct merge_options *opt, const char *path)
>  	struct cache_entry *ce;
>  	int dirty = 1;
>
> -	if (opt->call_depth || !was_tracked(opt, path))
> +	if (opt->priv->call_depth || !was_tracked(opt, path))
>  		return !dirty;
>
> -	ce = index_file_exists(opt->unpack_opts.src_index,
> +	ce = index_file_exists(opt->priv->unpack_opts.src_index,
>  			       path, strlen(path), ignore_case);
> -	dirty = verify_uptodate(ce, &opt->unpack_opts) != 0;
> +	dirty = verify_uptodate(ce, &opt->priv->unpack_opts) != 0;
>  	return dirty;
>  }
>
> @@ -864,8 +874,8 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
>  	const char *msg = _("failed to create path '%s'%s");
>
>  	/* Unlink any D/F conflict files that are in the way */
> -	for (i = 0; i < opt->df_conflict_file_set.nr; i++) {
> -		const char *df_path = opt->df_conflict_file_set.items[i].string;
> +	for (i = 0; i < opt->priv->df_conflict_file_set.nr; i++) {
> +		const char *df_path = opt->priv->df_conflict_file_set.items[i].string;
>  		size_t pathlen = strlen(path);
>  		size_t df_pathlen = strlen(df_path);
>  		if (df_pathlen < pathlen &&
> @@ -875,7 +885,7 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
>  			       _("Removing %s to make room for subdirectory\n"),
>  			       df_path);
>  			unlink(df_path);
> -			unsorted_string_list_delete_item(&opt->df_conflict_file_set,
> +			unsorted_string_list_delete_item(&opt->priv->df_conflict_file_set,
>  							 i, 0);
>  			break;
>  		}
> @@ -916,7 +926,7 @@ static int update_file_flags(struct merge_options *opt,
>  {
>  	int ret = 0;
>
> -	if (opt->call_depth)
> +	if (opt->priv->call_depth)
>  		update_wd = 0;
>
>  	if (update_wd) {
> @@ -1001,7 +1011,7 @@ static int update_file(struct merge_options *opt,
>  		       const char *path)
>  {
>  	return update_file_flags(opt, contents, path,
> -				 opt->call_depth || clean, !opt->call_depth);
> +				 opt->priv->call_depth || clean, !opt->priv->call_depth);
>  }
>
>  /* Low level file merging, update and removal */
> @@ -1030,7 +1040,7 @@ static int merge_3way(struct merge_options *opt,
>  	ll_opts.extra_marker_size = extra_marker_size;
>  	ll_opts.xdl_opts = opt->xdl_opts;
>
> -	if (opt->call_depth) {
> +	if (opt->priv->call_depth) {
>  		ll_opts.virtual_ancestor = 1;
>  		ll_opts.variant = 0;
>  	} else {
> @@ -1164,7 +1174,7 @@ static int merge_submodule(struct merge_options *opt,
>  	struct object_array merges;
>
>  	int i;
> -	int search = !opt->call_depth;
> +	int search = !opt->priv->call_depth;
>
>  	/* store a in result in case we fail */
>  	oidcpy(result, a);
> @@ -1385,7 +1395,7 @@ static int handle_rename_via_dir(struct merge_options *opt,
>  	int mark_conflicted = (opt->detect_directory_renames == 1);
>  	assert(ren->dir_rename_original_dest);
>
> -	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
> +	if (!opt->priv->call_depth && would_lose_untracked(opt, dest->path)) {
>  		mark_conflicted = 1;
>  		file_path = unique_path(opt, dest->path, ren->branch);
>  		output(opt, 1, _("Error: Refusing to lose untracked file at %s; "
> @@ -1428,12 +1438,12 @@ static int handle_change_delete(struct merge_options *opt,
>  	const char *update_path = path;
>  	int ret = 0;
>
> -	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0) ||
> -	    (!opt->call_depth && would_lose_untracked(opt, path))) {
> +	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0) ||
> +	    (!opt->priv->call_depth && would_lose_untracked(opt, path))) {
>  		update_path = alt_path = unique_path(opt, path, change_branch);
>  	}
>
> -	if (opt->call_depth) {
> +	if (opt->priv->call_depth) {
>  		/*
>  		 * We cannot arbitrarily accept either a_sha or b_sha as
>  		 * correct; since there is no true "middle point" between
> @@ -1508,14 +1518,14 @@ static int handle_rename_delete(struct merge_options *opt,
>  				     opt->branch2 : opt->branch1);
>
>  	if (handle_change_delete(opt,
> -				 opt->call_depth ? orig->path : dest->path,
> -				 opt->call_depth ? NULL : orig->path,
> +				 opt->priv->call_depth ? orig->path : dest->path,
> +				 opt->priv->call_depth ? NULL : orig->path,
>  				 orig, dest,
>  				 rename_branch, delete_branch,
>  				 _("rename"), _("renamed")))
>  		return -1;
>
> -	if (opt->call_depth)
> +	if (opt->priv->call_depth)
>  		return remove_file_from_index(opt->repo->index, dest->path);
>  	else
>  		return update_stages(opt, dest->path, NULL,
> @@ -1552,7 +1562,7 @@ static int handle_file_collision(struct merge_options *opt,
>  	/*
>  	 * In the recursive case, we just opt to undo renames
>  	 */
> -	if (opt->call_depth && (prev_path1 || prev_path2)) {
> +	if (opt->priv->call_depth && (prev_path1 || prev_path2)) {
>  		/* Put first file (a->oid, a->mode) in its original spot */
>  		if (prev_path1) {
>  			if (update_file(opt, 1, a, prev_path1))
> @@ -1581,10 +1591,10 @@ static int handle_file_collision(struct merge_options *opt,
>  	/* Remove rename sources if rename/add or rename/rename(2to1) */
>  	if (prev_path1)
>  		remove_file(opt, 1, prev_path1,
> -			    opt->call_depth || would_lose_untracked(opt, prev_path1));
> +			    opt->priv->call_depth || would_lose_untracked(opt, prev_path1));
>  	if (prev_path2)
>  		remove_file(opt, 1, prev_path2,
> -			    opt->call_depth || would_lose_untracked(opt, prev_path2));
> +			    opt->priv->call_depth || would_lose_untracked(opt, prev_path2));
>
>  	/*
>  	 * Remove the collision path, if it wouldn't cause dirty contents
> @@ -1626,12 +1636,12 @@ static int handle_file_collision(struct merge_options *opt,
>  	null.mode = 0;
>
>  	if (merge_mode_and_contents(opt, &null, a, b, collide_path,
> -				    branch1, branch2, opt->call_depth * 2, &mfi))
> +				    branch1, branch2, opt->priv->call_depth * 2, &mfi))
>  		return -1;
>  	mfi.clean &= !alt_path;
>  	if (update_file(opt, mfi.clean, &mfi.blob, update_path))
>  		return -1;
> -	if (!mfi.clean && !opt->call_depth &&
> +	if (!mfi.clean && !opt->priv->call_depth &&
>  	    update_stages(opt, collide_path, NULL, a, b))
>  		return -1;
>  	free(alt_path);
> @@ -1671,7 +1681,7 @@ static int handle_rename_add(struct merge_options *opt,
>  				    &ci->ren1->src_entry->stages[other_stage],
>  				    prev_path_desc,
>  				    opt->branch1, opt->branch2,
> -				    1 + opt->call_depth * 2, &mfi))
> +				    1 + opt->priv->call_depth * 2, &mfi))
>  		return -1;
>  	free(prev_path_desc);
>
> @@ -1689,7 +1699,7 @@ static char *find_path_for_conflict(struct merge_options *opt,
>  				    const char *branch2)
>  {
>  	char *new_path = NULL;
> -	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0)) {
> +	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0)) {
>  		new_path = unique_path(opt, path, branch1);
>  		output(opt, 1, _("%s is a directory in %s adding "
>  			       "as %s instead"),
> @@ -1720,17 +1730,17 @@ static int handle_rename_rename_1to2(struct merge_options *opt,
>  	       "rename \"%s\"->\"%s\" in \"%s\"%s"),
>  	       o->path, a->path, ci->ren1->branch,
>  	       o->path, b->path, ci->ren2->branch,
> -	       opt->call_depth ? _(" (left unresolved)") : "");
> +	       opt->priv->call_depth ? _(" (left unresolved)") : "");
>
>  	path_desc = xstrfmt("%s and %s, both renamed from %s",
>  			    a->path, b->path, o->path);
>  	if (merge_mode_and_contents(opt, o, a, b, path_desc,
>  				    ci->ren1->branch, ci->ren2->branch,
> -				    opt->call_depth * 2, &mfi))
> +				    opt->priv->call_depth * 2, &mfi))
>  		return -1;
>  	free(path_desc);
>
> -	if (opt->call_depth) {
> +	if (opt->priv->call_depth) {
>  		/*
>  		 * FIXME: For rename/add-source conflicts (if we could detect
>  		 * such), this is wrong.  We should instead find a unique
> @@ -1845,12 +1855,12 @@ static int handle_rename_rename_2to1(struct merge_options *opt,
>  				    &ci->ren1->src_entry->stages[ostage1],
>  				    path_side_1_desc,
>  				    opt->branch1, opt->branch2,
> -				    1 + opt->call_depth * 2, &mfi_c1) ||
> +				    1 + opt->priv->call_depth * 2, &mfi_c1) ||
>  	    merge_mode_and_contents(opt, b,
>  				    &ci->ren2->src_entry->stages[ostage2],
>  				    c2, path_side_2_desc,
>  				    opt->branch1, opt->branch2,
> -				    1 + opt->call_depth * 2, &mfi_c2))
> +				    1 + opt->priv->call_depth * 2, &mfi_c2))
>  		return -1;
>  	free(path_side_1_desc);
>  	free(path_side_2_desc);
> @@ -1891,8 +1901,8 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
>  	diff_setup_done(&opts);
>  	diff_tree_oid(&o_tree->object.oid, &tree->object.oid, "", &opts);
>  	diffcore_std(&opts);
> -	if (opts.needed_rename_limit > opt->needed_rename_limit)
> -		opt->needed_rename_limit = opts.needed_rename_limit;
> +	if (opts.needed_rename_limit > opt->priv->needed_rename_limit)
> +		opt->priv->needed_rename_limit = opts.needed_rename_limit;
>
>  	ret = xmalloc(sizeof(*ret));
>  	*ret = diff_queued_diff;
> @@ -3022,13 +3032,13 @@ static int handle_content_merge(struct merge_file_info *mfi,
>  		reason = _("add/add");
>
>  	assert(o->path && a->path && b->path);
> -	if (ci && dir_in_way(opt->repo->index, path, !opt->call_depth,
> +	if (ci && dir_in_way(opt->repo->index, path, !opt->priv->call_depth,
>  			     S_ISGITLINK(ci->ren1->pair->two->mode)))
>  		df_conflict_remains = 1;
>
>  	if (merge_mode_and_contents(opt, o, a, b, path,
>  				    opt->branch1, opt->branch2,
> -				    opt->call_depth * 2, mfi))
> +				    opt->priv->call_depth * 2, mfi))
>  		return -1;
>
>  	/*
> @@ -3044,7 +3054,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
>
>  		output(opt, 3, _("Skipped %s (merged same as existing)"), path);
>  		if (add_cacheinfo(opt, &mfi->blob, path,
> -				  0, (!opt->call_depth && !is_dirty), 0))
> +				  0, (!opt->priv->call_depth && !is_dirty), 0))
>  			return -1;
>  		/*
>  		 * However, add_cacheinfo() will delete the old cache entry
> @@ -3052,8 +3062,8 @@ static int handle_content_merge(struct merge_file_info *mfi,
>  		 * flag to avoid making the file appear as if it were
>  		 * deleted by the user.
>  		 */
> -		pos = index_name_pos(&opt->orig_index, path, strlen(path));
> -		ce = opt->orig_index.cache[pos];
> +		pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
> +		ce = opt->priv->orig_index.cache[pos];
>  		if (ce_skip_worktree(ce)) {
>  			pos = index_name_pos(opt->repo->index, path, strlen(path));
>  			ce = opt->repo->index->cache[pos];
> @@ -3074,7 +3084,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
>
>  	if (df_conflict_remains || is_dirty) {
>  		char *new_path;
> -		if (opt->call_depth) {
> +		if (opt->priv->call_depth) {
>  			remove_file_from_index(opt->repo->index, path);
>  		} else {
>  			if (!mfi->clean) {
> @@ -3332,7 +3342,7 @@ static int process_entry(struct merge_options *opt,
>  			conf = _("directory/file");
>  		}
>  		if (dir_in_way(opt->repo->index, path,
> -			       !opt->call_depth && !S_ISGITLINK(a->mode),
> +			       !opt->priv->call_depth && !S_ISGITLINK(a->mode),
>  			       0)) {
>  			char *new_path = unique_path(opt, path, add_branch);
>  			clean_merge = 0;
> @@ -3341,7 +3351,7 @@ static int process_entry(struct merge_options *opt,
>  			       conf, path, other_branch, path, new_path);
>  			if (update_file(opt, 0, contents, new_path))
>  				clean_merge = -1;
> -			else if (opt->call_depth)
> +			else if (opt->priv->call_depth)
>  				remove_file_from_index(opt->repo->index, path);
>  			free(new_path);
>  		} else {
> @@ -3406,7 +3416,7 @@ static int merge_trees_internal(struct merge_options *opt,
>  	code = unpack_trees_start(opt, merge_base, head, merge);
>
>  	if (code != 0) {
> -		if (show(opt, 4) || opt->call_depth)
> +		if (show(opt, 4) || opt->priv->call_depth)
>  			err(opt, _("merging of trees %s and %s failed"),
>  			    oid_to_hex(&head->object.oid),
>  			    oid_to_hex(&merge->object.oid));
> @@ -3425,7 +3435,7 @@ static int merge_trees_internal(struct merge_options *opt,
>  		 * opposed to decaring a local hashmap is for convenience
>  		 * so that we don't have to pass it to around.
>  		 */
> -		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
> +		hashmap_init(&opt->priv->current_file_dir_set, path_hashmap_cmp,
>  			     NULL, 512);
>  		get_files_dirs(opt, head);
>  		get_files_dirs(opt, merge);
> @@ -3462,7 +3472,7 @@ static int merge_trees_internal(struct merge_options *opt,
>  		string_list_clear(entries, 1);
>  		free(entries);
>
> -		hashmap_free(&opt->current_file_dir_set, 1);
> +		hashmap_free(&opt->priv->current_file_dir_set, 1);
>
>  		if (clean < 0) {
>  			unpack_trees_finish(opt);
> @@ -3474,7 +3484,8 @@ static int merge_trees_internal(struct merge_options *opt,
>
>  	unpack_trees_finish(opt);
>
> -	if (opt->call_depth && !(*result = write_tree_from_memory(opt->repo)))
> +	if (opt->priv->call_depth &&
> +	    !(*result = write_tree_from_memory(opt->repo)))
>  		return -1;
>
>  	return clean;
> @@ -3538,7 +3549,7 @@ static int merge_recursive_internal(struct merge_options *opt,
>
>  	for (iter = merge_bases; iter; iter = iter->next) {
>  		const char *saved_b1, *saved_b2;
> -		opt->call_depth++;
> +		opt->priv->call_depth++;
>  		/*
>  		 * When the merge fails, the result contains files
>  		 * with conflict markers. The cleanness flag is
> @@ -3557,14 +3568,14 @@ static int merge_recursive_internal(struct merge_options *opt,
>  			return -1;
>  		opt->branch1 = saved_b1;
>  		opt->branch2 = saved_b2;
> -		opt->call_depth--;
> +		opt->priv->call_depth--;
>
>  		if (!merged_merge_bases)
>  			return err(opt, _("merge returned no commit"));
>  	}
>
>  	discard_index(opt->repo->index);
> -	if (!opt->call_depth)
> +	if (!opt->priv->call_depth)
>  		repo_read_index(opt->repo);
>
>  	opt->ancestor = "merged common ancestors";
> @@ -3579,14 +3590,14 @@ static int merge_recursive_internal(struct merge_options *opt,
>  		return clean;
>  	}
>
> -	if (opt->call_depth) {
> +	if (opt->priv->call_depth) {
>  		*result = make_virtual_commit(opt->repo, result_tree,
>  					      "merged tree");
>  		commit_list_insert(h1, &(*result)->parents);
>  		commit_list_insert(h2, &(*result)->parents->next);
>  	}
>  	flush_output(opt);
> -	if (!opt->call_depth && opt->buffer_output < 2)
> +	if (!opt->priv->call_depth && opt->buffer_output < 2)
>  		strbuf_release(&opt->obuf);
>  	return clean;
>  }
> @@ -3603,6 +3614,8 @@ static int merge_start(struct merge_options *opt, struct tree *head)
>  		return -1;
>  	}
>
> +	opt->priv = xcalloc(1, sizeof(*opt->priv));
> +	string_list_init(&opt->priv->df_conflict_file_set, 1);
>  	return 0;
>  }
>
> @@ -3610,7 +3623,9 @@ static void merge_finalize(struct merge_options *opt)
>  {
>  	if (show(opt, 2))
>  		diff_warn_rename_limit("merge.renamelimit",
> -				       opt->needed_rename_limit, 0);
> +				       opt->priv->needed_rename_limit, 0);
> +	free(opt->priv);
> +	opt->priv = NULL;
>  }
>
>  int merge_trees(struct merge_options *opt,
> @@ -3748,8 +3763,6 @@ void init_merge_options(struct merge_options *opt,
>
>  	opt->renormalize = 0;
>
> -	string_list_init(&opt->df_conflict_file_set, 1);
> -
>  	merge_recursive_config(opt);
>  	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>  	if (merge_verbosity)
> diff --git a/merge-recursive.h b/merge-recursive.h
> index d57fce0daa..a1bb29dc33 100644
> --- a/merge-recursive.h
> +++ b/merge-recursive.h
> @@ -1,13 +1,15 @@
>  #ifndef MERGE_RECURSIVE_H
>  #define MERGE_RECURSIVE_H
>
> -#include "string-list.h"
> -#include "unpack-trees.h"
> +#include "strbuf.h"
>
>  struct commit;
> -
> +struct commit_list;
> +struct object_id;
>  struct repository;
> +struct tree;
>
> +struct merge_options_internal;
>  struct merge_options {
>  	struct repository *repo;
>
> @@ -40,13 +42,8 @@ struct merge_options {
>  	const char *subtree_shift;
>  	unsigned renormalize : 1;
>
> -	/* internal fields used by the implementation (do NOT set these) */
> -	int call_depth;
> -	int needed_rename_limit;
> -	struct hashmap current_file_dir_set;
> -	struct string_list df_conflict_file_set;
> -	struct unpack_trees_options unpack_opts;
> -	struct index_state orig_index;
> +	/* internal fields used by the implementation */
> +	struct merge_options_internal *priv;
>  };
>
>  void init_merge_options(struct merge_options *opt, struct repository *repo);
> --
> 2.22.0.559.g28a8880890.dirty
>
>

Re: [PATCH 15/19] merge-recursive: split internal fields into a separate struct

[Elijah Newren]

On Thu, Jul 25, 2019 at 1:12 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Thu, 25 Jul 2019, Elijah Newren wrote:
>
> > merge_options has several internal fields that should not be set or read
> > by external callers.  This just complicates the API.  Move them into an
> > opaque merge_options_internal struct that is defined only in
> > merge-recursive.c and keep these out of merge-recursive.h.
>
> This method requires an extra `malloc()`/`free()` pair, leaving room for
> leaks.
>
> I'd rather keep the definition of that struct in `merge-recursive.h`,
> and make the `priv` field a full struct instead of a pointer. That would
> save on that extra allocation.

Yes, it has one extra malloc()/free() pair for the entire merge operation.  But:
  * That's a drop in the bucket compared to the allocations already
performed during a merge
  * The allocation and free happen in merge_start() and
merge_finalize().  Not much to keep track of, so not much room for
leaks.
  * I want what's currently in merge-recursive.h to be as simple as
possible; I implemented your suggestion first, but it doesn't simplify
as much as possible.

But, more importantly:
  * I want to write an alternative merge strategy providing drop-in
replacement functions for merge_trees(), merge_recursive(), and
merge_recursive_generic(). Defining merge_options_internal inside
merge-recursive.h would mean that I have to have _exactly_ the same
internal options in my implementation of merge-ort.c.  That doesn't
make sense.

Elijah

Re: [PATCH 19/19] merge-recursive: provide a better label for diff3 common ancestor

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> In commit 7ca56aa07619 ("merge-recursive: add a label for ancestor",
> 2010-03-20), a label was added for the '||||||' line to make it have
> the more informative heading '|||||| merged common ancestors', with
> the statement:
>
>     It would be nicer to use a more informative label.  Perhaps someone
>     will provide one some day.
>
> This chosen label was perfectly reasonable when recursiveness kicks in,
> i.e. when there are multiple merge bases.  (I can't think of a better
> label in such cases.)  But it is actually somewhat misleading when there
> is a unique merge base or no merge base.  Change this based on the
> number of merge bases:
>     >=2: "merged common ancestors"
>     1:   <abbreviated commit hash>
>     0:   "<empty tree>"

Nice!

> Also, the code in merge_3way making use of opt->ancestor was overly
> complex because it tried to handle the case of opt->ancestor being NULL.
> We always set it first, though, so just add an assert that opt->ancestor
> is not NULL and simplify the surrounding code.
>
> Tests have also been added to check that we get the right ancestor name
> for each of the three cases.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c                 |  35 ++++--
>  t/t6036-recursive-corner-cases.sh |   8 +-
>  t/t6047-diff3-conflict-markers.sh | 191 ++++++++++++++++++++++++++++++
>  3 files changed, 220 insertions(+), 14 deletions(-)
>  create mode 100755 t/t6047-diff3-conflict-markers.sh
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 4cd6599296..8ac53cacdf 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -1034,7 +1034,7 @@ static int merge_3way(struct merge_options *opt,
>  {
>  	mmfile_t orig, src1, src2;
>  	struct ll_merge_options ll_opts = {0};
> -	char *base_name, *name1, *name2;
> +	char *base, *name1, *name2;

Renaming this variable at the same time as doing other, more involved
changes, made this patch a bit harder to review for me, I must admit.

>  	int merge_status;
>
>  	ll_opts.renormalize = opt->renormalize;
> @@ -1058,16 +1058,13 @@ static int merge_3way(struct merge_options *opt,
>  		}
>  	}
>
> -	assert(a->path && b->path && o->path);
> -	if (strcmp(a->path, b->path) ||
> -	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
> -		base_name = opt->ancestor == NULL ? NULL :
> -			mkpathdup("%s:%s", opt->ancestor, o->path);
> +	assert(a->path && b->path && o->path && opt->ancestor);
> +	if (strcmp(a->path, b->path) || strcmp(a->path, o->path) != 0) {
> +		base  = mkpathdup("%s:%s", opt->ancestor, o->path);
>  		name1 = mkpathdup("%s:%s", branch1, a->path);
>  		name2 = mkpathdup("%s:%s", branch2, b->path);
>  	} else {
> -		base_name = opt->ancestor == NULL ? NULL :
> -			mkpathdup("%s", opt->ancestor);
> +		base  = mkpathdup("%s", opt->ancestor);
>  		name1 = mkpathdup("%s", branch1);
>  		name2 = mkpathdup("%s", branch2);
>  	}
> @@ -1076,11 +1073,11 @@ static int merge_3way(struct merge_options *opt,
>  	read_mmblob(&src1, &a->oid);
>  	read_mmblob(&src2, &b->oid);
>
> -	merge_status = ll_merge(result_buf, a->path, &orig, base_name,
> +	merge_status = ll_merge(result_buf, a->path, &orig, base,
>  				&src1, name1, &src2, name2,
>  				opt->repo->index, &ll_opts);
>
> -	free(base_name);
> +	free(base);
>  	free(name1);
>  	free(name2);
>  	free(orig.ptr);
> @@ -3517,6 +3514,8 @@ static int merge_recursive_internal(struct merge_options *opt,
>  	struct commit *merged_merge_bases;
>  	struct tree *result_tree;
>  	int clean;
> +	int num_merge_bases;
> +	struct strbuf commit_name = STRBUF_INIT;
>
>  	if (show(opt, 4)) {
>  		output(opt, 4, _("Merging:"));
> @@ -3538,6 +3537,7 @@ static int merge_recursive_internal(struct merge_options *opt,
>  			output_commit_title(opt, iter->item);
>  	}
>
> +	num_merge_bases = commit_list_count(merge_bases);

At first, I thought that this does not require a separate variable
because it is used exactly once.

But then I realized that the next line changes the number of commits in
`merge_bases`, so it actually _is_ required.

>  	merged_merge_bases = pop_commit(&merge_bases);
>  	if (merged_merge_bases == NULL) {
>  		/* if there is no common ancestor, use an empty tree */
> @@ -3579,13 +3579,26 @@ static int merge_recursive_internal(struct merge_options *opt,
>  	if (!opt->priv->call_depth)
>  		repo_read_index(opt->repo);
>
> -	opt->ancestor = "merged common ancestors";
> +	switch (num_merge_bases) {
> +	case 0:
> +		opt->ancestor = "<empty tree>";
> +		break;
> +	case 1:
> +		strbuf_add_unique_abbrev(&commit_name,

The name `commit_name` would suggest a different usage to me. How about
`pretty_merge_base`?

> +					 &merged_merge_bases->object.oid,
> +					 DEFAULT_ABBREV);
> +		opt->ancestor = commit_name.buf;
> +		break;
> +	default:
> +		opt->ancestor = "merged common ancestors";
> +	}
>  	clean = merge_trees_internal(opt,
>  				     repo_get_commit_tree(opt->repo, h1),
>  				     repo_get_commit_tree(opt->repo, h2),
>  				     repo_get_commit_tree(opt->repo,
>  							  merged_merge_bases),
>  				     &result_tree);
> +	strbuf_release(&commit_name);
>  	if (clean < 0) {
>  		flush_output(opt);
>  		return clean;

I was a bit too tired to look more closely at the test cases, in
particular after seeing the enormous complexity of the added test
script. I wonder whether it really has to be all that complex (e.g. why
use a complicated `test_seq` when a `test_commit A A.t
"1${LF}2${LF}3${LF}A${LF}` would suffice? Why start by `git checkout
--orphan` on a just-created repository?)

But otherwise, the patch series looks pretty good to me.

Thanks,
Dscho

Re: [PATCH 00/19] Cleanup merge API

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> On Thu, Jul 25, 2019 at 11:12 AM Junio C Hamano <gitster@pobox.com> wrote:
>
>> > Stuff I'd most welcome review on:
>> >   * Is cache-tree.c the right place for write_tree_from_memory()?
>> >     [see patch 7]  Should there be docs on how it differs from
>> >     write_index_as_tree(), already found in cache-tree?  What does
>> >     the latter even do?
>>
>> write_index_as_tree() is supposed to write the contents of an index
>> state as a tree object, and return the object ID for the resulting
>> tree.  It is the primary interface designed to be used by
>> write-tree.
>
> Other than the last sentence, that also sounds like the description of
> write_index_as_tree() -- at least as best I understood it.

Yes, I didn't even know merge-recursive had its own variant.  I
suspect that back when "merge-recursive in C" was being developed,
it first used "git write-tree" via run_command(), and then just
copied and pasted what was done in the write-tree implementation
without bothering to refactor it into write_index_as_tree() and its
own bits about the unmerged index.

Re: [PATCH 06/19] Change call signature of write_tree_from_memory()

[Elijah Newren]

Hi Dscho,

On Thu, Jul 25, 2019 at 12:55 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Thu, 25 Jul 2019, Elijah Newren wrote:
>
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index 7f56cb0ed1..1a3c6ab7f3 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > [...]
> > @@ -434,11 +434,10 @@ struct tree *write_tree_from_memory(struct merge_options *opt)
> >
> >       if (!cache_tree_fully_valid(istate->cache_tree) &&
> >           cache_tree_update(istate, 0) < 0) {
> > -             err(opt, _("error building trees"));
> > -             return NULL;
> > +             BUG("error building trees");
>
> Hmm. Is it possible that something else than a bug in Git causes this to
> fail?
>
> I wonder, for example, whether a full disk can cause
> `cache_tree_update()` to return a negative value.

Yeah, you're right.  Based on the conversation with Junio, I think I
can modify the write_index_as_tree() function slightly to do what we
need, then call it instead of write_tree_from_memory().  Since
write_index_as_tree() doesn't try to output any error messages but
just returns various error codes, I can then have the caller handle
the output.  So I'll do that instead of this patch and the previous
one.

Re: [PATCH 03/19] Ensure index matches head before invoking merge machinery, round N

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> On Thu, Jul 25, 2019 at 12:41 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Thu, 25 Jul 2019, Elijah Newren wrote:
> >
> > > diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
> > > index 5b910e351e..a4bfd8fc51 100644
> > > --- a/builtin/merge-recursive.c
> > > +++ b/builtin/merge-recursive.c
> > > @@ -1,3 +1,4 @@
> > > +#include "cache.h"
> > >  #include "builtin.h"
> > >  #include "commit.h"
> > >  #include "tag.h"
> > > @@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
> > >       if (argc - i != 3) /* "--" "<head>" "<remote>" */
> > >               die(_("not handling anything other than two heads merge."));
> > >
> > > +     if (repo_read_index_unmerged(the_repository))
> > > +             die_resolve_conflict("merge");
> >
> > For a moment I was unsure whether `_unmerged()` is the right thing to do
> > here, as it specifically allows to read the index even when there are
> > conflict stages. But I guess it does not matter too much here. I
> > probably would have opted for `repo_read_index()` instead, though.
>
> The names repo_read_index() and repo_read_index_unmerged() actually
> seem slightly misleading to me; they seem to do the opposite of what
> you'd think they do.
>
> repo_read_index() reads in an index and allows unmerged entries and
> returns istate->cache_nr.
>
> repo_read_index_unmerged() calls repo_read_index(), then checks to see
> if any of the entries are unmerged and returns whether or not any
> unmerged entries were found.
>
> So, the way to disallow conflict stages isn't to use
> repo_read_index(), but to use repo_read_index_unmerged(), as I did.
> Counter-intuitive, I know.

Whoa. No wonder I get it wrong all the time.

Thanks for educating me,
Dscho

Re: [PATCH 15/19] merge-recursive: split internal fields into a separate struct

[Johannes Schindelin]

Hi Elijah,

On Thu, 25 Jul 2019, Elijah Newren wrote:

> On Thu, Jul 25, 2019 at 1:12 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Hi Elijah,
> >
> > On Thu, 25 Jul 2019, Elijah Newren wrote:
> >
> > > merge_options has several internal fields that should not be set or read
> > > by external callers.  This just complicates the API.  Move them into an
> > > opaque merge_options_internal struct that is defined only in
> > > merge-recursive.c and keep these out of merge-recursive.h.
> >
> > This method requires an extra `malloc()`/`free()` pair, leaving room for
> > leaks.
> >
> > I'd rather keep the definition of that struct in `merge-recursive.h`,
> > and make the `priv` field a full struct instead of a pointer. That would
> > save on that extra allocation.
>
> Yes, it has one extra malloc()/free() pair for the entire merge operation.  But:
>   * That's a drop in the bucket compared to the allocations already
> performed during a merge
>   * The allocation and free happen in merge_start() and
> merge_finalize().  Not much to keep track of, so not much room for
> leaks.
>   * I want what's currently in merge-recursive.h to be as simple as
> possible; I implemented your suggestion first, but it doesn't simplify
> as much as possible.
>
> But, more importantly:
>   * I want to write an alternative merge strategy providing drop-in
> replacement functions for merge_trees(), merge_recursive(), and
> merge_recursive_generic(). Defining merge_options_internal inside
> merge-recursive.h would mean that I have to have _exactly_ the same
> internal options in my implementation of merge-ort.c.  That doesn't
> make sense.

Fair enough.

I'm curious: what merge strategy are you planning on implementing?

Ciao,
Dscho

Re: [PATCH 00/19] Cleanup merge API

[Johannes Schindelin]

Hi Junio,

On Thu, 25 Jul 2019, Junio C Hamano wrote:

> Elijah Newren <newren@gmail.com> writes:
>
> > On Thu, Jul 25, 2019 at 11:12 AM Junio C Hamano <gitster@pobox.com> wrote:
> >
> >> > Stuff I'd most welcome review on:
> >> >   * Is cache-tree.c the right place for write_tree_from_memory()?
> >> >     [see patch 7]  Should there be docs on how it differs from
> >> >     write_index_as_tree(), already found in cache-tree?  What does
> >> >     the latter even do?
> >>
> >> write_index_as_tree() is supposed to write the contents of an index
> >> state as a tree object, and return the object ID for the resulting
> >> tree.  It is the primary interface designed to be used by
> >> write-tree.
> >
> > Other than the last sentence, that also sounds like the description of
> > write_index_as_tree() -- at least as best I understood it.
>
> Yes, I didn't even know merge-recursive had its own variant.  I
> suspect that back when "merge-recursive in C" was being developed,
> it first used "git write-tree" via run_command(), and then just
> copied and pasted what was done in the write-tree implementation
> without bothering to refactor it into write_index_as_tree() and its
> own bits about the unmerged index.

FWIW that matches my understanding (i.e. one of those authors' of
"merge-recursive in C").

Thanks,
Dscho

Re: [PATCH 00/19] Cleanup merge API

[Elijah Newren]

Hi Dscho,

On Thu, Jul 25, 2019 at 12:15 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Thu, 25 Jul 2019, Elijah Newren wrote:
>
> >   * All current callers (3 of them?) of merge_recursive() always pass
> >     it a specially created reversed-list for the merge_bases.  Some
> >     history spelunking provides no details on any of these about why;
> >     it appears that the 2nd and 3rd callers reversed the list because
> >     the first did, and I'm guessing the first did in an attempt to
> >     exactly match the git-merge-recursive.py scripts' behavior.
>
> That is part of the reason.
>
> After I worked on converting that Python script with Alex Riesen, I
> tested it on all merge commits I could find at that point in linux.git,
> to compare the outcomes between scripted and non-scripted versions. IIRC
> I even reported those findings back to the mailing list.
>
> *clickety-click*
>
> Ah, I did the extensive test on git.git first [*1*]:
> https://public-inbox.org/git/Pine.LNX.4.63.0608092232000.13885@wbgn013.biozentrum.uni-wuerzburg.de/
>
> As you can see, it seemed that the reverse order of merge bases (trying
> to merge the oldest two merge bases first, then merging with the
> 3rd-oldest, etc) avoided more merge conflicts than the original order.
>
> >     But if the API needs them in a reverse order from what people
> >     would normally expect to pass them in, shouldn't it reverse them
> >     itself instead of making all callers do it?
>
> I would not, as the order the merge bases are passed in defines the
> order in which they are handled.
>
> >     Also, the order shouldn't matter when there are no conflicts, and
> >     when there are conflicts it'd only change which side of the
> >     conflict markers the made-up virtual merge base would list things
> >     in.
>
> That's what I thought, right up until I re-created those merge commits.
>
> I really forgot most of the details, but I seem to remember that there
> was a puzzling one where the reverse order caused no merge conflicts,
> and the original order caused a double merge conflict.

Thanks for providing this history; very helpful.  In v2 (which I will
be posting soon), I included a comment in merge-recursive.h just above
the merge_recursive() function with a summary of these details:

NOTE: empirically, about a decade ago it was determined that with more
      than two merge bases, optimal behavior was found when the
      merge_bases were passed in the order of oldest commit to newest
      commit.  Also, merge_bases will be consumed (emptied) so make a
      copy if you need it.

Re: [PATCH 15/19] merge-recursive: split internal fields into a separate struct

[Elijah Newren]

Hi Dscho,

On Fri, Jul 26, 2019 at 4:25 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> On Thu, 25 Jul 2019, Elijah Newren wrote:
>
> > On Thu, Jul 25, 2019 at 1:12 PM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:

> > But, more importantly:
> >   * I want to write an alternative merge strategy providing drop-in
> > replacement functions for merge_trees(), merge_recursive(), and
> > merge_recursive_generic(). Defining merge_options_internal inside
> > merge-recursive.h would mean that I have to have _exactly_ the same
> > internal options in my implementation of merge-ort.c.  That doesn't
> > make sense.
>
> Fair enough.
>
> I'm curious: what merge strategy are you planning on implementing?

recursive, done right[1].  For now, I'm calling it "Ostensibly
Recursive's Twin", or "ort" for short.  At first, people shouldn't be
able to notice any difference between it and the current recursive
strategy, other than the fact that I think I can make it a bit faster
(especially for big repos).  But it should allow me to fix some
(admittedly corner case) bugs that are harder to handle in the current
design, and I think that a merge that doesn't touch $GIT_WORK_TREE or
$GIT_INDEX_FILE will allow for some fun new features.  That's the hope
anyway.

[1] https://public-inbox.org/git/xmqqd147kpdm.fsf@gitster.mtv.corp.google.com/

[PATCH v2 00/20] Cleanup merge API

[Elijah Newren]

Before writing a replacement merge strategy for recursive, I decided
to first cleanup the merge API -- streamlining merge-recursive.h and
making it more readable.  It includes some fixes I noticed along the
way, and two forgotten patches of mine from months ago that I rebased
and included at the end.  As with v1:

    While there are minor textual and semantic dependencies between
    these patches (preventing me from splitting up this series), they
    are logically separate and can be reviewed independently.

Notable highlights (full range-diff below):
  * Patches 8, 11, and 16 show up in the range-diff but have no
    substantive changes (these diffs merely represents the minimal
    changes needed to adjust due to the patches before them in the
    series).  A few other patches (e.g. 9, 12) had some real changes
    but most of the range-diff for those was similar noise.
  * [Patch 4] Fixed a (pre-existing) memory leak pointed out by Dscho
    and renamed a couple new variables to have a clearer meaning.
  * [Patch 5, new] Added a new commit that simplified and clarified the
    merge_trees() API by removing a parameter that would never be
    written anyway.  I also added a comment to the header summarizing
    some of Dscho's remarks about how best to pass the merge_bases to
    the merge_recursive() function.
  * [Patch 7] Made a small extension to write_index_as_tree(), deleted
    write_tree_from_memory(), and converted callers of the latter into
    callers of the former.  This patch replaces patches 6 & 7 from v1.
  * [Patch 9] The only substantive change was to update the new comment
    from patch 5 to also be affected by the parameter renaming that this
    patch was for.
  * [Patch 12] Added documentation carefully stating the expected outputs
    of merge_trees(), merge_recursive(), and merge_recursive_generic()
    and how they differ.
  * [Patch 15, new] I noticed a case where we could accidentally lose
    output and the memory holding it for future callers of merge_trees()
    and fixed it.
  * [Patch 20 (used to be 19)] Renamed a var Dscho pointed out as needing
    a better name, added some asserts around how merge_trees() and
    merge_recursive() have opposite expectations to avoid future callers
    messing it up, and simplified the new t6047 testcase.

Stuff I'd most welcome review on:
  * Patch 7 again -- do my changes to write_index_as_tree() look sane?
  * Patches 5 & 15, since they are new and not reviewed last time.

Some notes (same as last time, but still true):
  * Applies on master, merges cleanly to next & pu
  * Only patches 3, 5-7 touch anything outside of merge-recursive
  * I'm going to be out next week (July 29-Aug 3), so I can only
    respond to feedback today and tomorrow or it'll have to wait until
    the 5th.

Elijah Newren (20):
  merge-recursive: fix minor memory leak in error condition
  merge-recursive: remove another implicit dependency on the_repository
  Ensure index matches head before invoking merge machinery, round N
  merge-recursive: exit early if index != head
  merge-recursive: remove useless parameter in merge_trees()
  merge-recursive: don't force external callers to do our logging
  Use write_index_as_tree() in lieu of write_tree_from_memory()
  merge-recursive: fix some overly long lines
  merge-recursive: use common name for ancestors/common/base_list
  merge-recursive: rename 'mrtree' to 'result_tree', for clarity
  merge-recursive: rename merge_options argument to opt in header
  merge-recursive: move some definitions around to clean up the header
  merge-recursive: consolidate unnecessary fields in merge_options
  merge-recursive: comment and reorder the merge_options fields
  merge-recursive: avoid losing output and leaking memory holding that
    output
  merge-recursive: split internal fields into a separate struct
  merge-recursive: alphabetize include list
  merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
  merge-recursive: be consistent with assert
  merge-recursive: provide a better label for diff3 common ancestor

 builtin/checkout.c                |  10 +-
 builtin/merge-recursive.c         |   4 +
 builtin/stash.c                   |   2 +
 cache-tree.c                      |  36 +-
 cache-tree.h                      |   3 +-
 merge-recursive.c                 | 536 ++++++++++++++++++------------
 merge-recursive.h                 | 160 +++++----
 sequencer.c                       |   5 +-
 t/t3030-merge-recursive.sh        |   9 +-
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 189 +++++++++++
 11 files changed, 646 insertions(+), 316 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

Range-diff:
 1:  28d4fb4710 =  1:  a640f0f2d0 merge-recursive: fix minor memory leak in error condition
 2:  5aa56bacce =  2:  34f0891d96 merge-recursive: remove another implicit dependency on the_repository
 3:  f38e2c4dcc =  3:  26739a7ed0 Ensure index matches head before invoking merge machinery, round N
 4:  858ec5c6e7 !  4:  76cb459b99 merge-recursive: exit early if index != head
    @@ -19,7 +19,8 @@
         other callers (which were fixed in the commit prior to this one).
     
         Make sure we do the index == head check at the beginning of the merge,
    -    and error out immediately if it fails.
    +    and error out immediately if it fails.  While we're at it, fix a small
    +    leak in the show-the-error codepath.
     
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
    @@ -111,6 +112,7 @@
     +	if (repo_index_has_changes(opt->repo, head, &sb)) {
     +		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
     +		    sb.buf);
    ++		strbuf_release(&sb);
     +		return -1;
     +	}
     +
    @@ -128,14 +130,14 @@
     +		struct tree *common,
     +		struct tree **result)
     +{
    -+	int ret;
    ++	int clean;
     +
     +	if (merge_start(opt, head))
     +		return -1;
    -+	ret = merge_trees_internal(opt, head, merge, common, result);
    ++	clean = merge_trees_internal(opt, head, merge, common, result);
     +	merge_finalize(opt);
     +
    -+	return ret;
    ++	return clean;
     +}
     +
     +int merge_recursive(struct merge_options *opt,
    @@ -144,14 +146,14 @@
     +		    struct commit_list *ca,
     +		    struct commit **result)
     +{
    -+	int ret;
    ++	int clean;
     +
     +	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
     +		return -1;
    -+	ret = merge_recursive_internal(opt, h1, h2, ca, result);
    ++	clean = merge_recursive_internal(opt, h1, h2, ca, result);
     +	merge_finalize(opt);
     +
    -+	return ret;
    ++	return clean;
     +}
     +
      static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
 -:  ---------- >  5:  2560458522 merge-recursive: remove useless parameter in merge_trees()
 5:  71c37a0928 =  6:  fb340fbe56 merge-recursive: don't force external callers to do our logging
 6:  286221dbf1 <  -:  ---------- Change call signature of write_tree_from_memory()
 7:  725bda6b1d <  -:  ---------- Move write_tree_from_memory() from merge-recursive to cache-tree
 -:  ---------- >  7:  884305a3a6 Use write_index_as_tree() in lieu of write_tree_from_memory()
 8:  d7fca78573 !  8:  870937b31f merge-recursive: fix some overly long lines
    @@ -54,7 +54,7 @@
      
      	for (iter = ca; iter; iter = iter->next) {
     @@
    - 	return ret;
    + 	return clean;
      }
      
     -static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
 9:  6090301564 !  9:  5127b75ac4 merge-recursive: use common name for ancestors/common/base_list
    @@ -146,19 +146,19 @@
      int merge_trees(struct merge_options *opt,
      		struct tree *head,
      		struct tree *merge,
    --		struct tree *common,
    -+		struct tree *merge_base,
    - 		struct tree **result)
    +-		struct tree *common)
    ++		struct tree *merge_base)
      {
    - 	int ret;
    + 	int clean;
    + 	struct tree *ignored;
      
      	if (merge_start(opt, head))
      		return -1;
    --	ret = merge_trees_internal(opt, head, merge, common, result);
    -+	ret = merge_trees_internal(opt, head, merge, merge_base, result);
    +-	clean = merge_trees_internal(opt, head, merge, common, &ignored);
    ++	clean = merge_trees_internal(opt, head, merge, merge_base, &ignored);
      	merge_finalize(opt);
      
    - 	return ret;
    + 	return clean;
     @@
      int merge_recursive(struct merge_options *opt,
      		    struct commit *h1,
    @@ -167,15 +167,15 @@
     +		    struct commit_list *merge_bases,
      		    struct commit **result)
      {
    - 	int ret;
    + 	int clean;
      
      	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
      		return -1;
    --	ret = merge_recursive_internal(opt, h1, h2, ca, result);
    -+	ret = merge_recursive_internal(opt, h1, h2, merge_bases, result);
    +-	clean = merge_recursive_internal(opt, h1, h2, ca, result);
    ++	clean = merge_recursive_internal(opt, h1, h2, merge_bases, result);
      	merge_finalize(opt);
      
    - 	return ret;
    + 	return clean;
     @@
      int merge_recursive_generic(struct merge_options *opt,
      			    const struct object_id *head,
    @@ -212,6 +212,16 @@
      --- a/merge-recursive.h
      +++ b/merge-recursive.h
     @@
    +  *
    +  * NOTE: empirically, about a decade ago it was determined that with more
    +  *       than two merge bases, optimal behavior was found when the
    +- *       ancestors were passed in the order of oldest merge base to newest
    +- *       one.  Also, ancestors will be consumed (emptied) so make a copy if
    +- *       you need it.
    ++ *       merge_bases were passed in the order of oldest commit to newest
    ++ *       commit.  Also, merge_bases will be consumed (emptied) so make a
    ++ *       copy if you need it.
    +  */
      int merge_recursive(struct merge_options *o,
      		    struct commit *h1,
      		    struct commit *h2,
    @@ -219,15 +229,16 @@
     +		    struct commit_list *merge_bases,
      		    struct commit **result);
      
    - /* rename-detecting three-way merge, no recursion */
    + /*
    +@@
      int merge_trees(struct merge_options *o,
      		struct tree *head,
      		struct tree *merge,
    --		struct tree *common,
    -+		struct tree *merge_base,
    - 		struct tree **result);
    +-		struct tree *common);
    ++		struct tree *merge_base);
      
      /*
    +  * "git-merge-recursive" can be fed trees; wrap them into
     @@
      int merge_recursive_generic(struct merge_options *o,
      			    const struct object_id *head,
10:  33228a4b3d = 10:  daee364ce1 merge-recursive: rename 'mrtree' to 'result_tree', for clarity
11:  3be41685ad ! 11:  50a7a6f671 merge-recursive: rename merge_options argument to opt in header
    @@ -25,20 +25,25 @@
     +		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
      }
      
    - /* merge_trees() but with recursive ancestor consolidation */
    + /*
    +@@
    +  *       commit.  Also, merge_bases will be consumed (emptied) so make a
    +  *       copy if you need it.
    +  */
     -int merge_recursive(struct merge_options *o,
     +int merge_recursive(struct merge_options *opt,
      		    struct commit *h1,
      		    struct commit *h2,
      		    struct commit_list *merge_bases,
    - 		    struct commit **result);
    - 
    - /* rename-detecting three-way merge, no recursion */
    +@@
    +  * rename-detecting three-way merge, no recursion; result of merge is written
    +  * to opt->repo->index.
    +  */
     -int merge_trees(struct merge_options *o,
     +int merge_trees(struct merge_options *opt,
      		struct tree *head,
      		struct tree *merge,
    - 		struct tree *merge_base,
    + 		struct tree *merge_base);
     @@
       * "git-merge-recursive" can be fed trees; wrap them into
       * virtual commits and call merge_recursive() proper.
12:  b1c396e505 ! 12:  4e9e774dc5 merge-recursive: move some definitions around to clean up the header
    @@ -74,7 +74,12 @@
      	struct repository *repo;
      };
      
    --/*
    ++void init_merge_options(struct merge_options *opt, struct repository *repo);
    ++
    ++/* parse the option in s and update the relevant field of opt */
    ++int parse_merge_opt(struct merge_options *opt, const char *s);
    ++
    + /*
     - * For dir_rename_entry, directory names are stored as a full path from the
     - * toplevel of the repository and do not include a trailing '/'.  Also:
     - *
    @@ -83,7 +88,12 @@
     - *   new_dir:            final name of directory being renamed
     - *   possible_new_dirs:  temporary used to help determine new_dir; see comments
     - *                       in get_directory_renames() for details
    -- */
    ++ * RETURN VALUES: All the merge_* functions below return a value as follows:
    ++ *   > 0     Merge was clean
    ++ *   = 0     Merge had conflicts
    ++ *   < 0     Merge hit an unexpected and unrecoverable problem (e.g. disk
    ++ *             full) and aborted merge part-way through.
    +  */
     -struct dir_rename_entry {
     -	struct hashmap_entry ent; /* must be the first member! */
     -	char *dir;
    @@ -91,47 +101,80 @@
     -	struct strbuf new_dir;
     -	struct string_list possible_new_dirs;
     -};
    -+void init_merge_options(struct merge_options *opt, struct repository *repo);
    - 
    +-
     -struct collision_entry {
     -	struct hashmap_entry ent; /* must be the first member! */
     -	char *target_file;
     -	struct string_list source_files;
     -	unsigned reported_already:1;
     -};
    -+/* parse the option in s and update the relevant field of opt */
    -+int parse_merge_opt(struct merge_options *opt, const char *s);
      
     -static inline int merge_detect_rename(struct merge_options *opt)
     -{
     -	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
     -		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
     -}
    -+/* rename-detecting three-way merge, no recursion */
    ++/*
    ++ * rename-detecting three-way merge, no recursion.
    ++ *
    ++ * Outputs:
    ++ *   - See RETURN VALUES above
    ++ *   - No commit is created
    ++ *   - opt->repo->index has the new index
    ++ *   - $GIT_INDEX_FILE is not updated
    ++ *   - The working tree is updated with results of the merge
    ++ */
     +int merge_trees(struct merge_options *opt,
     +		struct tree *head,
     +		struct tree *merge,
    -+		struct tree *merge_base,
    -+		struct tree **result);
    ++		struct tree *merge_base);
      
    - /* merge_trees() but with recursive ancestor consolidation */
    + /*
    +  * merge_recursive is like merge_trees() but with recursive ancestor
    +- * consolidation, and when successful, it creates an actual commit
    +- * and writes its address to *result.
    ++ * consolidation and, if the commit is clean, creation of a commit.
    +  *
    +  * NOTE: empirically, about a decade ago it was determined that with more
    +  *       than two merge bases, optimal behavior was found when the
    +  *       merge_bases were passed in the order of oldest commit to newest
    +  *       commit.  Also, merge_bases will be consumed (emptied) so make a
    +  *       copy if you need it.
    ++ *
    ++ * Outputs:
    ++ *   - See RETURN VALUES above
    ++ *   - If merge is clean, a commit is created and its address written to *result
    ++ *   - opt->repo->index has the new index
    ++ *   - $GIT_INDEX_FILE is not updated
    ++ *   - The working tree is updated with results of the merge
    +  */
      int merge_recursive(struct merge_options *opt,
    + 		    struct commit *h1,
     @@
    - 		    struct commit_list *merge_bases,
      		    struct commit **result);
      
    --/* rename-detecting three-way merge, no recursion */
    + /*
    +- * rename-detecting three-way merge, no recursion; result of merge is written
    +- * to opt->repo->index.
    +- */
     -int merge_trees(struct merge_options *opt,
     -		struct tree *head,
     -		struct tree *merge,
    --		struct tree *merge_base,
    --		struct tree **result);
    +-		struct tree *merge_base);
     -
    - /*
    +-/*
     - * "git-merge-recursive" can be fed trees; wrap them into
     - * virtual commits and call merge_recursive() proper.
     + * merge_recursive_generic can operate on trees instead of commits, by
     + * wrapping the trees into virtual commits, and calling merge_recursive().
    ++ * It also writes out the in-memory index to disk if the merge is successful.
    ++ *
    ++ * Outputs:
    ++ *   - See RETURN VALUES above
    ++ *   - If merge is clean, a commit is created and its address written to *result
    ++ *   - opt->repo->index has the new index
    ++ *   - $GIT_INDEX_FILE is updated
    ++ *   - The working tree is updated with results of the merge
       */
      int merge_recursive_generic(struct merge_options *opt,
      			    const struct object_id *head,
13:  bc653085af = 13:  bf40502fd8 merge-recursive: consolidate unnecessary fields in merge_options
14:  28a8880890 = 14:  2c39a4be36 merge-recursive: comment and reorder the merge_options fields
 -:  ---------- > 15:  c1c71816eb merge-recursive: avoid losing output and leaking memory holding that output
15:  8937e231d9 ! 16:  be47a6bfdf merge-recursive: split internal fields into a separate struct
    @@ -525,12 +525,11 @@
      
      	unpack_trees_finish(opt);
      
    --	if (opt->call_depth && !(*result = write_tree_from_memory(opt->repo)))
    -+	if (opt->priv->call_depth &&
    -+	    !(*result = write_tree_from_memory(opt->repo)))
    - 		return -1;
    - 
    - 	return clean;
    +-	if (opt->call_depth) {
    ++	if (opt->priv->call_depth) {
    + 		struct object_id tree_id;
    + 		if (write_index_as_tree(&tree_id, opt->repo->index, NULL,
    + 					WRITE_TREE_FROM_MEMORY, NULL) ||
     @@
      
      	for (iter = merge_bases; iter; iter = iter->next) {
    @@ -566,14 +565,6 @@
      		*result = make_virtual_commit(opt->repo, result_tree,
      					      "merged tree");
      		commit_list_insert(h1, &(*result)->parents);
    - 		commit_list_insert(h2, &(*result)->parents->next);
    - 	}
    - 	flush_output(opt);
    --	if (!opt->call_depth && opt->buffer_output < 2)
    -+	if (!opt->priv->call_depth && opt->buffer_output < 2)
    - 		strbuf_release(&opt->obuf);
    - 	return clean;
    - }
     @@
      		return -1;
      	}
    @@ -583,8 +574,12 @@
      	return 0;
      }
      
    -@@
    + static void merge_finalize(struct merge_options *opt)
      {
    + 	flush_output(opt);
    +-	if (!opt->call_depth && opt->buffer_output < 2)
    ++	if (!opt->priv->call_depth && opt->buffer_output < 2)
    + 		strbuf_release(&opt->obuf);
      	if (show(opt, 2))
      		diff_warn_rename_limit("merge.renamelimit",
     -				       opt->needed_rename_limit, 0);
16:  0ba049d6a2 = 17:  f440ee1e64 merge-recursive: alphabetize include list
17:  43eed3490b = 18:  40161dc352 merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
18:  3afc6d987a = 19:  d6158e555d merge-recursive: be consistent with assert
19:  ac7176acaf ! 20:  814a65ecab merge-recursive: provide a better label for diff3 common ancestor
    @@ -81,7 +81,7 @@
      	struct tree *result_tree;
      	int clean;
     +	int num_merge_bases;
    -+	struct strbuf commit_name = STRBUF_INIT;
    ++	struct strbuf merge_base_abbrev = STRBUF_INIT;
      
      	if (show(opt, 4)) {
      		output(opt, 4, _("Merging:"));
    @@ -103,10 +103,10 @@
     +		opt->ancestor = "<empty tree>";
     +		break;
     +	case 1:
    -+		strbuf_add_unique_abbrev(&commit_name,
    ++		strbuf_add_unique_abbrev(&merge_base_abbrev,
     +					 &merged_merge_bases->object.oid,
     +					 DEFAULT_ABBREV);
    -+		opt->ancestor = commit_name.buf;
    ++		opt->ancestor = merge_base_abbrev.buf;
     +		break;
     +	default:
     +		opt->ancestor = "merged common ancestors";
    @@ -117,10 +117,28 @@
      				     repo_get_commit_tree(opt->repo,
      							  merged_merge_bases),
      				     &result_tree);
    -+	strbuf_release(&commit_name);
    ++	strbuf_release(&merge_base_abbrev);
      	if (clean < 0) {
      		flush_output(opt);
      		return clean;
    +@@
    + 	int clean;
    + 	struct tree *ignored;
    + 
    ++	assert(opt->ancestor != NULL);
    ++
    + 	if (merge_start(opt, head))
    + 		return -1;
    + 	clean = merge_trees_internal(opt, head, merge, merge_base, &ignored);
    +@@
    + {
    + 	int clean;
    + 
    ++	assert(opt->ancestor == NULL);
    ++
    + 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
    + 		return -1;
    + 	clean = merge_recursive_internal(opt, h1, h2, merge_bases, result);
     
      diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
      --- a/t/t6036-recursive-corner-cases.sh
    @@ -196,20 +214,11 @@
     +	(
     +		cd no_merge_base &&
     +
    -+		git checkout --orphan L &&
    -+		test_seq 1 9 >content &&
    -+		echo "A" >>content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m "version L1 of content" &&
    ++		git checkout -b L &&
    ++		test_commit A content A &&
     +
    -+		# Create R
     +		git checkout --orphan R &&
    -+		test_seq 1 9 >content &&
    -+		echo "10" >>content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m "version R1 of content"
    ++		test_commit B content B
     +	)
     +'
     +
    @@ -241,30 +250,32 @@
     +	(
     +		cd unique_merge_base &&
     +
    -+		test_seq 1 9 >content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m initial &&
    ++		test_commit base content "1
    ++2
    ++3
    ++4
    ++5
    ++" &&
     +
     +		git branch L &&
     +		git branch R &&
     +
    -+		# Create L1
     +		git checkout L &&
    -+		test_seq 0 9 >content &&
    -+		echo "A" >>content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m "version L1 of content" &&
    ++		test_commit L content "1
    ++2
    ++3
    ++4
    ++5
    ++7" &&
     +
    -+		# Create R1
     +		git checkout R &&
    -+		test_seq 0 9 >content &&
    -+		echo "ten" >>content &&
    -+		git add content &&
    -+		git mv content renamed &&
    -+		test_tick &&
    -+		git commit -m "version R1 of content"
    ++		git rm content &&
    ++		test_commit R renamed "1
    ++2
    ++3
    ++4
    ++5
    ++six"
     +	)
     +'
     +
    @@ -300,30 +311,32 @@
     +	(
     +		cd multiple_merge_bases &&
     +
    -+		# Create some related files now
    -+		test_seq 1 9 >content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m initial &&
    ++		test_commit initial content "1
    ++2
    ++3
    ++4
    ++5" &&
     +
     +		git branch L &&
     +		git branch R &&
     +
     +		# Create L1
     +		git checkout L &&
    -+		test_seq 0 9 >content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m "version L1 of content" &&
    -+		git tag L1 &&
    ++		test_commit L1 content "0
    ++1
    ++2
    ++3
    ++4
    ++5" &&
     +
     +		# Create R1
     +		git checkout R &&
    -+		test_seq 1 10 >content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m "verson R1 of content" &&
    -+		git tag R1 &&
    ++		test_commit R1 content "1
    ++2
    ++3
    ++4
    ++5
    ++6" &&
     +
     +		# Create L2
     +		git checkout L &&
    @@ -335,20 +348,23 @@
     +
     +		# Create L3
     +		git checkout L &&
    -+		test_seq 0 9 >content &&
    -+		echo "A" >>content &&
    -+		git add content &&
    -+		test_tick &&
    -+		git commit -m "version L3 of content" &&
    ++		test_commit L3 content "0
    ++1
    ++2
    ++3
    ++4
    ++5
    ++A" &&
     +
     +		# Create R3
     +		git checkout R &&
    -+		test_seq 0 9 >content &&
    -+		echo "ten" >>content &&
    -+		git add content &&
    -+		git mv content renamed &&
    -+		test_tick &&
    -+		git commit -m "version R3 of content"
    ++		git rm content &&
    ++		test_commit R3 renamed "0
    ++2
    ++3
    ++4
    ++5
    ++six"
     +	)
     +'
     +
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 01/20] merge-recursive: fix minor memory leak in error condition

[Elijah Newren]

Returning before freeing the allocated buffer is suboptimal; as with
elsewhere in the same function, make sure buf gets free'd.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 12300131fc..1163508811 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -934,9 +934,11 @@ static int update_file_flags(struct merge_options *opt,
 		}
 
 		buf = read_object_file(&contents->oid, &type, &size);
-		if (!buf)
-			return err(opt, _("cannot read object %s '%s'"),
-				   oid_to_hex(&contents->oid), path);
+		if (!buf) {
+			ret = err(opt, _("cannot read object %s '%s'"),
+				  oid_to_hex(&contents->oid), path);
+			goto free_buf;
+		}
 		if (type != OBJ_BLOB) {
 			ret = err(opt, _("blob expected for %s '%s'"),
 				  oid_to_hex(&contents->oid), path);
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 02/20] merge-recursive: remove another implicit dependency on the_repository

[Elijah Newren]

Commit d7cf3a96e9a0 ("merge-recursive.c: remove implicit dependency on
the_repository", 2019-01-12) and follow-ups like commit 34e7771bc644
("Use the right 'struct repository' instead of the_repository",
2019-06-27), removed most implicit uses of the_repository.  Convert
calls to get_commit_tree() to instead use repo_get_commit_tree() to get
rid of another.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1163508811..37bb94fb4d 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3571,8 +3571,11 @@ int merge_recursive(struct merge_options *opt,
 		repo_read_index(opt->repo);
 
 	opt->ancestor = "merged common ancestors";
-	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
-			    get_commit_tree(merged_common_ancestors),
+	clean = merge_trees(opt,
+			    repo_get_commit_tree(opt->repo, h1),
+			    repo_get_commit_tree(opt->repo, h2),
+			    repo_get_commit_tree(opt->repo,
+						 merged_common_ancestors),
 			    &mrtree);
 	if (clean < 0) {
 		flush_output(opt);
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 03/20] Ensure index matches head before invoking merge machinery, round N

[Elijah Newren]

This is the bug that just won't die; there always seems to be another
form of it somewhere.  See the commit message of 55f39cf7551b ("merge:
fix misleading pre-merge check documentation", 2018-06-30) for a more
detailed explanation), but in short:

<quick summary>

builtin/merge.c contains this important requirement for merge
strategies:

    ...the index must be in sync with the head commit.  The strategies are
    responsible to ensure this.

This condition is important to enforce because there are two likely
failure cases when the index isn't in sync with the head commit:

  * we silently throw away changes the user had staged before the merge

  * we accidentally (and silently) include changes in the merge that
    were not part of either of the branches/trees being merged

Discarding users' work and mis-merging are both bad outcomes, especially
when done silently, so naturally this rule was stated sternly -- but,
unfortunately totally ignored in practice unless and until actual bugs
were found.  But, fear not: the bugs from this were fixed in commit
  ee6566e8d70d ("[PATCH] Rewrite read-tree", 2005-09-05)
through a rewrite of read-tree (again, commit 55f39cf7551b has a more
detailed explanation of how this affected merge).  And it was fixed
again in commit
  160252f81626 ("git-merge-ours: make sure our index matches HEAD", 2005-11-03)
...and it was fixed again in commit
  3ec62ad9ffba ("merge-octopus: abort if index does not match HEAD", 2016-04-09)
...and again in commit
  65170c07d466 ("merge-recursive: avoid incorporating uncommitted changes in a merge", 2017-12-21)
...and again in commit
  eddd1a411d93 ("merge-recursive: enforce rule that index matches head before merging", 2018-06-30)

...with multiple testcases added to the testsuite that could be
enumerated in even more commits.

Then, finally, in a patch in the same series as the last fix above, the
documentation about this requirement was fixed in commit 55f39cf7551b
("merge: fix misleading pre-merge check documentation", 2018-06-30), and
we all lived happily ever after...

</quick summary>

Unfortunately, "ever after" apparently denotes a limited time and it
expired today.  The merge-recursive rule to enforce that index matches
head was at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially HUGE amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Trying to enforce that all of merge_trees(), merge_recursive(), and
merge_recursive_generic() checked the index == head condition earlier
resulted in a bunch of broken tests.  It turns out that
merge_recursive() has code to drop and reload the cache while recursing
to create intermediate virtual merge bases, but unfortunately that code
runs even when no recursion is necessary.  This unconditional dropping
and reloading of the cache masked a few bugs:

  * builtin/merge-recursive.c: didn't even bother loading the index.

  * builtin/stash.c: feels like a fake 'builtin' because it repeatedly
    invokes git subprocesses all over the place, mixed with other
    operations.  In particular, invoking "git reset" will reset the
    index on disk, but the parent process that invoked it won't
    automatically have its in-memory index updated.

  * t3030-merge-recursive.h: this test has always been broken in that it
    didn't make sure to make index match head before running.  But, it
    didn't care about the index or even the merge result, just the
    verbose output while running.  While commit eddd1a411d93
    ("merge-recursive: enforce rule that index matches head before
    merging", 2018-06-30) should have uncovered this broken test, it
    used a test_must_fail wrapper around the merge-recursive call
    because it was known that the merge resulted in a rename/rename
    conflict.  Thus, that fix only made this test fail for a different
    reason, and since the index == head check didn't happen until after
    coming all the way back out of the recursion, the testcase had
    enough information to pass the one check that it did perform.

So, load the index in builtin/merge-recursive.c, reload the in-memory
index in builtin/stash.c, and modify the t3030 testcase to correctly
setup the index and make sure that the test fails in the expected way
(meaning it reports a rename/rename conflict).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-recursive.c  | 4 ++++
 builtin/stash.c            | 2 ++
 t/t3030-merge-recursive.sh | 9 ++++++++-
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
index 5b910e351e..a4bfd8fc51 100644
--- a/builtin/merge-recursive.c
+++ b/builtin/merge-recursive.c
@@ -1,3 +1,4 @@
+#include "cache.h"
 #include "builtin.h"
 #include "commit.h"
 #include "tag.h"
@@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
 	if (argc - i != 3) /* "--" "<head>" "<remote>" */
 		die(_("not handling anything other than two heads merge."));
 
+	if (repo_read_index_unmerged(the_repository))
+		die_resolve_conflict("merge");
+
 	o.branch1 = argv[++i];
 	o.branch2 = argv[++i];
 
diff --git a/builtin/stash.c b/builtin/stash.c
index b5a301f24d..4aa47785f9 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -427,6 +427,8 @@ static int do_apply_stash(const char *prefix, struct stash_info *info,
 				return error(_("could not save index tree"));
 
 			reset_head();
+			discard_cache();
+			read_cache();
 		}
 	}
 
diff --git a/t/t3030-merge-recursive.sh b/t/t3030-merge-recursive.sh
index ff641b348a..a37bcc58a0 100755
--- a/t/t3030-merge-recursive.sh
+++ b/t/t3030-merge-recursive.sh
@@ -667,15 +667,22 @@ test_expect_success 'merging with triple rename across D/F conflict' '
 test_expect_success 'merge-recursive remembers the names of all base trees' '
 	git reset --hard HEAD &&
 
+	# make the index match $c1 so that merge-recursive below does not
+	# fail early
+	git diff --binary HEAD $c1 -- | git apply --cached &&
+
 	# more trees than static slots used by oid_to_hex()
 	for commit in $c0 $c2 $c4 $c5 $c6 $c7
 	do
 		git rev-parse "$commit^{tree}"
 	done >trees &&
 
-	# ignore the return code -- it only fails because the input is weird
+	# ignore the return code; it only fails because the input is weird...
 	test_must_fail git -c merge.verbosity=5 merge-recursive $(cat trees) -- $c1 $c3 >out &&
 
+	# ...but make sure it fails in the expected way
+	test_i18ngrep CONFLICT.*rename/rename out &&
+
 	# merge-recursive prints in reverse order, but we do not care
 	sort <trees >expect &&
 	sed -n "s/^virtual //p" out | sort >actual &&
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 04/20] merge-recursive: exit early if index != head

[Elijah Newren]

We had a rule to enforce that the index matches head, but it was found
at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially huge amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Further, not enforcing this requirement earlier allowed other bugs (such
as an unintentional unconditional dropping and reloading of the index in
merge_recursive() even when no recursion was necessary), to mask bugs in
other callers (which were fixed in the commit prior to this one).

Make sure we do the index == head check at the beginning of the merge,
and error out immediately if it fails.  While we're at it, fix a small
leak in the show-the-error codepath.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 94 +++++++++++++++++++++++++++++++++++------------
 1 file changed, 70 insertions(+), 24 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 37bb94fb4d..f5cf21e2e0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3381,21 +3381,14 @@ static int process_entry(struct merge_options *opt,
 	return clean_merge;
 }
 
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *common,
-		struct tree **result)
+static int merge_trees_internal(struct merge_options *opt,
+				struct tree *head,
+				struct tree *merge,
+				struct tree *common,
+				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
-	struct strbuf sb = STRBUF_INIT;
-
-	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
-		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
-		    sb.buf);
-		return -1;
-	}
 
 	if (opt->subtree_shift) {
 		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
@@ -3499,11 +3492,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
  * Merge the commits h1 and h2, return the resulting virtual
  * commit object and a flag indicating the cleanness of the merge.
  */
-int merge_recursive(struct merge_options *opt,
-		    struct commit *h1,
-		    struct commit *h2,
-		    struct commit_list *ca,
-		    struct commit **result)
+static int merge_recursive_internal(struct merge_options *opt,
+				    struct commit *h1,
+				    struct commit *h2,
+				    struct commit_list *ca,
+				    struct commit **result)
 {
 	struct commit_list *iter;
 	struct commit *merged_common_ancestors;
@@ -3555,7 +3548,7 @@ int merge_recursive(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive(opt, merged_common_ancestors, iter->item,
+		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
 				    NULL, &merged_common_ancestors) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
@@ -3571,12 +3564,12 @@ int merge_recursive(struct merge_options *opt,
 		repo_read_index(opt->repo);
 
 	opt->ancestor = "merged common ancestors";
-	clean = merge_trees(opt,
-			    repo_get_commit_tree(opt->repo, h1),
-			    repo_get_commit_tree(opt->repo, h2),
-			    repo_get_commit_tree(opt->repo,
-						 merged_common_ancestors),
-			    &mrtree);
+	clean = merge_trees_internal(opt,
+				     repo_get_commit_tree(opt->repo, h1),
+				     repo_get_commit_tree(opt->repo, h2),
+				     repo_get_commit_tree(opt->repo,
+							  merged_common_ancestors),
+				     &mrtree);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
@@ -3596,6 +3589,59 @@ int merge_recursive(struct merge_options *opt,
 	return clean;
 }
 
+static int merge_start(struct merge_options *opt, struct tree *head)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	assert(opt->branch1 && opt->branch2);
+
+	if (repo_index_has_changes(opt->repo, head, &sb)) {
+		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
+		    sb.buf);
+		strbuf_release(&sb);
+		return -1;
+	}
+
+	return 0;
+}
+
+static void merge_finalize(struct merge_options *opt)
+{
+	/* Common code for wrapping up merges will be added here later */
+}
+
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *common,
+		struct tree **result)
+{
+	int clean;
+
+	if (merge_start(opt, head))
+		return -1;
+	clean = merge_trees_internal(opt, head, merge, common, result);
+	merge_finalize(opt);
+
+	return clean;
+}
+
+int merge_recursive(struct merge_options *opt,
+		    struct commit *h1,
+		    struct commit *h2,
+		    struct commit_list *ca,
+		    struct commit **result)
+{
+	int clean;
+
+	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
+		return -1;
+	clean = merge_recursive_internal(opt, h1, h2, ca, result);
+	merge_finalize(opt);
+
+	return clean;
+}
+
 static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
 			      const char *name)
 {
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 05/20] merge-recursive: remove useless parameter in merge_trees()

[Elijah Newren]

merge_trees() took a results parameter that would only be written when
opt->call_depth was positive, which is never the case now that
merge_trees_internal() has been split from merge_trees().  Remove the
misleading and unused parameter from merge_trees().

While at it, add some comments explaining how the output of
merge_trees() and merge_recursive() differ.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c |  4 +---
 merge-recursive.c  |  6 +++---
 merge-recursive.h  | 20 ++++++++++++++++----
 sequencer.c        |  4 ++--
 4 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 91f8509f85..20e38c5edc 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -708,7 +708,6 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 * give up or do a real merge, depending on
 			 * whether the merge flag was used.
 			 */
-			struct tree *result;
 			struct tree *work;
 			struct tree *old_tree;
 			struct merge_options o;
@@ -773,8 +772,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			ret = merge_trees(&o,
 					  new_tree,
 					  work,
-					  old_tree,
-					  &result);
+					  old_tree);
 			if (ret < 0)
 				exit(128);
 			ret = reset_tree(new_tree,
diff --git a/merge-recursive.c b/merge-recursive.c
index f5cf21e2e0..4a481c3929 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3613,14 +3613,14 @@ static void merge_finalize(struct merge_options *opt)
 int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
-		struct tree **result)
+		struct tree *common)
 {
 	int clean;
+	struct tree *ignored;
 
 	if (merge_start(opt, head))
 		return -1;
-	clean = merge_trees_internal(opt, head, merge, common, result);
+	clean = merge_trees_internal(opt, head, merge, common, &ignored);
 	merge_finalize(opt);
 
 	return clean;
diff --git a/merge-recursive.h b/merge-recursive.h
index c2b7bb65c6..812c456f1b 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -70,19 +70,31 @@ static inline int merge_detect_rename(struct merge_options *o)
 		o->diff_detect_rename >= 0 ? o->diff_detect_rename : 1;
 }
 
-/* merge_trees() but with recursive ancestor consolidation */
+/*
+ * merge_recursive is like merge_trees() but with recursive ancestor
+ * consolidation, and when successful, it creates an actual commit
+ * and writes its address to *result.
+ *
+ * NOTE: empirically, about a decade ago it was determined that with more
+ *       than two merge bases, optimal behavior was found when the
+ *       ancestors were passed in the order of oldest merge base to newest
+ *       one.  Also, ancestors will be consumed (emptied) so make a copy if
+ *       you need it.
+ */
 int merge_recursive(struct merge_options *o,
 		    struct commit *h1,
 		    struct commit *h2,
 		    struct commit_list *ancestors,
 		    struct commit **result);
 
-/* rename-detecting three-way merge, no recursion */
+/*
+ * rename-detecting three-way merge, no recursion; result of merge is written
+ * to opt->repo->index.
+ */
 int merge_trees(struct merge_options *o,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
-		struct tree **result);
+		struct tree *common);
 
 /*
  * "git-merge-recursive" can be fed trees; wrap them into
diff --git a/sequencer.c b/sequencer.c
index 34ebf8ed94..c4ed30f1b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -586,7 +586,7 @@ static int do_recursive_merge(struct repository *r,
 			      struct replay_opts *opts)
 {
 	struct merge_options o;
-	struct tree *result, *next_tree, *base_tree, *head_tree;
+	struct tree *next_tree, *base_tree, *head_tree;
 	int clean;
 	char **xopt;
 	struct lock_file index_lock = LOCK_INIT;
@@ -613,7 +613,7 @@ static int do_recursive_merge(struct repository *r,
 
 	clean = merge_trees(&o,
 			    head_tree,
-			    next_tree, base_tree, &result);
+			    next_tree, base_tree);
 	if (is_rebase_i(opts) && clean <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 06/20] merge-recursive: don't force external callers to do our logging

[Elijah Newren]

Alternatively, you can view this as "make the merge functions behave
more similarly."  merge-recursive has three different entry points:
merge_trees(), merge_recursive(), and merge_recursive_generic().  Two of
these would call diff_warn_rename_limit(), but merge_trees() didn't.
This lead to callers of merge_trees() needing to manually call
diff_warn_rename_limit() themselves.  Move this to the new
merge_finalize() function to make sure that all three entry points run
this function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++----
 sequencer.c       | 1 -
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 4a481c3929..308e350ff1 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3583,9 +3583,6 @@ static int merge_recursive_internal(struct merge_options *opt,
 	flush_output(opt);
 	if (!opt->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
-	if (show(opt, 2))
-		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
 	return clean;
 }
 
@@ -3607,7 +3604,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 
 static void merge_finalize(struct merge_options *opt)
 {
-	/* Common code for wrapping up merges will be added here later */
+	if (show(opt, 2))
+		diff_warn_rename_limit("merge.renamelimit",
+				       opt->needed_rename_limit, 0);
 }
 
 int merge_trees(struct merge_options *opt,
diff --git a/sequencer.c b/sequencer.c
index c4ed30f1b4..094a4dd03d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -617,7 +617,6 @@ static int do_recursive_merge(struct repository *r,
 	if (is_rebase_i(opts) && clean <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
-	diff_warn_rename_limit("merge.renamelimit", o.needed_rename_limit, 0);
 	if (clean < 0) {
 		rollback_lock_file(&index_lock);
 		return clean;
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 07/20] Use write_index_as_tree() in lieu of write_tree_from_memory()

[Elijah Newren]

write_tree_from_memory() appeared to be a merge-recursive special that
basically duplicated write_index_as_tree().  The two had a slightly
different call structure but the big difference was just that
write_index_as_tree() would always unconditionally read the index off of
disk instead of working on the current in-memory index.  Add a flag to
allow using the in-memory index, and then switch over to
write_index_as_tree().

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c |  6 +++++-
 cache-tree.c       | 36 +++++++++++++++++++++++-------------
 cache-tree.h       |  3 ++-
 merge-recursive.c  | 42 +++++++++---------------------------------
 merge-recursive.h  |  1 -
 5 files changed, 39 insertions(+), 49 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 20e38c5edc..a7b6454061 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -709,6 +709,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 * whether the merge flag was used.
 			 */
 			struct tree *work;
+			struct object_id work_oid;
 			struct tree *old_tree;
 			struct merge_options o;
 			struct strbuf sb = STRBUF_INIT;
@@ -759,7 +760,10 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 */
 			init_merge_options(&o, the_repository);
 			o.verbosity = 0;
-			work = write_tree_from_memory(&o);
+			if (write_index_as_tree(&work_oid, &the_index, NULL,
+					WRITE_TREE_FROM_MEMORY, NULL) ||
+			    !(work = lookup_tree(the_repository, &work_oid)))
+				die(_("error building trees"));
 
 			ret = reset_tree(new_tree,
 					 opts, 1,
diff --git a/cache-tree.c b/cache-tree.c
index 706ffcf188..99144b1704 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -613,14 +613,19 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 	int entries, was_valid;
 	struct lock_file lock_file = LOCK_INIT;
 	int ret = 0;
+	int access_disk = !(flags & WRITE_TREE_FROM_MEMORY);
 
-	hold_lock_file_for_update(&lock_file, index_path, LOCK_DIE_ON_ERROR);
+	if (access_disk) {
+		hold_lock_file_for_update(&lock_file, index_path,
+					  LOCK_DIE_ON_ERROR);
 
-	entries = read_index_from(index_state, index_path, get_git_dir());
-	if (entries < 0) {
-		ret = WRITE_TREE_UNREADABLE_INDEX;
-		goto out;
+		entries = read_index_from(index_state, index_path, get_git_dir());
+		if (entries < 0) {
+			ret = WRITE_TREE_UNREADABLE_INDEX;
+			goto out;
+		}
 	}
+
 	if (flags & WRITE_TREE_IGNORE_CACHE_TREE)
 		cache_tree_free(&index_state->cache_tree);
 
@@ -633,13 +638,16 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 			ret = WRITE_TREE_UNMERGED_INDEX;
 			goto out;
 		}
-		write_locked_index(index_state, &lock_file, COMMIT_LOCK);
-		/* Not being able to write is fine -- we are only interested
-		 * in updating the cache-tree part, and if the next caller
-		 * ends up using the old index with unupdated cache-tree part
-		 * it misses the work we did here, but that is just a
-		 * performance penalty and not a big deal.
-		 */
+		if (access_disk) {
+			write_locked_index(index_state, &lock_file, COMMIT_LOCK);
+			/* Not being able to write is fine -- we are only
+			 * interested in updating the cache-tree part, and if
+			 * the next caller ends up using the old index with
+			 * unupdated cache-tree part it misses the work we
+			 * did here, but that is just a performance penalty
+			 * and not a big deal.
+			 */
+		}
 	}
 
 	if (prefix) {
@@ -655,7 +663,9 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 		oidcpy(oid, &index_state->cache_tree->oid);
 
 out:
-	rollback_lock_file(&lock_file);
+	if (access_disk)
+		rollback_lock_file(&lock_file);
+
 	return ret;
 }
 
diff --git a/cache-tree.h b/cache-tree.h
index 757bbc48bc..481ec45dfa 100644
--- a/cache-tree.h
+++ b/cache-tree.h
@@ -34,12 +34,13 @@ int cache_tree_fully_valid(struct cache_tree *);
 int cache_tree_update(struct index_state *, int);
 void cache_tree_verify(struct repository *, struct index_state *);
 
-/* bitmasks to write_cache_as_tree flags */
+/* bitmasks to write_index_as_tree flags */
 #define WRITE_TREE_MISSING_OK 1
 #define WRITE_TREE_IGNORE_CACHE_TREE 2
 #define WRITE_TREE_DRY_RUN 4
 #define WRITE_TREE_SILENT 8
 #define WRITE_TREE_REPAIR 16
+#define WRITE_TREE_FROM_MEMORY 32
 
 /* error return codes */
 #define WRITE_TREE_UNREADABLE_INDEX (-1)
diff --git a/merge-recursive.c b/merge-recursive.c
index 308e350ff1..6c38c02e3f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -412,37 +412,6 @@ static void unpack_trees_finish(struct merge_options *opt)
 	clear_unpack_trees_porcelain(&opt->unpack_opts);
 }
 
-struct tree *write_tree_from_memory(struct merge_options *opt)
-{
-	struct tree *result = NULL;
-	struct index_state *istate = opt->repo->index;
-
-	if (unmerged_index(istate)) {
-		int i;
-		fprintf(stderr, "BUG: There are unmerged index entries:\n");
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
-			if (ce_stage(ce))
-				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
-					(int)ce_namelen(ce), ce->name);
-		}
-		BUG("unmerged index entries in merge-recursive.c");
-	}
-
-	if (!istate->cache_tree)
-		istate->cache_tree = cache_tree();
-
-	if (!cache_tree_fully_valid(istate->cache_tree) &&
-	    cache_tree_update(istate, 0) < 0) {
-		err(opt, _("error building trees"));
-		return NULL;
-	}
-
-	result = lookup_tree(opt->repo, &istate->cache_tree->oid);
-
-	return result;
-}
-
 static int save_files_dirs(const struct object_id *oid,
 			   struct strbuf *base, const char *path,
 			   unsigned int mode, int stage, void *context)
@@ -3471,8 +3440,15 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth && !(*result = write_tree_from_memory(opt)))
-		return -1;
+	if (opt->call_depth) {
+		struct object_id tree_id;
+		if (write_index_as_tree(&tree_id, opt->repo->index, NULL,
+					WRITE_TREE_FROM_MEMORY, NULL) ||
+		    !(*result = lookup_tree(opt->repo, &tree_id))) {
+			err(opt, _("error building trees"));
+			return -1;
+		}
+	}
 
 	return clean;
 }
diff --git a/merge-recursive.h b/merge-recursive.h
index 812c456f1b..b3394502c7 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -109,7 +109,6 @@ int merge_recursive_generic(struct merge_options *o,
 
 void init_merge_options(struct merge_options *o,
 			struct repository *repo);
-struct tree *write_tree_from_memory(struct merge_options *o);
 
 int parse_merge_opt(struct merge_options *out, const char *s);
 
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 08/20] merge-recursive: fix some overly long lines

[Elijah Newren]

No substantive code change, just add some line breaks to fix lines that
have grown in length due to various refactorings.  Most remaining lines
of excessive length in merge-recursive include error messages and it's
not clear that splitting those improves things.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 6c38c02e3f..e81dec8f1f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -681,7 +681,9 @@ static void add_flattened_path(struct strbuf *out, const char *s)
 			out->buf[i] = '_';
 }
 
-static char *unique_path(struct merge_options *opt, const char *path, const char *branch)
+static char *unique_path(struct merge_options *opt,
+			 const char *path,
+			 const char *branch)
 {
 	struct path_hashmap_entry *entry;
 	struct strbuf newpath = STRBUF_INIT;
@@ -915,7 +917,8 @@ static int update_file_flags(struct merge_options *opt,
 		}
 		if (S_ISREG(contents->mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
-			if (convert_to_working_tree(opt->repo->index, path, buf, size, &strbuf)) {
+			if (convert_to_working_tree(opt->repo->index,
+						    path, buf, size, &strbuf)) {
 				free(buf);
 				size = strbuf.len;
 				buf = strbuf_detach(&strbuf, NULL);
@@ -3392,7 +3395,8 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp, NULL, 512);
+		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
 
@@ -3505,7 +3509,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo, tree, "ancestor");
+		merged_common_ancestors = make_virtual_commit(opt->repo,
+							      tree, "ancestor");
 	}
 
 	for (iter = ca; iter; iter = iter->next) {
@@ -3617,7 +3622,8 @@ int merge_recursive(struct merge_options *opt,
 	return clean;
 }
 
-static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
+static struct commit *get_ref(struct repository *repo,
+			      const struct object_id *oid,
 			      const char *name)
 {
 	struct object *object;
@@ -3652,7 +3658,8 @@ int merge_recursive_generic(struct merge_options *opt,
 		int i;
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i], oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, base_list[i],
+					     oid_to_hex(base_list[i]))))
 				return err(opt, _("Could not parse object '%s'"),
 					   oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 09/20] merge-recursive: use common name for ancestors/common/base_list

[Elijah Newren]

merge_trees(), merge_recursive(), and merge_recursive_generic() in
their function headers used four different names for the merge base or
list of merge bases they were passed:
  * 'common'
  * 'ancestors'
  * 'ca'
  * 'base_list'
They were able to refer to it four different ways instead of only three
by using a different name in the signature for the .c file than the .h
file.  Change all of these to 'merge_base' or 'merge_bases'.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 68 ++++++++++++++++++++++++-----------------------
 merge-recursive.h | 14 +++++-----
 2 files changed, 42 insertions(+), 40 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index e81dec8f1f..8114e77b98 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3356,24 +3356,26 @@ static int process_entry(struct merge_options *opt,
 static int merge_trees_internal(struct merge_options *opt,
 				struct tree *head,
 				struct tree *merge,
-				struct tree *common,
+				struct tree *merge_base,
 				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
 
 	if (opt->subtree_shift) {
-		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
-		common = shift_tree_object(opt->repo, head, common, opt->subtree_shift);
+		merge = shift_tree_object(opt->repo, head, merge,
+					  opt->subtree_shift);
+		merge_base = shift_tree_object(opt->repo, head, merge_base,
+					       opt->subtree_shift);
 	}
 
-	if (oid_eq(&common->object.oid, &merge->object.oid)) {
+	if (oid_eq(&merge_base->object.oid, &merge->object.oid)) {
 		output(opt, 0, _("Already up to date!"));
 		*result = head;
 		return 1;
 	}
 
-	code = unpack_trees_start(opt, common, head, merge);
+	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
 		if (show(opt, 4) || opt->call_depth)
@@ -3401,7 +3403,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		get_files_dirs(opt, merge);
 
 		entries = get_unmerged(opt->repo->index);
-		clean = detect_and_process_renames(opt, common, head, merge,
+		clean = detect_and_process_renames(opt, merge_base, head, merge,
 						   entries, &re_info);
 		record_df_conflict_files(opt, entries);
 		if (clean < 0)
@@ -3475,11 +3477,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
 static int merge_recursive_internal(struct merge_options *opt,
 				    struct commit *h1,
 				    struct commit *h2,
-				    struct commit_list *ca,
+				    struct commit_list *merge_bases,
 				    struct commit **result)
 {
 	struct commit_list *iter;
-	struct commit *merged_common_ancestors;
+	struct commit *merged_merge_bases;
 	struct tree *mrtree;
 	int clean;
 
@@ -3489,31 +3491,31 @@ static int merge_recursive_internal(struct merge_options *opt,
 		output_commit_title(opt, h2);
 	}
 
-	if (!ca) {
-		ca = get_merge_bases(h1, h2);
-		ca = reverse_commit_list(ca);
+	if (!merge_bases) {
+		merge_bases = get_merge_bases(h1, h2);
+		merge_bases = reverse_commit_list(merge_bases);
 	}
 
 	if (show(opt, 5)) {
-		unsigned cnt = commit_list_count(ca);
+		unsigned cnt = commit_list_count(merge_bases);
 
 		output(opt, 5, Q_("found %u common ancestor:",
 				"found %u common ancestors:", cnt), cnt);
-		for (iter = ca; iter; iter = iter->next)
+		for (iter = merge_bases; iter; iter = iter->next)
 			output_commit_title(opt, iter->item);
 	}
 
-	merged_common_ancestors = pop_commit(&ca);
-	if (merged_common_ancestors == NULL) {
+	merged_merge_bases = pop_commit(&merge_bases);
+	if (merged_merge_bases == NULL) {
 		/* if there is no common ancestor, use an empty tree */
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo,
-							      tree, "ancestor");
+		merged_merge_bases = make_virtual_commit(opt->repo, tree,
+							 "ancestor");
 	}
 
-	for (iter = ca; iter; iter = iter->next) {
+	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
 		opt->call_depth++;
 		/*
@@ -3529,14 +3531,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
-				    NULL, &merged_common_ancestors) < 0)
+		if (merge_recursive_internal(opt, merged_merge_bases, iter->item,
+					     NULL, &merged_merge_bases) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
 		opt->call_depth--;
 
-		if (!merged_common_ancestors)
+		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
@@ -3549,7 +3551,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h1),
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
-							  merged_common_ancestors),
+							  merged_merge_bases),
 				     &mrtree);
 	if (clean < 0) {
 		flush_output(opt);
@@ -3593,14 +3595,14 @@ static void merge_finalize(struct merge_options *opt)
 int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common)
+		struct tree *merge_base)
 {
 	int clean;
 	struct tree *ignored;
 
 	if (merge_start(opt, head))
 		return -1;
-	clean = merge_trees_internal(opt, head, merge, common, &ignored);
+	clean = merge_trees_internal(opt, head, merge, merge_base, &ignored);
 	merge_finalize(opt);
 
 	return clean;
@@ -3609,14 +3611,14 @@ int merge_trees(struct merge_options *opt,
 int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ca,
+		    struct commit_list *merge_bases,
 		    struct commit **result)
 {
 	int clean;
 
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
-	clean = merge_recursive_internal(opt, h1, h2, ca, result);
+	clean = merge_recursive_internal(opt, h1, h2, merge_bases, result);
 	merge_finalize(opt);
 
 	return clean;
@@ -3644,8 +3646,8 @@ static struct commit *get_ref(struct repository *repo,
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_base_list,
-			    const struct object_id **base_list,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result)
 {
 	int clean;
@@ -3654,14 +3656,14 @@ int merge_recursive_generic(struct merge_options *opt,
 	struct commit *next_commit = get_ref(opt->repo, merge, opt->branch2);
 	struct commit_list *ca = NULL;
 
-	if (base_list) {
+	if (merge_bases) {
 		int i;
-		for (i = 0; i < num_base_list; ++i) {
+		for (i = 0; i < num_merge_bases; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i],
-					     oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, merge_bases[i],
+					     oid_to_hex(merge_bases[i]))))
 				return err(opt, _("Could not parse object '%s'"),
-					   oid_to_hex(base_list[i]));
+					   oid_to_hex(merge_bases[i]));
 			commit_list_insert(base, &ca);
 		}
 	}
diff --git a/merge-recursive.h b/merge-recursive.h
index b3394502c7..c7ba8d0a71 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -77,14 +77,14 @@ static inline int merge_detect_rename(struct merge_options *o)
  *
  * NOTE: empirically, about a decade ago it was determined that with more
  *       than two merge bases, optimal behavior was found when the
- *       ancestors were passed in the order of oldest merge base to newest
- *       one.  Also, ancestors will be consumed (emptied) so make a copy if
- *       you need it.
+ *       merge_bases were passed in the order of oldest commit to newest
+ *       commit.  Also, merge_bases will be consumed (emptied) so make a
+ *       copy if you need it.
  */
 int merge_recursive(struct merge_options *o,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ancestors,
+		    struct commit_list *merge_bases,
 		    struct commit **result);
 
 /*
@@ -94,7 +94,7 @@ int merge_recursive(struct merge_options *o,
 int merge_trees(struct merge_options *o,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common);
+		struct tree *merge_base);
 
 /*
  * "git-merge-recursive" can be fed trees; wrap them into
@@ -103,8 +103,8 @@ int merge_trees(struct merge_options *o,
 int merge_recursive_generic(struct merge_options *o,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_ca,
-			    const struct object_id **ca,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result);
 
 void init_merge_options(struct merge_options *o,
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 10/20] merge-recursive: rename 'mrtree' to 'result_tree', for clarity

[Elijah Newren]

It is not at all clear what 'mr' was supposed to stand for, at least not
to me.  Pick a clearer name for this variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 8114e77b98..f36962b7b5 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3482,7 +3482,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 {
 	struct commit_list *iter;
 	struct commit *merged_merge_bases;
-	struct tree *mrtree;
+	struct tree *result_tree;
 	int clean;
 
 	if (show(opt, 4)) {
@@ -3552,14 +3552,15 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
 							  merged_merge_bases),
-				     &mrtree);
+				     &result_tree);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
 	}
 
 	if (opt->call_depth) {
-		*result = make_virtual_commit(opt->repo, mrtree, "merged tree");
+		*result = make_virtual_commit(opt->repo, result_tree,
+					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 11/20] merge-recursive: rename merge_options argument to opt in header

[Elijah Newren]

In commit 259ccb6cc324 ("merge-recursive: rename merge_options argument
from 'o' to 'opt'", 2019-04-05), I renamed a bunch of function
arguments in merge-recursive.c, but forgot to make that same change to
merge-recursive.h.  Make the two match.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.h | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/merge-recursive.h b/merge-recursive.h
index c7ba8d0a71..c201de0fc3 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -64,10 +64,10 @@ struct collision_entry {
 	unsigned reported_already:1;
 };
 
-static inline int merge_detect_rename(struct merge_options *o)
+static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return o->merge_detect_rename >= 0 ? o->merge_detect_rename :
-		o->diff_detect_rename >= 0 ? o->diff_detect_rename : 1;
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
 }
 
 /*
@@ -81,7 +81,7 @@ static inline int merge_detect_rename(struct merge_options *o)
  *       commit.  Also, merge_bases will be consumed (emptied) so make a
  *       copy if you need it.
  */
-int merge_recursive(struct merge_options *o,
+int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
 		    struct commit_list *merge_bases,
@@ -91,7 +91,7 @@ int merge_recursive(struct merge_options *o,
  * rename-detecting three-way merge, no recursion; result of merge is written
  * to opt->repo->index.
  */
-int merge_trees(struct merge_options *o,
+int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
 		struct tree *merge_base);
@@ -100,16 +100,16 @@ int merge_trees(struct merge_options *o,
  * "git-merge-recursive" can be fed trees; wrap them into
  * virtual commits and call merge_recursive() proper.
  */
-int merge_recursive_generic(struct merge_options *o,
+int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
 			    int num_merge_bases,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *o,
+void init_merge_options(struct merge_options *opt,
 			struct repository *repo);
 
-int parse_merge_opt(struct merge_options *out, const char *s);
+int parse_merge_opt(struct merge_options *opt, const char *s);
 
 #endif
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 12/20] merge-recursive: move some definitions around to clean up the header

[Elijah Newren]

No substantive code changes (view this with diff --color-moved), but
a few small code cleanups:
  * Move structs and an inline function only used by merge-recursive.c
    into merge-recursive.c
  * Re-order function declarations to be more logical
  * Add or fix some explanatory comments

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 31 +++++++++++++++++
 merge-recursive.h | 87 +++++++++++++++++++++++------------------------
 2 files changed, 73 insertions(+), 45 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index f36962b7b5..6820578258 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -54,6 +54,24 @@ static unsigned int path_hash(const char *path)
 	return ignore_case ? strihash(path) : strhash(path);
 }
 
+/*
+ * For dir_rename_entry, directory names are stored as a full path from the
+ * toplevel of the repository and do not include a trailing '/'.  Also:
+ *
+ *   dir:                original name of directory being renamed
+ *   non_unique_new_dir: if true, could not determine new_dir
+ *   new_dir:            final name of directory being renamed
+ *   possible_new_dirs:  temporary used to help determine new_dir; see comments
+ *                       in get_directory_renames() for details
+ */
+struct dir_rename_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *dir;
+	unsigned non_unique_new_dir:1;
+	struct strbuf new_dir;
+	struct string_list possible_new_dirs;
+};
+
 static struct dir_rename_entry *dir_rename_find_entry(struct hashmap *hashmap,
 						      char *dir)
 {
@@ -92,6 +110,13 @@ static void dir_rename_entry_init(struct dir_rename_entry *entry,
 	string_list_init(&entry->possible_new_dirs, 0);
 }
 
+struct collision_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *target_file;
+	struct string_list source_files;
+	unsigned reported_already:1;
+};
+
 static struct collision_entry *collision_find_entry(struct hashmap *hashmap,
 						    char *target_file)
 {
@@ -358,6 +383,12 @@ static int add_cacheinfo(struct merge_options *opt,
 	return ret;
 }
 
+static inline int merge_detect_rename(struct merge_options *opt)
+{
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+}
+
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 {
 	parse_tree(tree);
diff --git a/merge-recursive.h b/merge-recursive.h
index c201de0fc3..ebec855a65 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -39,47 +39,50 @@ struct merge_options {
 	struct repository *repo;
 };
 
+void init_merge_options(struct merge_options *opt, struct repository *repo);
+
+/* parse the option in s and update the relevant field of opt */
+int parse_merge_opt(struct merge_options *opt, const char *s);
+
 /*
- * For dir_rename_entry, directory names are stored as a full path from the
- * toplevel of the repository and do not include a trailing '/'.  Also:
- *
- *   dir:                original name of directory being renamed
- *   non_unique_new_dir: if true, could not determine new_dir
- *   new_dir:            final name of directory being renamed
- *   possible_new_dirs:  temporary used to help determine new_dir; see comments
- *                       in get_directory_renames() for details
+ * RETURN VALUES: All the merge_* functions below return a value as follows:
+ *   > 0     Merge was clean
+ *   = 0     Merge had conflicts
+ *   < 0     Merge hit an unexpected and unrecoverable problem (e.g. disk
+ *             full) and aborted merge part-way through.
  */
-struct dir_rename_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *dir;
-	unsigned non_unique_new_dir:1;
-	struct strbuf new_dir;
-	struct string_list possible_new_dirs;
-};
-
-struct collision_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *target_file;
-	struct string_list source_files;
-	unsigned reported_already:1;
-};
 
-static inline int merge_detect_rename(struct merge_options *opt)
-{
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
-}
+/*
+ * rename-detecting three-way merge, no recursion.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - No commit is created
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is not updated
+ *   - The working tree is updated with results of the merge
+ */
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *merge_base);
 
 /*
  * merge_recursive is like merge_trees() but with recursive ancestor
- * consolidation, and when successful, it creates an actual commit
- * and writes its address to *result.
+ * consolidation and, if the commit is clean, creation of a commit.
  *
  * NOTE: empirically, about a decade ago it was determined that with more
  *       than two merge bases, optimal behavior was found when the
  *       merge_bases were passed in the order of oldest commit to newest
  *       commit.  Also, merge_bases will be consumed (emptied) so make a
  *       copy if you need it.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - If merge is clean, a commit is created and its address written to *result
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is not updated
+ *   - The working tree is updated with results of the merge
  */
 int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
@@ -88,17 +91,16 @@ int merge_recursive(struct merge_options *opt,
 		    struct commit **result);
 
 /*
- * rename-detecting three-way merge, no recursion; result of merge is written
- * to opt->repo->index.
- */
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *merge_base);
-
-/*
- * "git-merge-recursive" can be fed trees; wrap them into
- * virtual commits and call merge_recursive() proper.
+ * merge_recursive_generic can operate on trees instead of commits, by
+ * wrapping the trees into virtual commits, and calling merge_recursive().
+ * It also writes out the in-memory index to disk if the merge is successful.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - If merge is clean, a commit is created and its address written to *result
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is updated
+ *   - The working tree is updated with results of the merge
  */
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
@@ -107,9 +109,4 @@ int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *opt,
-			struct repository *repo);
-
-int parse_merge_opt(struct merge_options *opt, const char *s);
-
 #endif
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 13/20] merge-recursive: consolidate unnecessary fields in merge_options

[Elijah Newren]

We provided users with the ability to state whether they wanted rename
detection, and to put a limit on how much CPU would be spent.  Both of
these fields had multiple configuration parameters for setting them,
with one being a fallback and the other being an override.  However,
instead of implementing the logic for how to combine the multiple
source locations into the appropriate setting at config loading time,
we loaded and tracked both values and then made the code combine them
every time it wanted to check the overall value.  This had a few
minor drawbacks:
  * it seems more complicated than necessary
  * it runs the risk of people using the independent settings in the
    future and breaking the intent of how the options are used
    together
  * it makes merge_options more complicated than necessary for other
    potential users of the API

Fix these problems by moving the logic for combining the pairs of
options into a single value; make it apply at time-of-config-loading
instead of each-time-of-use.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 27 +++++++++++----------------
 merge-recursive.h |  6 ++----
 2 files changed, 13 insertions(+), 20 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 6820578258..bafac63c45 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -385,8 +385,7 @@ static int add_cacheinfo(struct merge_options *opt,
 
 static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+	return (opt->detect_renames != -1) ? opt->detect_renames : 1;
 }
 
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
@@ -1885,9 +1884,7 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	 */
 	if (opts.detect_rename > DIFF_DETECT_RENAME)
 		opts.detect_rename = DIFF_DETECT_RENAME;
-	opts.rename_limit = opt->merge_rename_limit >= 0 ? opt->merge_rename_limit :
-			    opt->diff_rename_limit >= 0 ? opt->diff_rename_limit :
-			    1000;
+	opts.rename_limit = (opt->rename_limit != -1) ? opt->rename_limit : 1000;
 	opts.rename_score = opt->rename_score;
 	opts.show_rename_progress = opt->show_rename_progress;
 	opts.output_format = DIFF_FORMAT_NO_OUTPUT;
@@ -3719,14 +3716,14 @@ static void merge_recursive_config(struct merge_options *opt)
 {
 	char *value = NULL;
 	git_config_get_int("merge.verbosity", &opt->verbosity);
-	git_config_get_int("diff.renamelimit", &opt->diff_rename_limit);
-	git_config_get_int("merge.renamelimit", &opt->merge_rename_limit);
+	git_config_get_int("diff.renamelimit", &opt->rename_limit);
+	git_config_get_int("merge.renamelimit", &opt->rename_limit);
 	if (!git_config_get_string("diff.renames", &value)) {
-		opt->diff_detect_rename = git_config_rename("diff.renames", value);
+		opt->detect_renames = git_config_rename("diff.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.renames", &value)) {
-		opt->merge_detect_rename = git_config_rename("merge.renames", value);
+		opt->detect_renames = git_config_rename("merge.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.directoryrenames", &value)) {
@@ -3749,11 +3746,9 @@ void init_merge_options(struct merge_options *opt,
 	opt->repo = repo;
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->diff_rename_limit = -1;
-	opt->merge_rename_limit = -1;
+	opt->rename_limit = -1;
 	opt->renormalize = 0;
-	opt->diff_detect_rename = -1;
-	opt->merge_detect_rename = -1;
+	opt->detect_renames = -1;
 	opt->detect_directory_renames = 1;
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
@@ -3805,16 +3800,16 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	else if (!strcmp(s, "no-renormalize"))
 		opt->renormalize = 0;
 	else if (!strcmp(s, "no-renames"))
-		opt->merge_detect_rename = 0;
+		opt->detect_renames = 0;
 	else if (!strcmp(s, "find-renames")) {
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 		opt->rename_score = 0;
 	}
 	else if (skip_prefix(s, "find-renames=", &arg) ||
 		 skip_prefix(s, "rename-threshold=", &arg)) {
 		if ((opt->rename_score = parse_rename_score(&arg)) == -1 || *arg != 0)
 			return -1;
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 	}
 	/*
 	 * Please update $__git_merge_strategy_options in
diff --git a/merge-recursive.h b/merge-recursive.h
index ebec855a65..b92a9ebce7 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -23,10 +23,8 @@ struct merge_options {
 	long xdl_opts;
 	int verbosity;
 	int detect_directory_renames;
-	int diff_detect_rename;
-	int merge_detect_rename;
-	int diff_rename_limit;
-	int merge_rename_limit;
+	int detect_renames;
+	int rename_limit;
 	int rename_score;
 	int needed_rename_limit;
 	int show_rename_progress;
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 14/20] merge-recursive: comment and reorder the merge_options fields

[Elijah Newren]

The merge_options struct had lots of fields, making it a little
imposing, but the options naturally fall into multiple different groups.
Grouping similar options and adding a comment or two makes it easier to
read, easier for new folks to figure out which options are related, and
thus easier for them to find the options they need.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 16 +++++++++++-----
 merge-recursive.h | 34 +++++++++++++++++++++++-----------
 2 files changed, 34 insertions(+), 16 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index bafac63c45..a8e4a6a531 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3743,21 +3743,27 @@ void init_merge_options(struct merge_options *opt,
 {
 	const char *merge_verbosity;
 	memset(opt, 0, sizeof(struct merge_options));
+
 	opt->repo = repo;
+
+	opt->detect_renames = -1;
+	opt->detect_directory_renames = 1;
+	opt->rename_limit = -1;
+
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->rename_limit = -1;
+	strbuf_init(&opt->obuf, 0);
+
 	opt->renormalize = 0;
-	opt->detect_renames = -1;
-	opt->detect_directory_renames = 1;
+
+	string_list_init(&opt->df_conflict_file_set, 1);
+
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
 		opt->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (opt->verbosity >= 5)
 		opt->buffer_output = 0;
-	strbuf_init(&opt->obuf, 0);
-	string_list_init(&opt->df_conflict_file_set, 1);
 }
 
 int parse_merge_opt(struct merge_options *opt, const char *s)
diff --git a/merge-recursive.h b/merge-recursive.h
index b92a9ebce7..b6d420960b 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -9,32 +9,44 @@ struct commit;
 struct repository;
 
 struct merge_options {
+	struct repository *repo;
+
+	/* ref names used in console messages and conflict markers */
 	const char *ancestor;
 	const char *branch1;
 	const char *branch2;
+
+	/* rename related options */
+	int detect_renames;
+	int detect_directory_renames;
+	int rename_limit;
+	int rename_score;
+	int show_rename_progress;
+
+	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
+	long xdl_opts;
 	enum {
 		MERGE_RECURSIVE_NORMAL = 0,
 		MERGE_RECURSIVE_OURS,
 		MERGE_RECURSIVE_THEIRS
 	} recursive_variant;
-	const char *subtree_shift;
+
+	/* console output related options */
+	int verbosity;
 	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
+	struct strbuf obuf;     /* output buffer */
+
+	/* miscellaneous control options */
+	const char *subtree_shift;
 	unsigned renormalize : 1;
-	long xdl_opts;
-	int verbosity;
-	int detect_directory_renames;
-	int detect_renames;
-	int rename_limit;
-	int rename_score;
-	int needed_rename_limit;
-	int show_rename_progress;
+
+	/* internal fields used by the implementation (do NOT set these) */
 	int call_depth;
-	struct strbuf obuf;
+	int needed_rename_limit;
 	struct hashmap current_file_dir_set;
 	struct string_list df_conflict_file_set;
 	struct unpack_trees_options unpack_opts;
 	struct index_state orig_index;
-	struct repository *repo;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 15/20] merge-recursive: avoid losing output and leaking memory holding that output

[Elijah Newren]

If opt->buffer_output is less than 2, then merge_trees(),
merge_recursive(), and merge_recursive_generic() are all supposed to
flush the opt->obuf output buffer to stdout and release any memory it
holds.  merge_trees() did not do this.  Move the logic that handles this
for merge_recursive_internal() to merge_finalize() so that all three
methods handle this requirement.

Note that this bug didn't cause any problems currently, because there
are only two callers of merge_trees() right now (a git grep for
'merge_trees(' is misleading because builtin/merge-tree.c also defines a
'merge_tree' function that is unrelated), and only one of those is
called with buffer_output less than 2 (builtin/checkout.c), but it set
opt->verbosity to 0, for which there is only currently one non-error
message that would be shown: "Already up to date!".  However, that one
message can only occur when the merge is utterly trivial (the merge base
tree exactly matches the merge tree), and builtin/checkout.c already
attempts a trivial merge via unpack_trees() before falling back to
merge_trees().

Also, if opt->buffer_output is 2, then the caller is responsible to
handle showing any output in opt->obuf and for free'ing it.  This
requirement might be easy to overlook, so add a comment to
merge-recursive.h pointing it out.  (There are currently two callers
that set buffer_output to 2, both in sequencer.c, and both of which
handle this correctly.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 6 +++---
 merge-recursive.h | 3 ++-
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index a8e4a6a531..49c7de0379 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3592,9 +3592,6 @@ static int merge_recursive_internal(struct merge_options *opt,
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-	flush_output(opt);
-	if (!opt->call_depth && opt->buffer_output < 2)
-		strbuf_release(&opt->obuf);
 	return clean;
 }
 
@@ -3616,6 +3613,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 
 static void merge_finalize(struct merge_options *opt)
 {
+	flush_output(opt);
+	if (!opt->call_depth && opt->buffer_output < 2)
+		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       opt->needed_rename_limit, 0);
diff --git a/merge-recursive.h b/merge-recursive.h
index b6d420960b..12249258ef 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -34,7 +34,8 @@ struct merge_options {
 	/* console output related options */
 	int verbosity;
 	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
-	struct strbuf obuf;     /* output buffer */
+	struct strbuf obuf;     /* output buffer; if buffer_output == 2, caller
+				 * must handle and call strbuf_release */
 
 	/* miscellaneous control options */
 	const char *subtree_shift;
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 16/20] merge-recursive: split internal fields into a separate struct

[Elijah Newren]

merge_options has several internal fields that should not be set or read
by external callers.  This just complicates the API.  Move them into an
opaque merge_options_internal struct that is defined only in
merge-recursive.c and keep these out of merge-recursive.h.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 184 ++++++++++++++++++++++++----------------------
 merge-recursive.h |  17 ++---
 2 files changed, 105 insertions(+), 96 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 49c7de0379..ba635ed753 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -29,6 +29,15 @@
 #include "revision.h"
 #include "commit-reach.h"
 
+struct merge_options_internal {
+	int call_depth;
+	int needed_rename_limit;
+	struct hashmap current_file_dir_set;
+	struct string_list df_conflict_file_set;
+	struct unpack_trees_options unpack_opts;
+	struct index_state orig_index;
+};
+
 struct path_hashmap_entry {
 	struct hashmap_entry e;
 	char path[FLEX_ARRAY];
@@ -309,7 +318,8 @@ static inline void setup_rename_conflict_info(enum rename_type rename_type,
 
 static int show(struct merge_options *opt, int v)
 {
-	return (!opt->call_depth && opt->verbosity >= v) || opt->verbosity >= 5;
+	return (!opt->priv->call_depth && opt->verbosity >= v) ||
+		opt->verbosity >= 5;
 }
 
 __attribute__((format (printf, 3, 4)))
@@ -320,7 +330,7 @@ static void output(struct merge_options *opt, int v, const char *fmt, ...)
 	if (!show(opt, v))
 		return;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 
 	va_start(ap, fmt);
 	strbuf_vaddf(&opt->obuf, fmt, ap);
@@ -335,7 +345,7 @@ static void output_commit_title(struct merge_options *opt, struct commit *commit
 {
 	struct merge_remote_desc *desc;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 	desc = merge_remote_util(commit);
 	if (desc)
 		strbuf_addf(&opt->obuf, "virtual %s\n", desc->name);
@@ -403,43 +413,43 @@ static int unpack_trees_start(struct merge_options *opt,
 	struct tree_desc t[3];
 	struct index_state tmp_index = { NULL };
 
-	memset(&opt->unpack_opts, 0, sizeof(opt->unpack_opts));
-	if (opt->call_depth)
-		opt->unpack_opts.index_only = 1;
+	memset(&opt->priv->unpack_opts, 0, sizeof(opt->priv->unpack_opts));
+	if (opt->priv->call_depth)
+		opt->priv->unpack_opts.index_only = 1;
 	else
-		opt->unpack_opts.update = 1;
-	opt->unpack_opts.merge = 1;
-	opt->unpack_opts.head_idx = 2;
-	opt->unpack_opts.fn = threeway_merge;
-	opt->unpack_opts.src_index = opt->repo->index;
-	opt->unpack_opts.dst_index = &tmp_index;
-	opt->unpack_opts.aggressive = !merge_detect_rename(opt);
-	setup_unpack_trees_porcelain(&opt->unpack_opts, "merge");
+		opt->priv->unpack_opts.update = 1;
+	opt->priv->unpack_opts.merge = 1;
+	opt->priv->unpack_opts.head_idx = 2;
+	opt->priv->unpack_opts.fn = threeway_merge;
+	opt->priv->unpack_opts.src_index = opt->repo->index;
+	opt->priv->unpack_opts.dst_index = &tmp_index;
+	opt->priv->unpack_opts.aggressive = !merge_detect_rename(opt);
+	setup_unpack_trees_porcelain(&opt->priv->unpack_opts, "merge");
 
 	init_tree_desc_from_tree(t+0, common);
 	init_tree_desc_from_tree(t+1, head);
 	init_tree_desc_from_tree(t+2, merge);
 
-	rc = unpack_trees(3, t, &opt->unpack_opts);
+	rc = unpack_trees(3, t, &opt->priv->unpack_opts);
 	cache_tree_free(&opt->repo->index->cache_tree);
 
 	/*
-	 * Update opt->repo->index to match the new results, AFTER saving a copy
-	 * in opt->orig_index.  Update src_index to point to the saved copy.
-	 * (verify_uptodate() checks src_index, and the original index is
-	 * the one that had the necessary modification timestamps.)
+	 * Update opt->repo->index to match the new results, AFTER saving a
+	 * copy in opt->priv->orig_index.  Update src_index to point to the
+	 * saved copy.  (verify_uptodate() checks src_index, and the original
+	 * index is the one that had the necessary modification timestamps.)
 	 */
-	opt->orig_index = *opt->repo->index;
+	opt->priv->orig_index = *opt->repo->index;
 	*opt->repo->index = tmp_index;
-	opt->unpack_opts.src_index = &opt->orig_index;
+	opt->priv->unpack_opts.src_index = &opt->priv->orig_index;
 
 	return rc;
 }
 
 static void unpack_trees_finish(struct merge_options *opt)
 {
-	discard_index(&opt->orig_index);
-	clear_unpack_trees_porcelain(&opt->unpack_opts);
+	discard_index(&opt->priv->orig_index);
+	clear_unpack_trees_porcelain(&opt->priv->unpack_opts);
 }
 
 static int save_files_dirs(const struct object_id *oid,
@@ -454,7 +464,7 @@ static int save_files_dirs(const struct object_id *oid,
 
 	FLEX_ALLOC_MEM(entry, path, base->buf, base->len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 
 	strbuf_setlen(base, baselen);
 	return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);
@@ -585,7 +595,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	 * If we're merging merge-bases, we don't want to bother with
 	 * any working directory changes.
 	 */
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return;
 
 	/* Ensure D/F conflicts are adjacent in the entries list. */
@@ -597,7 +607,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	df_sorted_entries.cmp = string_list_df_name_compare;
 	string_list_sort(&df_sorted_entries);
 
-	string_list_clear(&opt->df_conflict_file_set, 1);
+	string_list_clear(&opt->priv->df_conflict_file_set, 1);
 	for (i = 0; i < df_sorted_entries.nr; i++) {
 		const char *path = df_sorted_entries.items[i].string;
 		int len = strlen(path);
@@ -613,7 +623,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 		    len > last_len &&
 		    memcmp(path, last_file, last_len) == 0 &&
 		    path[last_len] == '/') {
-			string_list_insert(&opt->df_conflict_file_set, last_file);
+			string_list_insert(&opt->priv->df_conflict_file_set, last_file);
 		}
 
 		/*
@@ -680,8 +690,8 @@ static void update_entry(struct stage_data *entry,
 static int remove_file(struct merge_options *opt, int clean,
 		       const char *path, int no_wd)
 {
-	int update_cache = opt->call_depth || clean;
-	int update_working_directory = !opt->call_depth && !no_wd;
+	int update_cache = opt->priv->call_depth || clean;
+	int update_working_directory = !opt->priv->call_depth && !no_wd;
 
 	if (update_cache) {
 		if (remove_file_from_index(opt->repo->index, path))
@@ -724,16 +734,16 @@ static char *unique_path(struct merge_options *opt,
 	add_flattened_path(&newpath, branch);
 
 	base_len = newpath.len;
-	while (hashmap_get_from_hash(&opt->current_file_dir_set,
+	while (hashmap_get_from_hash(&opt->priv->current_file_dir_set,
 				     path_hash(newpath.buf), newpath.buf) ||
-	       (!opt->call_depth && file_exists(newpath.buf))) {
+	       (!opt->priv->call_depth && file_exists(newpath.buf))) {
 		strbuf_setlen(&newpath, base_len);
 		strbuf_addf(&newpath, "_%d", suffix++);
 	}
 
 	FLEX_ALLOC_MEM(entry, path, newpath.buf, newpath.len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 	return strbuf_detach(&newpath, NULL);
 }
 
@@ -775,7 +785,7 @@ static int dir_in_way(struct index_state *istate, const char *path,
 static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 				   const struct diff_filespec *blob)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 	struct cache_entry *ce;
 
 	if (0 > pos)
@@ -783,7 +793,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 		return 0;
 
 	/* See if the file we were tracking before matches */
-	ce = opt->orig_index.cache[pos];
+	ce = opt->priv->orig_index.cache[pos];
 	return (oid_eq(&ce->oid, &blob->oid) && ce->ce_mode == blob->mode);
 }
 
@@ -792,7 +802,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
  */
 static int was_tracked(struct merge_options *opt, const char *path)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 
 	if (0 <= pos)
 		/* we were tracking this path before the merge */
@@ -849,12 +859,12 @@ static int was_dirty(struct merge_options *opt, const char *path)
 	struct cache_entry *ce;
 	int dirty = 1;
 
-	if (opt->call_depth || !was_tracked(opt, path))
+	if (opt->priv->call_depth || !was_tracked(opt, path))
 		return !dirty;
 
-	ce = index_file_exists(opt->unpack_opts.src_index,
+	ce = index_file_exists(opt->priv->unpack_opts.src_index,
 			       path, strlen(path), ignore_case);
-	dirty = verify_uptodate(ce, &opt->unpack_opts) != 0;
+	dirty = verify_uptodate(ce, &opt->priv->unpack_opts) != 0;
 	return dirty;
 }
 
@@ -864,8 +874,8 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 	const char *msg = _("failed to create path '%s'%s");
 
 	/* Unlink any D/F conflict files that are in the way */
-	for (i = 0; i < opt->df_conflict_file_set.nr; i++) {
-		const char *df_path = opt->df_conflict_file_set.items[i].string;
+	for (i = 0; i < opt->priv->df_conflict_file_set.nr; i++) {
+		const char *df_path = opt->priv->df_conflict_file_set.items[i].string;
 		size_t pathlen = strlen(path);
 		size_t df_pathlen = strlen(df_path);
 		if (df_pathlen < pathlen &&
@@ -875,7 +885,7 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 			       _("Removing %s to make room for subdirectory\n"),
 			       df_path);
 			unlink(df_path);
-			unsorted_string_list_delete_item(&opt->df_conflict_file_set,
+			unsorted_string_list_delete_item(&opt->priv->df_conflict_file_set,
 							 i, 0);
 			break;
 		}
@@ -916,7 +926,7 @@ static int update_file_flags(struct merge_options *opt,
 {
 	int ret = 0;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		update_wd = 0;
 
 	if (update_wd) {
@@ -1001,7 +1011,7 @@ static int update_file(struct merge_options *opt,
 		       const char *path)
 {
 	return update_file_flags(opt, contents, path,
-				 opt->call_depth || clean, !opt->call_depth);
+				 opt->priv->call_depth || clean, !opt->priv->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1030,7 +1040,7 @@ static int merge_3way(struct merge_options *opt,
 	ll_opts.extra_marker_size = extra_marker_size;
 	ll_opts.xdl_opts = opt->xdl_opts;
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		ll_opts.virtual_ancestor = 1;
 		ll_opts.variant = 0;
 	} else {
@@ -1164,7 +1174,7 @@ static int merge_submodule(struct merge_options *opt,
 	struct object_array merges;
 
 	int i;
-	int search = !opt->call_depth;
+	int search = !opt->priv->call_depth;
 
 	/* store a in result in case we fail */
 	oidcpy(result, a);
@@ -1385,7 +1395,7 @@ static int handle_rename_via_dir(struct merge_options *opt,
 	int mark_conflicted = (opt->detect_directory_renames == 1);
 	assert(ren->dir_rename_original_dest);
 
-	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
+	if (!opt->priv->call_depth && would_lose_untracked(opt, dest->path)) {
 		mark_conflicted = 1;
 		file_path = unique_path(opt, dest->path, ren->branch);
 		output(opt, 1, _("Error: Refusing to lose untracked file at %s; "
@@ -1428,12 +1438,12 @@ static int handle_change_delete(struct merge_options *opt,
 	const char *update_path = path;
 	int ret = 0;
 
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0) ||
-	    (!opt->call_depth && would_lose_untracked(opt, path))) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0) ||
+	    (!opt->priv->call_depth && would_lose_untracked(opt, path))) {
 		update_path = alt_path = unique_path(opt, path, change_branch);
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * We cannot arbitrarily accept either a_sha or b_sha as
 		 * correct; since there is no true "middle point" between
@@ -1508,14 +1518,14 @@ static int handle_rename_delete(struct merge_options *opt,
 				     opt->branch2 : opt->branch1);
 
 	if (handle_change_delete(opt,
-				 opt->call_depth ? orig->path : dest->path,
-				 opt->call_depth ? NULL : orig->path,
+				 opt->priv->call_depth ? orig->path : dest->path,
+				 opt->priv->call_depth ? NULL : orig->path,
 				 orig, dest,
 				 rename_branch, delete_branch,
 				 _("rename"), _("renamed")))
 		return -1;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return remove_file_from_index(opt->repo->index, dest->path);
 	else
 		return update_stages(opt, dest->path, NULL,
@@ -1552,7 +1562,7 @@ static int handle_file_collision(struct merge_options *opt,
 	/*
 	 * In the recursive case, we just opt to undo renames
 	 */
-	if (opt->call_depth && (prev_path1 || prev_path2)) {
+	if (opt->priv->call_depth && (prev_path1 || prev_path2)) {
 		/* Put first file (a->oid, a->mode) in its original spot */
 		if (prev_path1) {
 			if (update_file(opt, 1, a, prev_path1))
@@ -1581,10 +1591,10 @@ static int handle_file_collision(struct merge_options *opt,
 	/* Remove rename sources if rename/add or rename/rename(2to1) */
 	if (prev_path1)
 		remove_file(opt, 1, prev_path1,
-			    opt->call_depth || would_lose_untracked(opt, prev_path1));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path1));
 	if (prev_path2)
 		remove_file(opt, 1, prev_path2,
-			    opt->call_depth || would_lose_untracked(opt, prev_path2));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path2));
 
 	/*
 	 * Remove the collision path, if it wouldn't cause dirty contents
@@ -1626,12 +1636,12 @@ static int handle_file_collision(struct merge_options *opt,
 	null.mode = 0;
 
 	if (merge_mode_and_contents(opt, &null, a, b, collide_path,
-				    branch1, branch2, opt->call_depth * 2, &mfi))
+				    branch1, branch2, opt->priv->call_depth * 2, &mfi))
 		return -1;
 	mfi.clean &= !alt_path;
 	if (update_file(opt, mfi.clean, &mfi.blob, update_path))
 		return -1;
-	if (!mfi.clean && !opt->call_depth &&
+	if (!mfi.clean && !opt->priv->call_depth &&
 	    update_stages(opt, collide_path, NULL, a, b))
 		return -1;
 	free(alt_path);
@@ -1671,7 +1681,7 @@ static int handle_rename_add(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[other_stage],
 				    prev_path_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi))
+				    1 + opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(prev_path_desc);
 
@@ -1689,7 +1699,7 @@ static char *find_path_for_conflict(struct merge_options *opt,
 				    const char *branch2)
 {
 	char *new_path = NULL;
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0)) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0)) {
 		new_path = unique_path(opt, path, branch1);
 		output(opt, 1, _("%s is a directory in %s adding "
 			       "as %s instead"),
@@ -1720,17 +1730,17 @@ static int handle_rename_rename_1to2(struct merge_options *opt,
 	       "rename \"%s\"->\"%s\" in \"%s\"%s"),
 	       o->path, a->path, ci->ren1->branch,
 	       o->path, b->path, ci->ren2->branch,
-	       opt->call_depth ? _(" (left unresolved)") : "");
+	       opt->priv->call_depth ? _(" (left unresolved)") : "");
 
 	path_desc = xstrfmt("%s and %s, both renamed from %s",
 			    a->path, b->path, o->path);
 	if (merge_mode_and_contents(opt, o, a, b, path_desc,
 				    ci->ren1->branch, ci->ren2->branch,
-				    opt->call_depth * 2, &mfi))
+				    opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(path_desc);
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1845,12 +1855,12 @@ static int handle_rename_rename_2to1(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[ostage1],
 				    path_side_1_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c1) ||
+				    1 + opt->priv->call_depth * 2, &mfi_c1) ||
 	    merge_mode_and_contents(opt, b,
 				    &ci->ren2->src_entry->stages[ostage2],
 				    c2, path_side_2_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c2))
+				    1 + opt->priv->call_depth * 2, &mfi_c2))
 		return -1;
 	free(path_side_1_desc);
 	free(path_side_2_desc);
@@ -1891,8 +1901,8 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	diff_setup_done(&opts);
 	diff_tree_oid(&o_tree->object.oid, &tree->object.oid, "", &opts);
 	diffcore_std(&opts);
-	if (opts.needed_rename_limit > opt->needed_rename_limit)
-		opt->needed_rename_limit = opts.needed_rename_limit;
+	if (opts.needed_rename_limit > opt->priv->needed_rename_limit)
+		opt->priv->needed_rename_limit = opts.needed_rename_limit;
 
 	ret = xmalloc(sizeof(*ret));
 	*ret = diff_queued_diff;
@@ -3022,13 +3032,13 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		reason = _("add/add");
 
 	assert(o->path && a->path && b->path);
-	if (ci && dir_in_way(opt->repo->index, path, !opt->call_depth,
+	if (ci && dir_in_way(opt->repo->index, path, !opt->priv->call_depth,
 			     S_ISGITLINK(ci->ren1->pair->two->mode)))
 		df_conflict_remains = 1;
 
 	if (merge_mode_and_contents(opt, o, a, b, path,
 				    opt->branch1, opt->branch2,
-				    opt->call_depth * 2, mfi))
+				    opt->priv->call_depth * 2, mfi))
 		return -1;
 
 	/*
@@ -3044,7 +3054,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 		output(opt, 3, _("Skipped %s (merged same as existing)"), path);
 		if (add_cacheinfo(opt, &mfi->blob, path,
-				  0, (!opt->call_depth && !is_dirty), 0))
+				  0, (!opt->priv->call_depth && !is_dirty), 0))
 			return -1;
 		/*
 		 * However, add_cacheinfo() will delete the old cache entry
@@ -3052,8 +3062,8 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		 * flag to avoid making the file appear as if it were
 		 * deleted by the user.
 		 */
-		pos = index_name_pos(&opt->orig_index, path, strlen(path));
-		ce = opt->orig_index.cache[pos];
+		pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
+		ce = opt->priv->orig_index.cache[pos];
 		if (ce_skip_worktree(ce)) {
 			pos = index_name_pos(opt->repo->index, path, strlen(path));
 			ce = opt->repo->index->cache[pos];
@@ -3074,7 +3084,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 	if (df_conflict_remains || is_dirty) {
 		char *new_path;
-		if (opt->call_depth) {
+		if (opt->priv->call_depth) {
 			remove_file_from_index(opt->repo->index, path);
 		} else {
 			if (!mfi->clean) {
@@ -3332,7 +3342,7 @@ static int process_entry(struct merge_options *opt,
 			conf = _("directory/file");
 		}
 		if (dir_in_way(opt->repo->index, path,
-			       !opt->call_depth && !S_ISGITLINK(a->mode),
+			       !opt->priv->call_depth && !S_ISGITLINK(a->mode),
 			       0)) {
 			char *new_path = unique_path(opt, path, add_branch);
 			clean_merge = 0;
@@ -3341,7 +3351,7 @@ static int process_entry(struct merge_options *opt,
 			       conf, path, other_branch, path, new_path);
 			if (update_file(opt, 0, contents, new_path))
 				clean_merge = -1;
-			else if (opt->call_depth)
+			else if (opt->priv->call_depth)
 				remove_file_from_index(opt->repo->index, path);
 			free(new_path);
 		} else {
@@ -3406,7 +3416,7 @@ static int merge_trees_internal(struct merge_options *opt,
 	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
-		if (show(opt, 4) || opt->call_depth)
+		if (show(opt, 4) || opt->priv->call_depth)
 			err(opt, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
@@ -3425,7 +3435,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+		hashmap_init(&opt->priv->current_file_dir_set, path_hashmap_cmp,
 			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
@@ -3462,7 +3472,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		string_list_clear(entries, 1);
 		free(entries);
 
-		hashmap_free(&opt->current_file_dir_set, 1);
+		hashmap_free(&opt->priv->current_file_dir_set, 1);
 
 		if (clean < 0) {
 			unpack_trees_finish(opt);
@@ -3474,7 +3484,7 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		struct object_id tree_id;
 		if (write_index_as_tree(&tree_id, opt->repo->index, NULL,
 					WRITE_TREE_FROM_MEMORY, NULL) ||
@@ -3545,7 +3555,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 
 	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
-		opt->call_depth++;
+		opt->priv->call_depth++;
 		/*
 		 * When the merge fails, the result contains files
 		 * with conflict markers. The cleanness flag is
@@ -3564,14 +3574,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
-		opt->call_depth--;
+		opt->priv->call_depth--;
 
 		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
 	discard_index(opt->repo->index);
-	if (!opt->call_depth)
+	if (!opt->priv->call_depth)
 		repo_read_index(opt->repo);
 
 	opt->ancestor = "merged common ancestors";
@@ -3586,7 +3596,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 		return clean;
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		*result = make_virtual_commit(opt->repo, result_tree,
 					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
@@ -3608,17 +3618,21 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 		return -1;
 	}
 
+	opt->priv = xcalloc(1, sizeof(*opt->priv));
+	string_list_init(&opt->priv->df_conflict_file_set, 1);
 	return 0;
 }
 
 static void merge_finalize(struct merge_options *opt)
 {
 	flush_output(opt);
-	if (!opt->call_depth && opt->buffer_output < 2)
+	if (!opt->priv->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0);
+	free(opt->priv);
+	opt->priv = NULL;
 }
 
 int merge_trees(struct merge_options *opt,
@@ -3756,8 +3770,6 @@ void init_merge_options(struct merge_options *opt,
 
 	opt->renormalize = 0;
 
-	string_list_init(&opt->df_conflict_file_set, 1);
-
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
diff --git a/merge-recursive.h b/merge-recursive.h
index 12249258ef..0d5e928832 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -1,13 +1,15 @@
 #ifndef MERGE_RECURSIVE_H
 #define MERGE_RECURSIVE_H
 
-#include "string-list.h"
-#include "unpack-trees.h"
+#include "strbuf.h"
 
 struct commit;
-
+struct commit_list;
+struct object_id;
 struct repository;
+struct tree;
 
+struct merge_options_internal;
 struct merge_options {
 	struct repository *repo;
 
@@ -41,13 +43,8 @@ struct merge_options {
 	const char *subtree_shift;
 	unsigned renormalize : 1;
 
-	/* internal fields used by the implementation (do NOT set these) */
-	int call_depth;
-	int needed_rename_limit;
-	struct hashmap current_file_dir_set;
-	struct string_list df_conflict_file_set;
-	struct unpack_trees_options unpack_opts;
-	struct index_state orig_index;
+	/* internal fields used by the implementation */
+	struct merge_options_internal *priv;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 17/20] merge-recursive: alphabetize include list

[Elijah Newren]

Other than cache.h which needs to appear first, and merge-recursive.h
which I want to be second so that we are more likely to notice if
merge-recursive.h has any missing includes, the rest of the list is
long and easier to look through if it's alphabetical.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index ba635ed753..27e0bdf48c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -4,30 +4,31 @@
  * The thieves were Alex Riesen and Johannes Schindelin, in June/July 2006
  */
 #include "cache.h"
-#include "config.h"
+#include "merge-recursive.h"
+
 #include "advice.h"
-#include "lockfile.h"
-#include "cache-tree.h"
-#include "object-store.h"
-#include "repository.h"
-#include "commit.h"
+#include "alloc.h"
+#include "attr.h"
 #include "blob.h"
 #include "builtin.h"
-#include "tree-walk.h"
+#include "cache-tree.h"
+#include "commit.h"
+#include "commit-reach.h"
+#include "config.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "dir.h"
+#include "ll-merge.h"
+#include "lockfile.h"
+#include "object-store.h"
+#include "repository.h"
+#include "revision.h"
+#include "string-list.h"
+#include "submodule.h"
 #include "tag.h"
-#include "alloc.h"
+#include "tree-walk.h"
 #include "unpack-trees.h"
-#include "string-list.h"
 #include "xdiff-interface.h"
-#include "ll-merge.h"
-#include "attr.h"
-#include "merge-recursive.h"
-#include "dir.h"
-#include "submodule.h"
-#include "revision.h"
-#include "commit-reach.h"
 
 struct merge_options_internal {
 	int call_depth;
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 18/20] merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*

[Elijah Newren]

I want to implement the same outward facing API as found within
merge-recursive.h in a different merge strategy.  However, that makes
names like MERGE_RECURSIVE_{NORMAL,OURS,THEIRS} look a little funny;
rename to MERGE_VARIANT_{NORMAL,OURS,THEIRS}.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 14 +++++++-------
 merge-recursive.h |  6 +++---
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 27e0bdf48c..49eb50c17b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1046,10 +1046,10 @@ static int merge_3way(struct merge_options *opt,
 		ll_opts.variant = 0;
 	} else {
 		switch (opt->recursive_variant) {
-		case MERGE_RECURSIVE_OURS:
+		case MERGE_VARIANT_OURS:
 			ll_opts.variant = XDL_MERGE_FAVOR_OURS;
 			break;
-		case MERGE_RECURSIVE_THEIRS:
+		case MERGE_VARIANT_THEIRS:
 			ll_opts.variant = XDL_MERGE_FAVOR_THEIRS;
 			break;
 		default:
@@ -1359,15 +1359,15 @@ static int merge_mode_and_contents(struct merge_options *opt,
 							&b->oid);
 		} else if (S_ISLNK(a->mode)) {
 			switch (opt->recursive_variant) {
-			case MERGE_RECURSIVE_NORMAL:
+			case MERGE_VARIANT_NORMAL:
 				oidcpy(&result->blob.oid, &a->oid);
 				if (!oid_eq(&a->oid, &b->oid))
 					result->clean = 0;
 				break;
-			case MERGE_RECURSIVE_OURS:
+			case MERGE_VARIANT_OURS:
 				oidcpy(&result->blob.oid, &a->oid);
 				break;
-			case MERGE_RECURSIVE_THEIRS:
+			case MERGE_VARIANT_THEIRS:
 				oidcpy(&result->blob.oid, &b->oid);
 				break;
 			}
@@ -3786,9 +3786,9 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	if (!s || !*s)
 		return -1;
 	if (!strcmp(s, "ours"))
-		opt->recursive_variant = MERGE_RECURSIVE_OURS;
+		opt->recursive_variant = MERGE_VARIANT_OURS;
 	else if (!strcmp(s, "theirs"))
-		opt->recursive_variant = MERGE_RECURSIVE_THEIRS;
+		opt->recursive_variant = MERGE_VARIANT_THEIRS;
 	else if (!strcmp(s, "subtree"))
 		opt->subtree_shift = "";
 	else if (skip_prefix(s, "subtree=", &arg))
diff --git a/merge-recursive.h b/merge-recursive.h
index 0d5e928832..350fb43a01 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -28,9 +28,9 @@ struct merge_options {
 	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
 	long xdl_opts;
 	enum {
-		MERGE_RECURSIVE_NORMAL = 0,
-		MERGE_RECURSIVE_OURS,
-		MERGE_RECURSIVE_THEIRS
+		MERGE_VARIANT_NORMAL = 0,
+		MERGE_VARIANT_OURS,
+		MERGE_VARIANT_THEIRS
 	} recursive_variant;
 
 	/* console output related options */
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 19/20] merge-recursive: be consistent with assert

[Elijah Newren]

In commit 8daec1df03de ("merge-recursive: switch from (oid,mode) pairs
to a diff_filespec", 2019-04-05), an assertion on a->path && b->path
was added for code readability to document that these both needed to be
non-NULL at this point in the code.  However, the subsequent lines also
read o->path, so it should be included in the assert.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 49eb50c17b..f1e2658457 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1058,7 +1058,7 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path);
+	assert(a->path && b->path && o->path);
 	if (strcmp(a->path, b->path) ||
 	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
 		base_name = opt->ancestor == NULL ? NULL :
-- 
2.22.0.550.g71c37a0928.dirty

[PATCH v2 20/20] merge-recursive: provide a better label for diff3 common ancestor

[Elijah Newren]

In commit 7ca56aa07619 ("merge-recursive: add a label for ancestor",
2010-03-20), a label was added for the '||||||' line to make it have
the more informative heading '|||||| merged common ancestors', with
the statement:

    It would be nicer to use a more informative label.  Perhaps someone
    will provide one some day.

This chosen label was perfectly reasonable when recursiveness kicks in,
i.e. when there are multiple merge bases.  (I can't think of a better
label in such cases.)  But it is actually somewhat misleading when there
is a unique merge base or no merge base.  Change this based on the
number of merge bases:
    >=2: "merged common ancestors"
    1:   <abbreviated commit hash>
    0:   "<empty tree>"

Also, the code in merge_3way making use of opt->ancestor was overly
complex because it tried to handle the case of opt->ancestor being NULL.
We always set it first, though, so just add an assert that opt->ancestor
is not NULL and simplify the surrounding code.

Tests have also been added to check that we get the right ancestor name
for each of the three cases.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |  39 ++++--
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 189 ++++++++++++++++++++++++++++++
 3 files changed, 222 insertions(+), 14 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index f1e2658457..6d5dce17fc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1034,7 +1034,7 @@ static int merge_3way(struct merge_options *opt,
 {
 	mmfile_t orig, src1, src2;
 	struct ll_merge_options ll_opts = {0};
-	char *base_name, *name1, *name2;
+	char *base, *name1, *name2;
 	int merge_status;
 
 	ll_opts.renormalize = opt->renormalize;
@@ -1058,16 +1058,13 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path && o->path);
-	if (strcmp(a->path, b->path) ||
-	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s:%s", opt->ancestor, o->path);
+	assert(a->path && b->path && o->path && opt->ancestor);
+	if (strcmp(a->path, b->path) || strcmp(a->path, o->path) != 0) {
+		base  = mkpathdup("%s:%s", opt->ancestor, o->path);
 		name1 = mkpathdup("%s:%s", branch1, a->path);
 		name2 = mkpathdup("%s:%s", branch2, b->path);
 	} else {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s", opt->ancestor);
+		base  = mkpathdup("%s", opt->ancestor);
 		name1 = mkpathdup("%s", branch1);
 		name2 = mkpathdup("%s", branch2);
 	}
@@ -1076,11 +1073,11 @@ static int merge_3way(struct merge_options *opt,
 	read_mmblob(&src1, &a->oid);
 	read_mmblob(&src2, &b->oid);
 
-	merge_status = ll_merge(result_buf, a->path, &orig, base_name,
+	merge_status = ll_merge(result_buf, a->path, &orig, base,
 				&src1, name1, &src2, name2,
 				opt->repo->index, &ll_opts);
 
-	free(base_name);
+	free(base);
 	free(name1);
 	free(name2);
 	free(orig.ptr);
@@ -3523,6 +3520,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 	struct commit *merged_merge_bases;
 	struct tree *result_tree;
 	int clean;
+	int num_merge_bases;
+	struct strbuf merge_base_abbrev = STRBUF_INIT;
 
 	if (show(opt, 4)) {
 		output(opt, 4, _("Merging:"));
@@ -3544,6 +3543,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 			output_commit_title(opt, iter->item);
 	}
 
+	num_merge_bases = commit_list_count(merge_bases);
 	merged_merge_bases = pop_commit(&merge_bases);
 	if (merged_merge_bases == NULL) {
 		/* if there is no common ancestor, use an empty tree */
@@ -3585,13 +3585,26 @@ static int merge_recursive_internal(struct merge_options *opt,
 	if (!opt->priv->call_depth)
 		repo_read_index(opt->repo);
 
-	opt->ancestor = "merged common ancestors";
+	switch (num_merge_bases) {
+	case 0:
+		opt->ancestor = "<empty tree>";
+		break;
+	case 1:
+		strbuf_add_unique_abbrev(&merge_base_abbrev,
+					 &merged_merge_bases->object.oid,
+					 DEFAULT_ABBREV);
+		opt->ancestor = merge_base_abbrev.buf;
+		break;
+	default:
+		opt->ancestor = "merged common ancestors";
+	}
 	clean = merge_trees_internal(opt,
 				     repo_get_commit_tree(opt->repo, h1),
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
 							  merged_merge_bases),
 				     &result_tree);
+	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
@@ -3644,6 +3657,8 @@ int merge_trees(struct merge_options *opt,
 	int clean;
 	struct tree *ignored;
 
+	assert(opt->ancestor != NULL);
+
 	if (merge_start(opt, head))
 		return -1;
 	clean = merge_trees_internal(opt, head, merge, merge_base, &ignored);
@@ -3660,6 +3675,8 @@ int merge_recursive(struct merge_options *opt,
 {
 	int clean;
 
+	assert(opt->ancestor == NULL);
+
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
 	clean = merge_recursive_internal(opt, h1, h2, merge_bases, result);
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index d23b948f27..7fddcc8c73 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -1562,6 +1562,7 @@ test_expect_success 'check nested conflicts' '
 		cd nested_conflicts &&
 
 		git clean -f &&
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L2^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1582,7 +1583,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:a >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1594,7 +1595,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:b >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1732,6 +1733,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 	(
 		cd virtual_merge_base_has_nested_conflicts &&
 
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L3^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1760,7 +1762,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 		cp left merged-once &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			merged-once \
 			base        \
diff --git a/t/t6047-diff3-conflict-markers.sh b/t/t6047-diff3-conflict-markers.sh
new file mode 100755
index 0000000000..f69c8256bc
--- /dev/null
+++ b/t/t6047-diff3-conflict-markers.sh
@@ -0,0 +1,189 @@
+#!/bin/sh
+
+test_description='recursive merge diff3 style conflict markers'
+
+. ./test-lib.sh
+
+# Setup:
+#          L1
+#            \
+#             ?
+#            /
+#          R1
+#
+# Where:
+#   L1 and R1 both have a file named 'content' but have no common history
+#
+
+test_expect_success 'setup no merge base' '
+	test_create_repo no_merge_base &&
+	(
+		cd no_merge_base &&
+
+		git checkout -b L &&
+		test_commit A content A &&
+
+		git checkout --orphan R &&
+		test_commit B content B
+	)
+'
+
+test_expect_success 'check no merge base' '
+	(
+		cd no_merge_base &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge --allow-unrelated-histories -s recursive R^0 &&
+
+		grep "|||||| <empty tree>" content
+	)
+'
+
+# Setup:
+#          L1
+#         /  \
+#   master    ?
+#         \  /
+#          R1
+#
+# Where:
+#   L1 and R1 have modified the same file ('content') in conflicting ways
+#
+
+test_expect_success 'setup unique merge base' '
+	test_create_repo unique_merge_base &&
+	(
+		cd unique_merge_base &&
+
+		test_commit base content "1
+2
+3
+4
+5
+" &&
+
+		git branch L &&
+		git branch R &&
+
+		git checkout L &&
+		test_commit L content "1
+2
+3
+4
+5
+7" &&
+
+		git checkout R &&
+		git rm content &&
+		test_commit R renamed "1
+2
+3
+4
+5
+six"
+	)
+'
+
+test_expect_success 'check unique merge base' '
+	(
+		cd unique_merge_base &&
+
+		git checkout L^0 &&
+		MASTER=$(git rev-parse --short master) &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| $MASTER:content" renamed
+	)
+'
+
+# Setup:
+#          L1---L2--L3
+#         /  \ /      \
+#   master    X1       ?
+#         \  / \      /
+#          R1---R2--R3
+#
+# Where:
+#   commits L1 and R1 have modified the same file in non-conflicting ways
+#   X1 is an auto-generated merge-base used when merging L1 and R1
+#   commits L2 and R2 are merges of R1 and L1 into L1 and R1, respectively
+#   commits L3 and R3 both modify 'content' in conflicting ways
+#
+
+test_expect_success 'setup multiple merge bases' '
+	test_create_repo multiple_merge_bases &&
+	(
+		cd multiple_merge_bases &&
+
+		test_commit initial content "1
+2
+3
+4
+5" &&
+
+		git branch L &&
+		git branch R &&
+
+		# Create L1
+		git checkout L &&
+		test_commit L1 content "0
+1
+2
+3
+4
+5" &&
+
+		# Create R1
+		git checkout R &&
+		test_commit R1 content "1
+2
+3
+4
+5
+6" &&
+
+		# Create L2
+		git checkout L &&
+		git merge R1 &&
+
+		# Create R2
+		git checkout R &&
+		git merge L1 &&
+
+		# Create L3
+		git checkout L &&
+		test_commit L3 content "0
+1
+2
+3
+4
+5
+A" &&
+
+		# Create R3
+		git checkout R &&
+		git rm content &&
+		test_commit R3 renamed "0
+2
+3
+4
+5
+six"
+	)
+'
+
+test_expect_success 'check multiple merge bases' '
+	(
+		cd multiple_merge_bases &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| merged common ancestors:content" renamed
+	)
+'
+
+test_done
-- 
2.22.0.550.g71c37a0928.dirty

Re: [PATCH v2 01/20] merge-recursive: fix minor memory leak in error condition

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Returning before freeing the allocated buffer is suboptimal; as with
> elsewhere in the same function, make sure buf gets free'd.

I do not have a real objection to the patch text, but the above
justification does not make much sense to me.  The original code
returned an error when buf is NULL, so there is no leak returning
directly, without jumping to free_buf: label (whose only effect is
to free(buf) and return the value in ret).

The real value of this change is it may future-proof the codepath by
making sure that everybody after an error goes to the same place to
free all resources---which happens to be only buf in the current
code, but this change allows it to change in the future, where some
additional resources may have been allocated before the call to
read_object_file() and freed after free_buf: label.

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 12300131fc..1163508811 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -934,9 +934,11 @@ static int update_file_flags(struct merge_options *opt,
>  		}
>  
>  		buf = read_object_file(&contents->oid, &type, &size);
> -		if (!buf)
> -			return err(opt, _("cannot read object %s '%s'"),
> -				   oid_to_hex(&contents->oid), path);
> +		if (!buf) {
> +			ret = err(opt, _("cannot read object %s '%s'"),
> +				  oid_to_hex(&contents->oid), path);
> +			goto free_buf;
> +		}
>  		if (type != OBJ_BLOB) {
>  			ret = err(opt, _("blob expected for %s '%s'"),
>  				  oid_to_hex(&contents->oid), path);

Re: [PATCH v2 02/20] merge-recursive: remove another implicit dependency on the_repository

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Commit d7cf3a96e9a0 ("merge-recursive.c: remove implicit dependency on
> the_repository", 2019-01-12) and follow-ups like commit 34e7771bc644
> ("Use the right 'struct repository' instead of the_repository",
> 2019-06-27), removed most implicit uses of the_repository.  Convert
> calls to get_commit_tree() to instead use repo_get_commit_tree() to get
> rid of another.

You are getting rid of three more!!!

Nice ;-)

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 1163508811..37bb94fb4d 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3571,8 +3571,11 @@ int merge_recursive(struct merge_options *opt,
>  		repo_read_index(opt->repo);
>  
>  	opt->ancestor = "merged common ancestors";
> -	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
> -			    get_commit_tree(merged_common_ancestors),
> +	clean = merge_trees(opt,
> +			    repo_get_commit_tree(opt->repo, h1),
> +			    repo_get_commit_tree(opt->repo, h2),
> +			    repo_get_commit_tree(opt->repo,
> +						 merged_common_ancestors),
>  			    &mrtree);
>  	if (clean < 0) {
>  		flush_output(opt);

Re: [PATCH v2 03/20] Ensure index matches head before invoking merge machinery, round N

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> ...
> So, load the index in builtin/merge-recursive.c, reload the in-memory
> index in builtin/stash.c, and modify the t3030 testcase to correctly
> setup the index and make sure that the test fails in the expected way
> (meaning it reports a rename/rename conflict).

The last paragraph is what describes what this step does.  What is
left unsaid is how that small step relates to the change in the
larger picture (including the title).  We do want to have a code
that ensures the index has no higher-stage entries and match HEAD
at the beginning of merge_recursive backend (as your lengthy preamble
leading to the paragraph explains), but adding the code to populate
in-core index to two codepaths does not make them check the condition
automatically---it is only half the story, no?

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/merge-recursive.c  | 4 ++++
>  builtin/stash.c            | 2 ++
>  t/t3030-merge-recursive.sh | 9 ++++++++-
>  3 files changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
> index 5b910e351e..a4bfd8fc51 100644
> --- a/builtin/merge-recursive.c
> +++ b/builtin/merge-recursive.c
> @@ -1,3 +1,4 @@
> +#include "cache.h"
>  #include "builtin.h"
>  #include "commit.h"
>  #include "tag.h"
> @@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
>  	if (argc - i != 3) /* "--" "<head>" "<remote>" */
>  		die(_("not handling anything other than two heads merge."));
>  
> +	if (repo_read_index_unmerged(the_repository))
> +		die_resolve_conflict("merge");
> +
>  	o.branch1 = argv[++i];
>  	o.branch2 = argv[++i];
>  
> diff --git a/builtin/stash.c b/builtin/stash.c
> index b5a301f24d..4aa47785f9 100644
> --- a/builtin/stash.c
> +++ b/builtin/stash.c
> @@ -427,6 +427,8 @@ static int do_apply_stash(const char *prefix, struct stash_info *info,
>  				return error(_("could not save index tree"));
>  
>  			reset_head();
> +			discard_cache();
> +			read_cache();
>  		}
>  	}
>  
> diff --git a/t/t3030-merge-recursive.sh b/t/t3030-merge-recursive.sh
> index ff641b348a..a37bcc58a0 100755
> --- a/t/t3030-merge-recursive.sh
> +++ b/t/t3030-merge-recursive.sh
> @@ -667,15 +667,22 @@ test_expect_success 'merging with triple rename across D/F conflict' '
>  test_expect_success 'merge-recursive remembers the names of all base trees' '
>  	git reset --hard HEAD &&
>  
> +	# make the index match $c1 so that merge-recursive below does not
> +	# fail early
> +	git diff --binary HEAD $c1 -- | git apply --cached &&
> +
>  	# more trees than static slots used by oid_to_hex()
>  	for commit in $c0 $c2 $c4 $c5 $c6 $c7
>  	do
>  		git rev-parse "$commit^{tree}"
>  	done >trees &&
>  
> -	# ignore the return code -- it only fails because the input is weird
> +	# ignore the return code; it only fails because the input is weird...
>  	test_must_fail git -c merge.verbosity=5 merge-recursive $(cat trees) -- $c1 $c3 >out &&
>  
> +	# ...but make sure it fails in the expected way
> +	test_i18ngrep CONFLICT.*rename/rename out &&
> +
>  	# merge-recursive prints in reverse order, but we do not care
>  	sort <trees >expect &&
>  	sed -n "s/^virtual //p" out | sort >actual &&

Re: [PATCH v2 04/20] merge-recursive: exit early if index != head

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Make sure we do the index == head check at the beginning of the merge,
> and error out immediately if it fails.  While we're at it, fix a small
> leak in the show-the-error codepath.

As the call to repo_index_has_changes() is moved to the very
beginning of merge_recursive() and merge_trees(), the workhorse of
the merge machinery, merge_trees_internal(), can lose it.

> +static int merge_start(struct merge_options *opt, struct tree *head)
> +{
> +	struct strbuf sb = STRBUF_INIT;
> +
> +	assert(opt->branch1 && opt->branch2);

This is a new assertion that did not exist in the original, isn't
it?  I do not object to new sensible assertions, and I think these
two fields must be non-null in a freshly initialized merge_options
structure, but shouldn't we be discussing if these two fields should
be non-NULL, and if there are other fields in the same structure
that we should be adding new assertions on, in a separate step on
its own?

> +	if (repo_index_has_changes(opt->repo, head, &sb)) {
> +		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
> +		    sb.buf);
> +		strbuf_release(&sb);
> +		return -1;
> +	}
> +
> +	return 0;
> +}

Re: [PATCH v2 05/20] merge-recursive: remove useless parameter in merge_trees()

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> merge_trees() took a results parameter that would only be written when
> opt->call_depth was positive, which is never the case now that
> merge_trees_internal() has been split from merge_trees().  Remove the
> misleading and unused parameter from merge_trees().
>
> While at it, add some comments explaining how the output of
> merge_trees() and merge_recursive() differ.

Makes sense.

Re: [PATCH v2 06/20] merge-recursive: don't force external callers to do our logging

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Alternatively, you can view this as "make the merge functions behave
> more similarly."  merge-recursive has three different entry points:
> merge_trees(), merge_recursive(), and merge_recursive_generic().  Two of
> these would call diff_warn_rename_limit(), but merge_trees() didn't.
> This lead to callers of merge_trees() needing to manually call
> diff_warn_rename_limit() themselves.  Move this to the new
> merge_finalize() function to make sure that all three entry points run
> this function.

Interesting.  It seems that b520abf1c8f did a suboptimal jobs but
this step cleans it up quite nicely.

Are there callers of merge_trees() in other codepaths, and do they
want this change?

There is only one, builtin/checkout.c::merge_working_tree(), which
is used when switching to a different commit.  I think it would not
hurt to give the same warning (but I also think it would not hurt to
simply disable rename detection in that context); we should record
the whatever decision we make in the log message.


> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c | 7 +++----
>  sequencer.c       | 1 -
>  2 files changed, 3 insertions(+), 5 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 4a481c3929..308e350ff1 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3583,9 +3583,6 @@ static int merge_recursive_internal(struct merge_options *opt,
>  	flush_output(opt);
>  	if (!opt->call_depth && opt->buffer_output < 2)
>  		strbuf_release(&opt->obuf);
> -	if (show(opt, 2))
> -		diff_warn_rename_limit("merge.renamelimit",
> -				       opt->needed_rename_limit, 0);
>  	return clean;
>  }
>  
> @@ -3607,7 +3604,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
>  
>  static void merge_finalize(struct merge_options *opt)
>  {
> -	/* Common code for wrapping up merges will be added here later */
> +	if (show(opt, 2))
> +		diff_warn_rename_limit("merge.renamelimit",
> +				       opt->needed_rename_limit, 0);
>  }
>  
>  int merge_trees(struct merge_options *opt,
> diff --git a/sequencer.c b/sequencer.c
> index c4ed30f1b4..094a4dd03d 100644
> --- a/sequencer.c
> +++ b/sequencer.c
> @@ -617,7 +617,6 @@ static int do_recursive_merge(struct repository *r,
>  	if (is_rebase_i(opts) && clean <= 0)
>  		fputs(o.obuf.buf, stdout);
>  	strbuf_release(&o.obuf);
> -	diff_warn_rename_limit("merge.renamelimit", o.needed_rename_limit, 0);
>  	if (clean < 0) {
>  		rollback_lock_file(&index_lock);
>  		return clean;

Re: [PATCH v2 07/20] Use write_index_as_tree() in lieu of write_tree_from_memory()

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> diff --git a/cache-tree.c b/cache-tree.c
> index 706ffcf188..99144b1704 100644
> --- a/cache-tree.c
> +++ b/cache-tree.c
> @@ -613,14 +613,19 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
>  	int entries, was_valid;
>  	struct lock_file lock_file = LOCK_INIT;
>  	int ret = 0;
> +	int access_disk = !(flags & WRITE_TREE_FROM_MEMORY);

Shouldn't we go one step futher and make the bulk of in-core index
processing into a new helper function, while making
write_index_as_tree() a thin-wrapper around it, i.e.

	write_index_as_tree() 
	{
		lock the index for update;
		read the on-disk index;
		call that new helper function to write a tree;
		update the on-disk index;
	}

and reuse the helper from
merge-recursive.c::write_tree_from_memory() while keeping the call
to the latter in merge_trees_internal()?  Wouldn't that approach
let you do this without adding an extra flag bit?

Also, there used to be a check to ensure that the in-core index fed
to write_tree_from_memory() is fully merged and otherwise dump the
unmerged entries with BUG().  Can we simply lose it?  I know you
return with "error building trees" from merge_trees_internal() but
it does not BUG().

Re: [PATCH v2 01/20] merge-recursive: fix minor memory leak in error condition

[Elijah Newren]

On Fri, Jul 26, 2019 at 11:31 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > Returning before freeing the allocated buffer is suboptimal; as with
> > elsewhere in the same function, make sure buf gets free'd.
>
> I do not have a real objection to the patch text, but the above
> justification does not make much sense to me.  The original code
> returned an error when buf is NULL, so there is no leak returning
> directly, without jumping to free_buf: label (whose only effect is
> to free(buf) and return the value in ret).
>
> The real value of this change is it may future-proof the codepath by
> making sure that everybody after an error goes to the same place to
> free all resources---which happens to be only buf in the current
> code, but this change allows it to change in the future, where some
> additional resources may have been allocated before the call to
> read_object_file() and freed after free_buf: label.

Indeed, not sure how I overlooked that buf was NULL since that was the
precise check I was modifying.  I'll clean up the commit message.

Re: [PATCH v2 03/20] Ensure index matches head before invoking merge machinery, round N

[Elijah Newren]

On Fri, Jul 26, 2019 at 12:15 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > ...
> > So, load the index in builtin/merge-recursive.c, reload the in-memory
> > index in builtin/stash.c, and modify the t3030 testcase to correctly
> > setup the index and make sure that the test fails in the expected way
> > (meaning it reports a rename/rename conflict).
>
> The last paragraph is what describes what this step does.  What is
> left unsaid is how that small step relates to the change in the
> larger picture (including the title).  We do want to have a code
> that ensures the index has no higher-stage entries and match HEAD
> at the beginning of merge_recursive backend (as your lengthy preamble
> leading to the paragraph explains), but adding the code to populate
> in-core index to two codepaths does not make them check the condition
> automatically---it is only half the story, no?

Indeed; the other half is the movement of the index==head check in
patch 4 so that it happens immediately and ensures we don't run into
this problem again.  I can add a note to the commit message that the
next patch in the series will be taking care of that half of the
problem.

Re: [PATCH v2 04/20] merge-recursive: exit early if index != head

[Elijah Newren]

On Fri, Jul 26, 2019 at 12:32 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > Make sure we do the index == head check at the beginning of the merge,
> > and error out immediately if it fails.  While we're at it, fix a small
> > leak in the show-the-error codepath.
>
> As the call to repo_index_has_changes() is moved to the very
> beginning of merge_recursive() and merge_trees(), the workhorse of
> the merge machinery, merge_trees_internal(), can lose it.

Is this just a re-summarization (a perfectly good one), or a
suggestion for alternate wording for the commit message?

> > +static int merge_start(struct merge_options *opt, struct tree *head)
> > +{
> > +     struct strbuf sb = STRBUF_INIT;
> > +
> > +     assert(opt->branch1 && opt->branch2);
>
> This is a new assertion that did not exist in the original, isn't
> it?  I do not object to new sensible assertions, and I think these
> two fields must be non-null in a freshly initialized merge_options
> structure, but shouldn't we be discussing if these two fields should
> be non-NULL, and if there are other fields in the same structure
> that we should be adding new assertions on, in a separate step on
> its own?

Good point.  The only other one I saw was opt->ancestor, and while it
does have conditions it should satisfy, somewhat surprisingly the
condition is the opposite in merge_trees() vs. merge_recursive().  So
I think this is the only check that makes sense to add to
merge_start(), but I can move that out into a separate patch and add
some words about why just these two.

Re: [PATCH v2 06/20] merge-recursive: don't force external callers to do our logging

[Elijah Newren]

On Fri, Jul 26, 2019 at 12:57 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > Alternatively, you can view this as "make the merge functions behave
> > more similarly."  merge-recursive has three different entry points:
> > merge_trees(), merge_recursive(), and merge_recursive_generic().  Two of
> > these would call diff_warn_rename_limit(), but merge_trees() didn't.
> > This lead to callers of merge_trees() needing to manually call
> > diff_warn_rename_limit() themselves.  Move this to the new
> > merge_finalize() function to make sure that all three entry points run
> > this function.
>
> Interesting.  It seems that b520abf1c8f did a suboptimal jobs but
> this step cleans it up quite nicely.
>
> Are there callers of merge_trees() in other codepaths, and do they
> want this change?
>
> There is only one, builtin/checkout.c::merge_working_tree(), which
> is used when switching to a different commit.  I think it would not
> hurt to give the same warning (but I also think it would not hurt to
> simply disable rename detection in that context); we should record
> the whatever decision we make in the log message.

I was surprised to read this because I thought I did that...but I
guess that was in patch 15 for a slightly different case.  Yeah, I can
add some words to the commit message about this.

Re: [PATCH v2 07/20] Use write_index_as_tree() in lieu of write_tree_from_memory()

[Elijah Newren]

On Fri, Jul 26, 2019 at 1:11 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > diff --git a/cache-tree.c b/cache-tree.c
> > index 706ffcf188..99144b1704 100644
> > --- a/cache-tree.c
> > +++ b/cache-tree.c
> > @@ -613,14 +613,19 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
> >       int entries, was_valid;
> >       struct lock_file lock_file = LOCK_INIT;
> >       int ret = 0;
> > +     int access_disk = !(flags & WRITE_TREE_FROM_MEMORY);
>
> Shouldn't we go one step futher and make the bulk of in-core index
> processing into a new helper function, while making
> write_index_as_tree() a thin-wrapper around it, i.e.
>
>         write_index_as_tree()
>         {
>                 lock the index for update;
>                 read the on-disk index;
>                 call that new helper function to write a tree;
>                 update the on-disk index;
>         }
>
> and reuse the helper from
> merge-recursive.c::write_tree_from_memory() while keeping the call
> to the latter in merge_trees_internal()?  Wouldn't that approach
> let you do this without adding an extra flag bit?

I thought about that briefly yesterday, but the fact that the
write_locked_index() call only happens if !cache_tree_fully_valid()
meant refactoring slightly more to get the helper to also return that
boolean value, and since I was a little unsure of myself with
cache-tree stuff in general I wanted to propose what looked like the
minimally invasive changes first (by which I mean smallest patch).
I'll take a closer look at this path.

> Also, there used to be a check to ensure that the in-core index fed
> to write_tree_from_memory() is fully merged and otherwise dump the
> unmerged entries with BUG().  Can we simply lose it?  I know you
> return with "error building trees" from merge_trees_internal() but
> it does not BUG().

I thought about that yesterday and decided, "Nah, it's a developer
only debug message used during development.  I've _never_ seen anyone
report those messages, and I only saw them when I was making bad
changes during development when I first started a decade ago."  Then
Emily posts a report today showing that exact BUG message being hit
with git-2.22.0, and how she wouldn't have been able to get all that
extra information in her analysis without that bit of information
being reported.

So, yeah, I need to put something from those BUG() messages back in;
they clearly helped with that issue, and might help again in the
future.

Re: [PATCH v2 07/20] Use write_index_as_tree() in lieu of write_tree_from_memory()

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> I thought about that briefly yesterday, but the fact that the
> write_locked_index() call only happens if !cache_tree_fully_valid()
> meant refactoring slightly more to get the helper to also return that
> boolean value, and since I was a little unsure of myself with
> cache-tree stuff in general I wanted to propose what looked like the
> minimally invasive changes first (by which I mean smallest patch).

Or have the caller check if cache-tree is fully valid, which is the
only case that you can build a tree (and a fully merged index would
be fully valid after you do cache_tree_update()).

> I'll take a closer look at this path.
> ...
> So, yeah, I need to put something from those BUG() messages back in;
> they clearly helped with that issue, and might help again in the
> future.

Thanks.

Re: [PATCH v2 01/20] merge-recursive: fix minor memory leak in error condition

[Derrick Stolee]

On 7/26/2019 2:31 PM, Junio C Hamano wrote:
> Elijah Newren <newren@gmail.com> writes:
> 
>> Returning before freeing the allocated buffer is suboptimal; as with
>> elsewhere in the same function, make sure buf gets free'd.
> 
> I do not have a real objection to the patch text, but the above
> justification does not make much sense to me.  The original code
> returned an error when buf is NULL, so there is no leak returning
> directly, without jumping to free_buf: label (whose only effect is
> to free(buf) and return the value in ret).
> 
> The real value of this change is it may future-proof the codepath by
> making sure that everybody after an error goes to the same place to
> free all resources---which happens to be only buf in the current
> code, but this change allows it to change in the future, where some
> additional resources may have been allocated before the call to
> read_object_file() and freed after free_buf: label.

It should be noted that any future change to make the "free_buf:" label
mean "free everything" then it should be renamed to "cleanup:" or similar.

Not that we should do that now, as that would muddy the blame.

Thanks,
-Stolee

Re: [PATCH v2 04/20] merge-recursive: exit early if index != head

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> On Fri, Jul 26, 2019 at 12:32 PM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> Elijah Newren <newren@gmail.com> writes:
>>
>> > Make sure we do the index == head check at the beginning of the merge,
>> > and error out immediately if it fails.  While we're at it, fix a small
>> > leak in the show-the-error codepath.
>>
>> As the call to repo_index_has_changes() is moved to the very
>> beginning of merge_recursive() and merge_trees(), the workhorse of
>> the merge machinery, merge_trees_internal(), can lose it.
>
> Is this just a re-summarization (a perfectly good one), or a
> suggestion for alternate wording for the commit message?

The usual "thinking aloud to see if I understood the patch and the
author's thought behind it, and give others a chance to correct me"
kind of review.

[PATCH v3 00/24] Clean up merge API

[Elijah Newren]

Before writing a replacement merge strategy for recursive, I decided
to first cleanup the merge API -- streamlining merge-recursive.h and
making it more readable.

Thanks to Dscho and Junio for feedback on v1 & v2.

Changes since v2:
  * Addressed feedback from Junio and Dscho
    * splitting a couple patches they highlighted as needing splitting
    * fixed up several commit messages
    * refactored cache-tree write-index-as-tree functions to share code
    * created a separate commit with more sanity checks for merge_options
  * Rebased on master (due to en/disable-dir-rename-in-recursive-merge)
  * Found and fixed a minor bug in checkout; put this and related patches
    at the beginning of the series
  * Incorporated a patch from Stolee, though I removed the repo-settings
    bits to make our patch series independent, and modified it to update
    more existing code to use the new enum values.

Things I'd like reviewers to focus on, in priority order:
  * Patch 12 -- I'm still unsure about cache-tree stuff
  * The new patches (2, 3, 5, 23); in more detail:
    * Patch 5 from Stolee -- I modified his original patch a fair amount
      but left him as author; it's a simple patch, but extra eyes to
      verify I didn't blame him for some nasty mistake of mine would be
      good.
    * Patch 2 fixes a minor issue with checkout -m
    * Patch 23 is the more thorough sanity checks Junio suggested
    * Patch 3 was mostly split out from what is now patch 4, so it was
      mostly reviewed before but it now ties in nicely with patch 2.

Derrick Stolee (1):
  merge-recursive: introduce an enum for detect_directory_renames values

Elijah Newren (23):
  merge-recursive: be consistent with assert
  checkout: provide better conflict hunk description with detached HEAD
  merge-recursive: enforce opt->ancestor != NULL when calling
    merge_trees()
  merge-recursive: provide a better label for diff3 common ancestor
  merge-recursive: future-proof update_file_flags() against memory leaks
  merge-recursive: remove another implicit dependency on the_repository
  Ensure index matches head before invoking merge machinery, round N
  merge-recursive: exit early if index != head
  merge-recursive: remove useless parameter in merge_trees()
  merge-recursive: don't force external callers to do our logging
  cache-tree: share code between functions writing an index as a tree
  merge-recursive: fix some overly long lines
  merge-recursive: use common name for ancestors/common/base_list
  merge-recursive: rename 'mrtree' to 'result_tree', for clarity
  merge-recursive: rename merge_options argument to opt in header
  merge-recursive: move some definitions around to clean up the header
  merge-recursive: consolidate unnecessary fields in merge_options
  merge-recursive: comment and reorder the merge_options fields
  merge-recursive: avoid losing output and leaking memory holding that
    output
  merge-recursive: split internal fields into a separate struct
  merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
  merge-recursive: add sanity checks for relevant merge_options
  merge-recursive: alphabetize include list

 builtin/am.c                      |   2 +-
 builtin/checkout.c                |  14 +-
 builtin/merge-recursive.c         |   4 +
 builtin/stash.c                   |   2 +
 cache-tree.c                      |  81 +++--
 cache-tree.h                      |   3 +-
 merge-recursive.c                 | 571 ++++++++++++++++++------------
 merge-recursive.h                 | 164 +++++----
 sequencer.c                       |   5 +-
 t/t3030-merge-recursive.sh        |   9 +-
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 189 ++++++++++
 12 files changed, 718 insertions(+), 334 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

Range-diff:
19:  d6158e555d =  1:  21f1e04dc9 merge-recursive: be consistent with assert
 -:  ---------- >  2:  ac24702773 checkout: provide better conflict hunk description with detached HEAD
 -:  ---------- >  3:  fd14ed9490 merge-recursive: enforce opt->ancestor != NULL when calling merge_trees()
20:  814a65ecab !  4:  540a1d17d7 merge-recursive: provide a better label for diff3 common ancestor
    @@ Commit message
             1:   <abbreviated commit hash>
             0:   "<empty tree>"
     
    -    Also, the code in merge_3way making use of opt->ancestor was overly
    -    complex because it tried to handle the case of opt->ancestor being NULL.
    -    We always set it first, though, so just add an assert that opt->ancestor
    -    is not NULL and simplify the surrounding code.
    -
         Tests have also been added to check that we get the right ancestor name
         for each of the three cases.
     
    +    Also, since merge_recursive() and merge_trees() have polar opposite
    +    pre-conditions for opt->ancestor, document merge_recursive()'s
    +    pre-condition with an assertion.  (An assertion was added to
    +    merge_trees() already a few commits ago.)  The differences in
    +    pre-conditions stem from two factors: (1) merge_trees() does not recurse
    +    and thus does not have multiple sub-merges to worry about -- each of
    +    which would require a different value for opt->ancestor, (2)
    +    merge_trees() is only passed trees rather than commits and thus cannot
    +    internally guess as good of a label.  Thus, while external callers of
    +    merge_trees() are required to provide a non-NULL opt->ancestor,
    +    merge_recursive() expects to set this value itself.
    +
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
      ## merge-recursive.c ##
    -@@ merge-recursive.c: static int merge_3way(struct merge_options *opt,
    - {
    - 	mmfile_t orig, src1, src2;
    - 	struct ll_merge_options ll_opts = {0};
    --	char *base_name, *name1, *name2;
    -+	char *base, *name1, *name2;
    - 	int merge_status;
    - 
    - 	ll_opts.renormalize = opt->renormalize;
    -@@ merge-recursive.c: static int merge_3way(struct merge_options *opt,
    - 		}
    - 	}
    - 
    --	assert(a->path && b->path && o->path);
    --	if (strcmp(a->path, b->path) ||
    --	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
    --		base_name = opt->ancestor == NULL ? NULL :
    --			mkpathdup("%s:%s", opt->ancestor, o->path);
    -+	assert(a->path && b->path && o->path && opt->ancestor);
    -+	if (strcmp(a->path, b->path) || strcmp(a->path, o->path) != 0) {
    -+		base  = mkpathdup("%s:%s", opt->ancestor, o->path);
    - 		name1 = mkpathdup("%s:%s", branch1, a->path);
    - 		name2 = mkpathdup("%s:%s", branch2, b->path);
    - 	} else {
    --		base_name = opt->ancestor == NULL ? NULL :
    --			mkpathdup("%s", opt->ancestor);
    -+		base  = mkpathdup("%s", opt->ancestor);
    - 		name1 = mkpathdup("%s", branch1);
    - 		name2 = mkpathdup("%s", branch2);
    - 	}
    -@@ merge-recursive.c: static int merge_3way(struct merge_options *opt,
    - 	read_mmblob(&src1, &a->oid);
    - 	read_mmblob(&src2, &b->oid);
    - 
    --	merge_status = ll_merge(result_buf, a->path, &orig, base_name,
    -+	merge_status = ll_merge(result_buf, a->path, &orig, base,
    - 				&src1, name1, &src2, name2,
    - 				opt->repo->index, &ll_opts);
    - 
    --	free(base_name);
    -+	free(base);
    - 	free(name1);
    - 	free(name2);
    - 	free(orig.ptr);
    -@@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
    - 	struct commit *merged_merge_bases;
    - 	struct tree *result_tree;
    +@@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    + 	struct commit *merged_common_ancestors;
    + 	struct tree *mrtree;
      	int clean;
     +	int num_merge_bases;
     +	struct strbuf merge_base_abbrev = STRBUF_INIT;
    ++
    ++	if (!opt->call_depth)
    ++		assert(opt->ancestor == NULL);
      
      	if (show(opt, 4)) {
      		output(opt, 4, _("Merging:"));
    -@@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
    +@@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      			output_commit_title(opt, iter->item);
      	}
      
    -+	num_merge_bases = commit_list_count(merge_bases);
    - 	merged_merge_bases = pop_commit(&merge_bases);
    - 	if (merged_merge_bases == NULL) {
    ++	num_merge_bases = commit_list_count(ca);
    + 	merged_common_ancestors = pop_commit(&ca);
    + 	if (merged_common_ancestors == NULL) {
      		/* if there is no common ancestor, use an empty tree */
    -@@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
    - 	if (!opt->priv->call_depth)
    +@@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    + 	if (!opt->call_depth)
      		repo_read_index(opt->repo);
      
     -	opt->ancestor = "merged common ancestors";
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     +		break;
     +	case 1:
     +		strbuf_add_unique_abbrev(&merge_base_abbrev,
    -+					 &merged_merge_bases->object.oid,
    ++					 &merged_common_ancestors->object.oid,
     +					 DEFAULT_ABBREV);
     +		opt->ancestor = merge_base_abbrev.buf;
     +		break;
     +	default:
     +		opt->ancestor = "merged common ancestors";
     +	}
    - 	clean = merge_trees_internal(opt,
    - 				     repo_get_commit_tree(opt->repo, h1),
    - 				     repo_get_commit_tree(opt->repo, h2),
    - 				     repo_get_commit_tree(opt->repo,
    - 							  merged_merge_bases),
    - 				     &result_tree);
    + 	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
    + 			    get_commit_tree(merged_common_ancestors),
    + 			    &mrtree);
     +	strbuf_release(&merge_base_abbrev);
      	if (clean < 0) {
      		flush_output(opt);
      		return clean;
    -@@ merge-recursive.c: int merge_trees(struct merge_options *opt,
    - 	int clean;
    - 	struct tree *ignored;
    - 
    -+	assert(opt->ancestor != NULL);
    -+
    - 	if (merge_start(opt, head))
    - 		return -1;
    - 	clean = merge_trees_internal(opt, head, merge, merge_base, &ignored);
    -@@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    - {
    - 	int clean;
    - 
    -+	assert(opt->ancestor == NULL);
    -+
    - 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
    - 		return -1;
    - 	clean = merge_recursive_internal(opt, h1, h2, merge_bases, result);
     
      ## t/t6036-recursive-corner-cases.sh ##
     @@ t/t6036-recursive-corner-cases.sh: test_expect_success 'check nested conflicts' '
 -:  ---------- >  5:  19ff6a9503 merge-recursive: introduce an enum for detect_directory_renames values
 1:  a640f0f2d0 !  6:  5e44146da1 merge-recursive: fix minor memory leak in error condition
    @@ Metadata
     Author: Elijah Newren <newren@gmail.com>
     
      ## Commit message ##
    -    merge-recursive: fix minor memory leak in error condition
    +    merge-recursive: future-proof update_file_flags() against memory leaks
     
    -    Returning before freeing the allocated buffer is suboptimal; as with
    -    elsewhere in the same function, make sure buf gets free'd.
    +    There is a 'free_buf' label to which all but one of the error paths in
    +    update_file_flags() jump; that error case involves a NULL buf and is
    +    thus not a memory leak.  However, make that error case execute the same
    +    deallocation code anyway so that if anyone adds any additional memory
    +    allocations or deallocations, then all error paths correctly deallocate
    +    resources.
     
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
 2:  34f0891d96 <  -:  ---------- merge-recursive: remove another implicit dependency on the_repository
 -:  ---------- >  7:  df210eb029 merge-recursive: remove another implicit dependency on the_repository
 3:  26739a7ed0 !  8:  74dd7b8f59 Ensure index matches head before invoking merge machinery, round N
    @@ Commit message
         So, load the index in builtin/merge-recursive.c, reload the in-memory
         index in builtin/stash.c, and modify the t3030 testcase to correctly
         setup the index and make sure that the test fails in the expected way
    -    (meaning it reports a rename/rename conflict).
    +    (meaning it reports a rename/rename conflict).  This makes sure that
    +    all callers actually make the index match head.  The next commit will
    +    then enforce the condition that index matches head earlier so this
    +    problem doesn't return in the future.
     
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
 4:  76cb459b99 !  9:  f04eba4184 merge-recursive: exit early if index != head
    @@ merge-recursive.c: static int process_entry(struct merge_options *opt,
      	int code, clean;
     -	struct strbuf sb = STRBUF_INIT;
     -
    +-	assert(opt->ancestor != NULL);
    +-
     -	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
     -		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
     -		    sb.buf);
    @@ merge-recursive.c: static struct commit_list *reverse_commit_list(struct commit_
      {
      	struct commit_list *iter;
      	struct commit *merged_common_ancestors;
    +@@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    + 	int num_merge_bases;
    + 	struct strbuf merge_base_abbrev = STRBUF_INIT;
    + 
    +-	if (!opt->call_depth)
    +-		assert(opt->ancestor == NULL);
    +-
    + 	if (show(opt, 4)) {
    + 		output(opt, 4, _("Merging:"));
    + 		output_commit_title(opt, h1);
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      		saved_b2 = opt->branch2;
      		opt->branch1 = "Temporary merge branch 1";
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      			return -1;
      		opt->branch1 = saved_b1;
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    - 		repo_read_index(opt->repo);
    - 
    - 	opt->ancestor = "merged common ancestors";
    + 	default:
    + 		opt->ancestor = "merged common ancestors";
    + 	}
     -	clean = merge_trees(opt,
     -			    repo_get_commit_tree(opt->repo, h1),
     -			    repo_get_commit_tree(opt->repo, h2),
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
     +				     repo_get_commit_tree(opt->repo,
     +							  merged_common_ancestors),
     +				     &mrtree);
    + 	strbuf_release(&merge_base_abbrev);
      	if (clean < 0) {
      		flush_output(opt);
    - 		return clean;
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      	return clean;
      }
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
     +{
     +	struct strbuf sb = STRBUF_INIT;
     +
    -+	assert(opt->branch1 && opt->branch2);
    -+
     +	if (repo_index_has_changes(opt->repo, head, &sb)) {
     +		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
     +		    sb.buf);
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
     +{
     +	int clean;
     +
    ++	assert(opt->ancestor != NULL);
    ++
     +	if (merge_start(opt, head))
     +		return -1;
     +	clean = merge_trees_internal(opt, head, merge, common, result);
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
     +{
     +	int clean;
     +
    ++	assert(opt->ancestor == NULL);
    ++
     +	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
     +		return -1;
     +	clean = merge_recursive_internal(opt, h1, h2, ca, result);
 5:  2560458522 ! 10:  8688d40739 merge-recursive: remove useless parameter in merge_trees()
    @@ merge-recursive.c: static void merge_finalize(struct merge_options *opt)
      	int clean;
     +	struct tree *ignored;
      
    + 	assert(opt->ancestor != NULL);
    + 
      	if (merge_start(opt, head))
      		return -1;
     -	clean = merge_trees_internal(opt, head, merge, common, result);
 6:  fb340fbe56 ! 11:  a92d460707 merge-recursive: don't force external callers to do our logging
    @@ Commit message
         merge_finalize() function to make sure that all three entry points run
         this function.
     
    +    Note that there are two external callers of merge_trees(), one in
    +    sequencer.c and one in builtin/checkout.c.  The one in sequencer.c is
    +    cleaned up by this patch and just transfers where the call to
    +    diff_warn_rename_limit() is made; the one in builtin/checkout.c is for
    +    switching to a different commit and in the very rare case where the
    +    warning might be triggered, it would probably be helpful to include
    +    (e.g. if someone is modifying a file that has been renamed in moving to
    +    the other commit, but there are so many renames between the commits that
    +    the limit kicks in and none are detected, it may help to have an
    +    explanation about why they got a delete/modify conflict instead of a
    +    proper content merge in a renamed file).
    +
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
      ## merge-recursive.c ##
 7:  884305a3a6 <  -:  ---------- Use write_index_as_tree() in lieu of write_tree_from_memory()
 -:  ---------- > 12:  dec0ea7409 cache-tree: share code between functions writing an index as a tree
 8:  870937b31f = 13:  b51f3d1924 merge-recursive: fix some overly long lines
 9:  5127b75ac4 ! 14:  a069cc4cca merge-recursive: use common name for ancestors/common/base_list
    @@ merge-recursive.c: static struct commit_list *reverse_commit_list(struct commit_
     +	struct commit *merged_merge_bases;
      	struct tree *mrtree;
      	int clean;
    - 
    + 	int num_merge_bases;
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      		output_commit_title(opt, h2);
      	}
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
      			output_commit_title(opt, iter->item);
      	}
      
    +-	num_merge_bases = commit_list_count(ca);
     -	merged_common_ancestors = pop_commit(&ca);
     -	if (merged_common_ancestors == NULL) {
    ++	num_merge_bases = commit_list_count(merge_bases);
     +	merged_merge_bases = pop_commit(&merge_bases);
     +	if (merged_merge_bases == NULL) {
      		/* if there is no common ancestor, use an empty tree */
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
      			return err(opt, _("merge returned no commit"));
      	}
      
    +@@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
    + 		break;
    + 	case 1:
    + 		strbuf_add_unique_abbrev(&merge_base_abbrev,
    +-					 &merged_common_ancestors->object.oid,
    ++					 &merged_merge_bases->object.oid,
    + 					 DEFAULT_ABBREV);
    + 		opt->ancestor = merge_base_abbrev.buf;
    + 		break;
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      				     repo_get_commit_tree(opt->repo, h1),
      				     repo_get_commit_tree(opt->repo, h2),
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     -							  merged_common_ancestors),
     +							  merged_merge_bases),
      				     &mrtree);
    + 	strbuf_release(&merge_base_abbrev);
      	if (clean < 0) {
    - 		flush_output(opt);
     @@ merge-recursive.c: static void merge_finalize(struct merge_options *opt)
      int merge_trees(struct merge_options *opt,
      		struct tree *head,
    @@ merge-recursive.c: static void merge_finalize(struct merge_options *opt)
      {
      	int clean;
      	struct tree *ignored;
    +@@ merge-recursive.c: int merge_trees(struct merge_options *opt,
      
      	if (merge_start(opt, head))
      		return -1;
    @@ merge-recursive.c: int merge_trees(struct merge_options *opt,
      		    struct commit **result)
      {
      	int clean;
    +@@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      
      	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
      		return -1;
10:  daee364ce1 ! 15:  93a3ce6b88 merge-recursive: rename 'mrtree' to 'result_tree', for clarity
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     -	struct tree *mrtree;
     +	struct tree *result_tree;
      	int clean;
    - 
    - 	if (show(opt, 4)) {
    + 	int num_merge_bases;
    + 	struct strbuf merge_base_abbrev = STRBUF_INIT;
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      				     repo_get_commit_tree(opt->repo, h2),
      				     repo_get_commit_tree(opt->repo,
      							  merged_merge_bases),
     -				     &mrtree);
     +				     &result_tree);
    + 	strbuf_release(&merge_base_abbrev);
      	if (clean < 0) {
      		flush_output(opt);
    - 		return clean;
    +@@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      	}
      
      	if (opt->call_depth) {
11:  50a7a6f671 = 16:  1b1df10c11 merge-recursive: rename merge_options argument to opt in header
12:  4e9e774dc5 = 17:  1526977a85 merge-recursive: move some definitions around to clean up the header
13:  bf40502fd8 ! 18:  c90f2f15cd merge-recursive: consolidate unnecessary fields in merge_options
    @@ merge-recursive.c: void init_merge_options(struct merge_options *opt,
     -	opt->diff_detect_rename = -1;
     -	opt->merge_detect_rename = -1;
     +	opt->detect_renames = -1;
    - 	opt->detect_directory_renames = 1;
    + 	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
      	merge_recursive_config(opt);
      	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
     @@ merge-recursive.c: int parse_merge_opt(struct merge_options *opt, const char *s)
    @@ merge-recursive.c: int parse_merge_opt(struct merge_options *opt, const char *s)
     
      ## merge-recursive.h ##
     @@ merge-recursive.h: struct merge_options {
    - 	long xdl_opts;
    - 	int verbosity;
    - 	int detect_directory_renames;
    + 		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
    + 		MERGE_DIRECTORY_RENAMES_TRUE = 2
    + 	} detect_directory_renames;
     -	int diff_detect_rename;
     -	int merge_detect_rename;
     -	int diff_rename_limit;
14:  2c39a4be36 ! 19:  6d930dba72 merge-recursive: comment and reorder the merge_options fields
    @@ merge-recursive.c: void init_merge_options(struct merge_options *opt,
      	opt->repo = repo;
     +
     +	opt->detect_renames = -1;
    -+	opt->detect_directory_renames = 1;
    ++	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
     +	opt->rename_limit = -1;
     +
      	opt->verbosity = 2;
    @@ merge-recursive.c: void init_merge_options(struct merge_options *opt,
     +
      	opt->renormalize = 0;
     -	opt->detect_renames = -1;
    --	opt->detect_directory_renames = 1;
    +-	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
     +
     +	string_list_init(&opt->df_conflict_file_set, 1);
     +
    @@ merge-recursive.h: struct commit;
      	const char *ancestor;
      	const char *branch1;
      	const char *branch2;
    +-	enum {
    +-		MERGE_RECURSIVE_NORMAL = 0,
    +-		MERGE_RECURSIVE_OURS,
    +-		MERGE_RECURSIVE_THEIRS
    +-	} recursive_variant;
    +-	const char *subtree_shift;
    +-	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
    +-	unsigned renormalize : 1;
    +-	long xdl_opts;
    +-	int verbosity;
     +
     +	/* rename related options */
     +	int detect_renames;
    -+	int detect_directory_renames;
    -+	int rename_limit;
    -+	int rename_score;
    -+	int show_rename_progress;
    + 	enum {
    + 		MERGE_DIRECTORY_RENAMES_NONE = 0,
    + 		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
    + 		MERGE_DIRECTORY_RENAMES_TRUE = 2
    + 	} detect_directory_renames;
    +-	int detect_renames;
    + 	int rename_limit;
    + 	int rename_score;
    +-	int needed_rename_limit;
    + 	int show_rename_progress;
     +
     +	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
     +	long xdl_opts;
    - 	enum {
    - 		MERGE_RECURSIVE_NORMAL = 0,
    - 		MERGE_RECURSIVE_OURS,
    - 		MERGE_RECURSIVE_THEIRS
    - 	} recursive_variant;
    --	const char *subtree_shift;
    ++	enum {
    ++		MERGE_RECURSIVE_NORMAL = 0,
    ++		MERGE_RECURSIVE_OURS,
    ++		MERGE_RECURSIVE_THEIRS
    ++	} recursive_variant;
     +
     +	/* console output related options */
     +	int verbosity;
    - 	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
    ++	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
     +	struct strbuf obuf;     /* output buffer */
     +
     +	/* miscellaneous control options */
     +	const char *subtree_shift;
    - 	unsigned renormalize : 1;
    --	long xdl_opts;
    --	int verbosity;
    --	int detect_directory_renames;
    --	int detect_renames;
    --	int rename_limit;
    --	int rename_score;
    --	int needed_rename_limit;
    --	int show_rename_progress;
    ++	unsigned renormalize : 1;
     +
     +	/* internal fields used by the implementation (do NOT set these) */
      	int call_depth;
15:  c1c71816eb = 20:  ec3e15f6a8 merge-recursive: avoid losing output and leaking memory holding that output
16:  be47a6bfdf ! 21:  7edfac7048 merge-recursive: split internal fields into a separate struct
    @@ merge-recursive.c: static int merge_submodule(struct merge_options *opt,
      	/* store a in result in case we fail */
      	oidcpy(result, a);
     @@ merge-recursive.c: static int handle_rename_via_dir(struct merge_options *opt,
    - 	int mark_conflicted = (opt->detect_directory_renames == 1);
    + 			       MERGE_DIRECTORY_RENAMES_CONFLICT);
      	assert(ren->dir_rename_original_dest);
      
     -	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
    @@ merge-recursive.c: static struct diff_queue_struct *get_diffpairs(struct merge_o
      
      	ret = xmalloc(sizeof(*ret));
      	*ret = diff_queued_diff;
    +@@ merge-recursive.c: static int detect_and_process_renames(struct merge_options *opt,
    + 
    + 	if ((opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE) ||
    + 	    (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_CONFLICT &&
    +-	     !opt->call_depth)) {
    ++	     !opt->priv->call_depth)) {
    + 		dir_re_head = get_directory_renames(head_pairs);
    + 		dir_re_merge = get_directory_renames(merge_pairs);
    + 
     @@ merge-recursive.c: static int handle_content_merge(struct merge_file_info *mfi,
      		reason = _("add/add");
      
    @@ merge-recursive.c: static int merge_trees_internal(struct merge_options *opt,
      
      	unpack_trees_finish(opt);
      
    --	if (opt->call_depth) {
    -+	if (opt->priv->call_depth) {
    - 		struct object_id tree_id;
    - 		if (write_index_as_tree(&tree_id, opt->repo->index, NULL,
    - 					WRITE_TREE_FROM_MEMORY, NULL) ||
    +-	if (opt->call_depth &&
    ++	if (opt->priv->call_depth &&
    + 	    !(*result = write_inmemory_index_as_tree(opt->repo)))
    + 		return -1;
    + 
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      
      	for (iter = merge_bases; iter; iter = iter->next) {
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     +	if (!opt->priv->call_depth)
      		repo_read_index(opt->repo);
      
    - 	opt->ancestor = "merged common ancestors";
    + 	switch (num_merge_bases) {
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      		return clean;
      	}
18:  40161dc352 = 22:  9a381873c2 merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
 -:  ---------- > 23:  c6bc8a196f merge-recursive: add sanity checks for relevant merge_options
17:  f440ee1e64 = 24:  2123e9e4e4 merge-recursive: alphabetize include list
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 01/24] merge-recursive: be consistent with assert

[Elijah Newren]

In commit 8daec1df03de ("merge-recursive: switch from (oid,mode) pairs
to a diff_filespec", 2019-04-05), an assertion on a->path && b->path
was added for code readability to document that these both needed to be
non-NULL at this point in the code.  However, the subsequent lines also
read o->path, so it should be included in the assert.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 6b812d67e3..1d960fa64b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1043,7 +1043,7 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path);
+	assert(a->path && b->path && o->path);
 	if (strcmp(a->path, b->path) ||
 	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
 		base_name = opt->ancestor == NULL ? NULL :
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 02/24] checkout: provide better conflict hunk description with detached HEAD

[Elijah Newren]

When running 'git checkout -m' and using diff3 style conflict markers,
we want all the conflict hunks (left-side, "common" or "merge base", and
right-side) to have label markers letting the user know where each came
from.  The "common" hunk label (o.ancestor) came from
old_branch_info->name, but that is NULL when HEAD is detached, which
resulted in a blank label.  Check for that case and provide an
abbreviated commit hash instead.

(Incidentally, this was the only case in the git codebase where
merge_trees() was called with opt->ancestor being NULL.  A subsequent
commit will prevent similar problems by enforcing that merge_trees()
always be called with opt->ancestor != NULL.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6123f732a2..d5b946dc3a 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -713,6 +713,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			struct tree *old_tree;
 			struct merge_options o;
 			struct strbuf sb = STRBUF_INIT;
+			struct strbuf old_commit_shortname = STRBUF_INIT;
 
 			if (!opts->merge)
 				return 1;
@@ -768,6 +769,12 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			if (ret)
 				return ret;
 			o.ancestor = old_branch_info->name;
+			if (old_branch_info->name == NULL) {
+				strbuf_add_unique_abbrev(&old_commit_shortname,
+							 &old_branch_info->commit->object.oid,
+							 DEFAULT_ABBREV);
+				o.ancestor = old_commit_shortname.buf;
+			}
 			o.branch1 = new_branch_info->name;
 			o.branch2 = "local";
 			ret = merge_trees(&o,
@@ -781,6 +788,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 					 opts, 0,
 					 writeout_error);
 			strbuf_release(&o.obuf);
+			strbuf_release(&old_commit_shortname);
 			if (ret)
 				return ret;
 		}
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 03/24] merge-recursive: enforce opt->ancestor != NULL when calling merge_trees()

[Elijah Newren]

We always want our conflict hunks to be labelled so that users can know
where each came from.  The previous commit fixed the one caller in the
codebase which was not setting opt->ancestor (and thus not providing a
label for the "merge base" conflict hunk in diff3-style conflict
markers); add an assertion to prevent future codepaths from also
overlooking this requirement.

Enforcing this requirement also allows us to simplify the code for
labelling the conflict hunks by no longer checking if the ancestor label
is NULL.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1d960fa64b..a67ea4957a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1019,7 +1019,7 @@ static int merge_3way(struct merge_options *opt,
 {
 	mmfile_t orig, src1, src2;
 	struct ll_merge_options ll_opts = {0};
-	char *base_name, *name1, *name2;
+	char *base, *name1, *name2;
 	int merge_status;
 
 	ll_opts.renormalize = opt->renormalize;
@@ -1043,16 +1043,13 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path && o->path);
-	if (strcmp(a->path, b->path) ||
-	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s:%s", opt->ancestor, o->path);
+	assert(a->path && b->path && o->path && opt->ancestor);
+	if (strcmp(a->path, b->path) || strcmp(a->path, o->path) != 0) {
+		base  = mkpathdup("%s:%s", opt->ancestor, o->path);
 		name1 = mkpathdup("%s:%s", branch1, a->path);
 		name2 = mkpathdup("%s:%s", branch2, b->path);
 	} else {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s", opt->ancestor);
+		base  = mkpathdup("%s", opt->ancestor);
 		name1 = mkpathdup("%s", branch1);
 		name2 = mkpathdup("%s", branch2);
 	}
@@ -1061,11 +1058,11 @@ static int merge_3way(struct merge_options *opt,
 	read_mmblob(&src1, &a->oid);
 	read_mmblob(&src2, &b->oid);
 
-	merge_status = ll_merge(result_buf, a->path, &orig, base_name,
+	merge_status = ll_merge(result_buf, a->path, &orig, base,
 				&src1, name1, &src2, name2,
 				opt->repo->index, &ll_opts);
 
-	free(base_name);
+	free(base);
 	free(name1);
 	free(name2);
 	free(orig.ptr);
@@ -3390,6 +3387,8 @@ int merge_trees(struct merge_options *opt,
 	int code, clean;
 	struct strbuf sb = STRBUF_INIT;
 
+	assert(opt->ancestor != NULL);
+
 	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
 		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
 		    sb.buf);
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 04/24] merge-recursive: provide a better label for diff3 common ancestor

[Elijah Newren]

In commit 7ca56aa07619 ("merge-recursive: add a label for ancestor",
2010-03-20), a label was added for the '||||||' line to make it have
the more informative heading '|||||| merged common ancestors', with
the statement:

    It would be nicer to use a more informative label.  Perhaps someone
    will provide one some day.

This chosen label was perfectly reasonable when recursiveness kicks in,
i.e. when there are multiple merge bases.  (I can't think of a better
label in such cases.)  But it is actually somewhat misleading when there
is a unique merge base or no merge base.  Change this based on the
number of merge bases:
    >=2: "merged common ancestors"
    1:   <abbreviated commit hash>
    0:   "<empty tree>"

Tests have also been added to check that we get the right ancestor name
for each of the three cases.

Also, since merge_recursive() and merge_trees() have polar opposite
pre-conditions for opt->ancestor, document merge_recursive()'s
pre-condition with an assertion.  (An assertion was added to
merge_trees() already a few commits ago.)  The differences in
pre-conditions stem from two factors: (1) merge_trees() does not recurse
and thus does not have multiple sub-merges to worry about -- each of
which would require a different value for opt->ancestor, (2)
merge_trees() is only passed trees rather than commits and thus cannot
internally guess as good of a label.  Thus, while external callers of
merge_trees() are required to provide a non-NULL opt->ancestor,
merge_recursive() expects to set this value itself.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |  21 +++-
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 189 ++++++++++++++++++++++++++++++
 3 files changed, 214 insertions(+), 4 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index a67ea4957a..3a7a9514b9 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3507,6 +3507,11 @@ int merge_recursive(struct merge_options *opt,
 	struct commit *merged_common_ancestors;
 	struct tree *mrtree;
 	int clean;
+	int num_merge_bases;
+	struct strbuf merge_base_abbrev = STRBUF_INIT;
+
+	if (!opt->call_depth)
+		assert(opt->ancestor == NULL);
 
 	if (show(opt, 4)) {
 		output(opt, 4, _("Merging:"));
@@ -3528,6 +3533,7 @@ int merge_recursive(struct merge_options *opt,
 			output_commit_title(opt, iter->item);
 	}
 
+	num_merge_bases = commit_list_count(ca);
 	merged_common_ancestors = pop_commit(&ca);
 	if (merged_common_ancestors == NULL) {
 		/* if there is no common ancestor, use an empty tree */
@@ -3568,10 +3574,23 @@ int merge_recursive(struct merge_options *opt,
 	if (!opt->call_depth)
 		repo_read_index(opt->repo);
 
-	opt->ancestor = "merged common ancestors";
+	switch (num_merge_bases) {
+	case 0:
+		opt->ancestor = "<empty tree>";
+		break;
+	case 1:
+		strbuf_add_unique_abbrev(&merge_base_abbrev,
+					 &merged_common_ancestors->object.oid,
+					 DEFAULT_ABBREV);
+		opt->ancestor = merge_base_abbrev.buf;
+		break;
+	default:
+		opt->ancestor = "merged common ancestors";
+	}
 	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
 			    get_commit_tree(merged_common_ancestors),
 			    &mrtree);
+	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index d23b948f27..7fddcc8c73 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -1562,6 +1562,7 @@ test_expect_success 'check nested conflicts' '
 		cd nested_conflicts &&
 
 		git clean -f &&
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L2^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1582,7 +1583,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:a >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1594,7 +1595,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:b >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1732,6 +1733,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 	(
 		cd virtual_merge_base_has_nested_conflicts &&
 
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L3^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1760,7 +1762,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 		cp left merged-once &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			merged-once \
 			base        \
diff --git a/t/t6047-diff3-conflict-markers.sh b/t/t6047-diff3-conflict-markers.sh
new file mode 100755
index 0000000000..f69c8256bc
--- /dev/null
+++ b/t/t6047-diff3-conflict-markers.sh
@@ -0,0 +1,189 @@
+#!/bin/sh
+
+test_description='recursive merge diff3 style conflict markers'
+
+. ./test-lib.sh
+
+# Setup:
+#          L1
+#            \
+#             ?
+#            /
+#          R1
+#
+# Where:
+#   L1 and R1 both have a file named 'content' but have no common history
+#
+
+test_expect_success 'setup no merge base' '
+	test_create_repo no_merge_base &&
+	(
+		cd no_merge_base &&
+
+		git checkout -b L &&
+		test_commit A content A &&
+
+		git checkout --orphan R &&
+		test_commit B content B
+	)
+'
+
+test_expect_success 'check no merge base' '
+	(
+		cd no_merge_base &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge --allow-unrelated-histories -s recursive R^0 &&
+
+		grep "|||||| <empty tree>" content
+	)
+'
+
+# Setup:
+#          L1
+#         /  \
+#   master    ?
+#         \  /
+#          R1
+#
+# Where:
+#   L1 and R1 have modified the same file ('content') in conflicting ways
+#
+
+test_expect_success 'setup unique merge base' '
+	test_create_repo unique_merge_base &&
+	(
+		cd unique_merge_base &&
+
+		test_commit base content "1
+2
+3
+4
+5
+" &&
+
+		git branch L &&
+		git branch R &&
+
+		git checkout L &&
+		test_commit L content "1
+2
+3
+4
+5
+7" &&
+
+		git checkout R &&
+		git rm content &&
+		test_commit R renamed "1
+2
+3
+4
+5
+six"
+	)
+'
+
+test_expect_success 'check unique merge base' '
+	(
+		cd unique_merge_base &&
+
+		git checkout L^0 &&
+		MASTER=$(git rev-parse --short master) &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| $MASTER:content" renamed
+	)
+'
+
+# Setup:
+#          L1---L2--L3
+#         /  \ /      \
+#   master    X1       ?
+#         \  / \      /
+#          R1---R2--R3
+#
+# Where:
+#   commits L1 and R1 have modified the same file in non-conflicting ways
+#   X1 is an auto-generated merge-base used when merging L1 and R1
+#   commits L2 and R2 are merges of R1 and L1 into L1 and R1, respectively
+#   commits L3 and R3 both modify 'content' in conflicting ways
+#
+
+test_expect_success 'setup multiple merge bases' '
+	test_create_repo multiple_merge_bases &&
+	(
+		cd multiple_merge_bases &&
+
+		test_commit initial content "1
+2
+3
+4
+5" &&
+
+		git branch L &&
+		git branch R &&
+
+		# Create L1
+		git checkout L &&
+		test_commit L1 content "0
+1
+2
+3
+4
+5" &&
+
+		# Create R1
+		git checkout R &&
+		test_commit R1 content "1
+2
+3
+4
+5
+6" &&
+
+		# Create L2
+		git checkout L &&
+		git merge R1 &&
+
+		# Create R2
+		git checkout R &&
+		git merge L1 &&
+
+		# Create L3
+		git checkout L &&
+		test_commit L3 content "0
+1
+2
+3
+4
+5
+A" &&
+
+		# Create R3
+		git checkout R &&
+		git rm content &&
+		test_commit R3 renamed "0
+2
+3
+4
+5
+six"
+	)
+'
+
+test_expect_success 'check multiple merge bases' '
+	(
+		cd multiple_merge_bases &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| merged common ancestors:content" renamed
+	)
+'
+
+test_done
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 05/24] merge-recursive: introduce an enum for detect_directory_renames values

[Elijah Newren]

From: Derrick Stolee <dstolee@microsoft.com>

Improve code readability by introducing an enum to replace the
not-quite-boolean values taken on by detect_directory_renames.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/am.c      |  2 +-
 merge-recursive.c | 24 +++++++++++++++---------
 merge-recursive.h |  6 +++++-
 3 files changed, 21 insertions(+), 11 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 1aea657a7f..037e828efe 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1538,7 +1538,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	o.branch1 = "HEAD";
 	their_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
 	o.branch2 = their_tree_name;
-	o.detect_directory_renames = 0;
+	o.detect_directory_renames = MERGE_DIRECTORY_RENAMES_NONE;
 
 	if (state->quiet)
 		o.verbosity = 0;
diff --git a/merge-recursive.c b/merge-recursive.c
index 3a7a9514b9..dd2ee5edee 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1375,7 +1375,8 @@ static int handle_rename_via_dir(struct merge_options *opt,
 	const struct rename *ren = ci->ren1;
 	const struct diff_filespec *dest = ren->pair->two;
 	char *file_path = dest->path;
-	int mark_conflicted = (opt->detect_directory_renames == 1);
+	int mark_conflicted = (opt->detect_directory_renames ==
+			       MERGE_DIRECTORY_RENAMES_CONFLICT);
 	assert(ren->dir_rename_original_dest);
 
 	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
@@ -2860,8 +2861,9 @@ static int detect_and_process_renames(struct merge_options *opt,
 	head_pairs = get_diffpairs(opt, common, head);
 	merge_pairs = get_diffpairs(opt, common, merge);
 
-	if ((opt->detect_directory_renames == 2) ||
-	    (opt->detect_directory_renames == 1 && !opt->call_depth)) {
+	if ((opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE) ||
+	    (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_CONFLICT &&
+	     !opt->call_depth)) {
 		dir_re_head = get_directory_renames(head_pairs);
 		dir_re_merge = get_directory_renames(merge_pairs);
 
@@ -3119,7 +3121,8 @@ static int handle_rename_normal(struct merge_options *opt,
 	clean = handle_content_merge(&mfi, opt, path, was_dirty(opt, path),
 				     o, a, b, ci);
 
-	if (clean && opt->detect_directory_renames == 1 &&
+	if (clean &&
+	    opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_CONFLICT &&
 	    ren->dir_rename_original_dest) {
 		if (update_stages(opt, path,
 				  NULL,
@@ -3164,12 +3167,12 @@ static int warn_about_dir_renamed_entries(struct merge_options *opt,
 		return clean;
 
 	/* Sanity checks */
-	assert(opt->detect_directory_renames > 0);
+	assert(opt->detect_directory_renames > MERGE_DIRECTORY_RENAMES_NONE);
 	assert(ren->dir_rename_original_type == 'A' ||
 	       ren->dir_rename_original_type == 'R');
 
 	/* Check whether to treat directory renames as a conflict */
-	clean = (opt->detect_directory_renames == 2);
+	clean = (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE);
 
 	is_add = (ren->dir_rename_original_type == 'A');
 	if (ren->dir_rename_original_type == 'A' && clean) {
@@ -3684,9 +3687,12 @@ static void merge_recursive_config(struct merge_options *opt)
 	if (!git_config_get_string("merge.directoryrenames", &value)) {
 		int boolval = git_parse_maybe_bool(value);
 		if (0 <= boolval) {
-			opt->detect_directory_renames = boolval ? 2 : 0;
+			opt->detect_directory_renames = boolval ?
+				MERGE_DIRECTORY_RENAMES_TRUE :
+				MERGE_DIRECTORY_RENAMES_NONE;
 		} else if (!strcasecmp(value, "conflict")) {
-			opt->detect_directory_renames = 1;
+			opt->detect_directory_renames =
+				MERGE_DIRECTORY_RENAMES_CONFLICT;
 		} /* avoid erroring on values from future versions of git */
 		free(value);
 	}
@@ -3706,7 +3712,7 @@ void init_merge_options(struct merge_options *opt,
 	opt->renormalize = 0;
 	opt->diff_detect_rename = -1;
 	opt->merge_detect_rename = -1;
-	opt->detect_directory_renames = 1;
+	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
diff --git a/merge-recursive.h b/merge-recursive.h
index c2b7bb65c6..f1b6ef38ae 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -22,7 +22,11 @@ struct merge_options {
 	unsigned renormalize : 1;
 	long xdl_opts;
 	int verbosity;
-	int detect_directory_renames;
+	enum {
+		MERGE_DIRECTORY_RENAMES_NONE = 0,
+		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
+		MERGE_DIRECTORY_RENAMES_TRUE = 2
+	} detect_directory_renames;
 	int diff_detect_rename;
 	int merge_detect_rename;
 	int diff_rename_limit;
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 06/24] merge-recursive: future-proof update_file_flags() against memory leaks

[Elijah Newren]

There is a 'free_buf' label to which all but one of the error paths in
update_file_flags() jump; that error case involves a NULL buf and is
thus not a memory leak.  However, make that error case execute the same
deallocation code anyway so that if anyone adds any additional memory
allocations or deallocations, then all error paths correctly deallocate
resources.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index dd2ee5edee..eae3e4fbcc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -934,9 +934,11 @@ static int update_file_flags(struct merge_options *opt,
 		}
 
 		buf = read_object_file(&contents->oid, &type, &size);
-		if (!buf)
-			return err(opt, _("cannot read object %s '%s'"),
-				   oid_to_hex(&contents->oid), path);
+		if (!buf) {
+			ret = err(opt, _("cannot read object %s '%s'"),
+				  oid_to_hex(&contents->oid), path);
+			goto free_buf;
+		}
 		if (type != OBJ_BLOB) {
 			ret = err(opt, _("blob expected for %s '%s'"),
 				  oid_to_hex(&contents->oid), path);
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 07/24] merge-recursive: remove another implicit dependency on the_repository

[Elijah Newren]

Commit d7cf3a96e9a0 ("merge-recursive.c: remove implicit dependency on
the_repository", 2019-01-12) and follow-ups like commit 34e7771bc644
("Use the right 'struct repository' instead of the_repository",
2019-06-27), removed most implicit uses of the_repository.  Convert
calls to get_commit_tree() to instead use repo_get_commit_tree() to get
rid of another.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index eae3e4fbcc..fb668fc84c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3592,8 +3592,11 @@ int merge_recursive(struct merge_options *opt,
 	default:
 		opt->ancestor = "merged common ancestors";
 	}
-	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
-			    get_commit_tree(merged_common_ancestors),
+	clean = merge_trees(opt,
+			    repo_get_commit_tree(opt->repo, h1),
+			    repo_get_commit_tree(opt->repo, h2),
+			    repo_get_commit_tree(opt->repo,
+						 merged_common_ancestors),
 			    &mrtree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 08/24] Ensure index matches head before invoking merge machinery, round N

[Elijah Newren]

This is the bug that just won't die; there always seems to be another
form of it somewhere.  See the commit message of 55f39cf7551b ("merge:
fix misleading pre-merge check documentation", 2018-06-30) for a more
detailed explanation), but in short:

<quick summary>

builtin/merge.c contains this important requirement for merge
strategies:

    ...the index must be in sync with the head commit.  The strategies are
    responsible to ensure this.

This condition is important to enforce because there are two likely
failure cases when the index isn't in sync with the head commit:

  * we silently throw away changes the user had staged before the merge

  * we accidentally (and silently) include changes in the merge that
    were not part of either of the branches/trees being merged

Discarding users' work and mis-merging are both bad outcomes, especially
when done silently, so naturally this rule was stated sternly -- but,
unfortunately totally ignored in practice unless and until actual bugs
were found.  But, fear not: the bugs from this were fixed in commit
  ee6566e8d70d ("[PATCH] Rewrite read-tree", 2005-09-05)
through a rewrite of read-tree (again, commit 55f39cf7551b has a more
detailed explanation of how this affected merge).  And it was fixed
again in commit
  160252f81626 ("git-merge-ours: make sure our index matches HEAD", 2005-11-03)
...and it was fixed again in commit
  3ec62ad9ffba ("merge-octopus: abort if index does not match HEAD", 2016-04-09)
...and again in commit
  65170c07d466 ("merge-recursive: avoid incorporating uncommitted changes in a merge", 2017-12-21)
...and again in commit
  eddd1a411d93 ("merge-recursive: enforce rule that index matches head before merging", 2018-06-30)

...with multiple testcases added to the testsuite that could be
enumerated in even more commits.

Then, finally, in a patch in the same series as the last fix above, the
documentation about this requirement was fixed in commit 55f39cf7551b
("merge: fix misleading pre-merge check documentation", 2018-06-30), and
we all lived happily ever after...

</quick summary>

Unfortunately, "ever after" apparently denotes a limited time and it
expired today.  The merge-recursive rule to enforce that index matches
head was at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially HUGE amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Trying to enforce that all of merge_trees(), merge_recursive(), and
merge_recursive_generic() checked the index == head condition earlier
resulted in a bunch of broken tests.  It turns out that
merge_recursive() has code to drop and reload the cache while recursing
to create intermediate virtual merge bases, but unfortunately that code
runs even when no recursion is necessary.  This unconditional dropping
and reloading of the cache masked a few bugs:

  * builtin/merge-recursive.c: didn't even bother loading the index.

  * builtin/stash.c: feels like a fake 'builtin' because it repeatedly
    invokes git subprocesses all over the place, mixed with other
    operations.  In particular, invoking "git reset" will reset the
    index on disk, but the parent process that invoked it won't
    automatically have its in-memory index updated.

  * t3030-merge-recursive.h: this test has always been broken in that it
    didn't make sure to make index match head before running.  But, it
    didn't care about the index or even the merge result, just the
    verbose output while running.  While commit eddd1a411d93
    ("merge-recursive: enforce rule that index matches head before
    merging", 2018-06-30) should have uncovered this broken test, it
    used a test_must_fail wrapper around the merge-recursive call
    because it was known that the merge resulted in a rename/rename
    conflict.  Thus, that fix only made this test fail for a different
    reason, and since the index == head check didn't happen until after
    coming all the way back out of the recursion, the testcase had
    enough information to pass the one check that it did perform.

So, load the index in builtin/merge-recursive.c, reload the in-memory
index in builtin/stash.c, and modify the t3030 testcase to correctly
setup the index and make sure that the test fails in the expected way
(meaning it reports a rename/rename conflict).  This makes sure that
all callers actually make the index match head.  The next commit will
then enforce the condition that index matches head earlier so this
problem doesn't return in the future.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-recursive.c  | 4 ++++
 builtin/stash.c            | 2 ++
 t/t3030-merge-recursive.sh | 9 ++++++++-
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
index 5b910e351e..a4bfd8fc51 100644
--- a/builtin/merge-recursive.c
+++ b/builtin/merge-recursive.c
@@ -1,3 +1,4 @@
+#include "cache.h"
 #include "builtin.h"
 #include "commit.h"
 #include "tag.h"
@@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
 	if (argc - i != 3) /* "--" "<head>" "<remote>" */
 		die(_("not handling anything other than two heads merge."));
 
+	if (repo_read_index_unmerged(the_repository))
+		die_resolve_conflict("merge");
+
 	o.branch1 = argv[++i];
 	o.branch2 = argv[++i];
 
diff --git a/builtin/stash.c b/builtin/stash.c
index b5a301f24d..4aa47785f9 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -427,6 +427,8 @@ static int do_apply_stash(const char *prefix, struct stash_info *info,
 				return error(_("could not save index tree"));
 
 			reset_head();
+			discard_cache();
+			read_cache();
 		}
 	}
 
diff --git a/t/t3030-merge-recursive.sh b/t/t3030-merge-recursive.sh
index ff641b348a..a37bcc58a0 100755
--- a/t/t3030-merge-recursive.sh
+++ b/t/t3030-merge-recursive.sh
@@ -667,15 +667,22 @@ test_expect_success 'merging with triple rename across D/F conflict' '
 test_expect_success 'merge-recursive remembers the names of all base trees' '
 	git reset --hard HEAD &&
 
+	# make the index match $c1 so that merge-recursive below does not
+	# fail early
+	git diff --binary HEAD $c1 -- | git apply --cached &&
+
 	# more trees than static slots used by oid_to_hex()
 	for commit in $c0 $c2 $c4 $c5 $c6 $c7
 	do
 		git rev-parse "$commit^{tree}"
 	done >trees &&
 
-	# ignore the return code -- it only fails because the input is weird
+	# ignore the return code; it only fails because the input is weird...
 	test_must_fail git -c merge.verbosity=5 merge-recursive $(cat trees) -- $c1 $c3 >out &&
 
+	# ...but make sure it fails in the expected way
+	test_i18ngrep CONFLICT.*rename/rename out &&
+
 	# merge-recursive prints in reverse order, but we do not care
 	sort <trees >expect &&
 	sed -n "s/^virtual //p" out | sort >actual &&
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 09/24] merge-recursive: exit early if index != head

[Elijah Newren]

We had a rule to enforce that the index matches head, but it was found
at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially huge amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Further, not enforcing this requirement earlier allowed other bugs (such
as an unintentional unconditional dropping and reloading of the index in
merge_recursive() even when no recursion was necessary), to mask bugs in
other callers (which were fixed in the commit prior to this one).

Make sure we do the index == head check at the beginning of the merge,
and error out immediately if it fails.  While we're at it, fix a small
leak in the show-the-error codepath.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 101 +++++++++++++++++++++++++++++++++-------------
 1 file changed, 72 insertions(+), 29 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index fb668fc84c..ded2e1bcfb 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3382,23 +3382,14 @@ static int process_entry(struct merge_options *opt,
 	return clean_merge;
 }
 
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *common,
-		struct tree **result)
+static int merge_trees_internal(struct merge_options *opt,
+				struct tree *head,
+				struct tree *merge,
+				struct tree *common,
+				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
-	struct strbuf sb = STRBUF_INIT;
-
-	assert(opt->ancestor != NULL);
-
-	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
-		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
-		    sb.buf);
-		return -1;
-	}
 
 	if (opt->subtree_shift) {
 		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
@@ -3502,11 +3493,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
  * Merge the commits h1 and h2, return the resulting virtual
  * commit object and a flag indicating the cleanness of the merge.
  */
-int merge_recursive(struct merge_options *opt,
-		    struct commit *h1,
-		    struct commit *h2,
-		    struct commit_list *ca,
-		    struct commit **result)
+static int merge_recursive_internal(struct merge_options *opt,
+				    struct commit *h1,
+				    struct commit *h2,
+				    struct commit_list *ca,
+				    struct commit **result)
 {
 	struct commit_list *iter;
 	struct commit *merged_common_ancestors;
@@ -3515,9 +3506,6 @@ int merge_recursive(struct merge_options *opt,
 	int num_merge_bases;
 	struct strbuf merge_base_abbrev = STRBUF_INIT;
 
-	if (!opt->call_depth)
-		assert(opt->ancestor == NULL);
-
 	if (show(opt, 4)) {
 		output(opt, 4, _("Merging:"));
 		output_commit_title(opt, h1);
@@ -3564,7 +3552,7 @@ int merge_recursive(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive(opt, merged_common_ancestors, iter->item,
+		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
 				    NULL, &merged_common_ancestors) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
@@ -3592,12 +3580,12 @@ int merge_recursive(struct merge_options *opt,
 	default:
 		opt->ancestor = "merged common ancestors";
 	}
-	clean = merge_trees(opt,
-			    repo_get_commit_tree(opt->repo, h1),
-			    repo_get_commit_tree(opt->repo, h2),
-			    repo_get_commit_tree(opt->repo,
-						 merged_common_ancestors),
-			    &mrtree);
+	clean = merge_trees_internal(opt,
+				     repo_get_commit_tree(opt->repo, h1),
+				     repo_get_commit_tree(opt->repo, h2),
+				     repo_get_commit_tree(opt->repo,
+							  merged_common_ancestors),
+				     &mrtree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
 		flush_output(opt);
@@ -3618,6 +3606,61 @@ int merge_recursive(struct merge_options *opt,
 	return clean;
 }
 
+static int merge_start(struct merge_options *opt, struct tree *head)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	if (repo_index_has_changes(opt->repo, head, &sb)) {
+		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
+		    sb.buf);
+		strbuf_release(&sb);
+		return -1;
+	}
+
+	return 0;
+}
+
+static void merge_finalize(struct merge_options *opt)
+{
+	/* Common code for wrapping up merges will be added here later */
+}
+
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *common,
+		struct tree **result)
+{
+	int clean;
+
+	assert(opt->ancestor != NULL);
+
+	if (merge_start(opt, head))
+		return -1;
+	clean = merge_trees_internal(opt, head, merge, common, result);
+	merge_finalize(opt);
+
+	return clean;
+}
+
+int merge_recursive(struct merge_options *opt,
+		    struct commit *h1,
+		    struct commit *h2,
+		    struct commit_list *ca,
+		    struct commit **result)
+{
+	int clean;
+
+	assert(opt->ancestor == NULL);
+
+	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
+		return -1;
+	clean = merge_recursive_internal(opt, h1, h2, ca, result);
+	merge_finalize(opt);
+
+	return clean;
+}
+
 static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
 			      const char *name)
 {
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 10/24] merge-recursive: remove useless parameter in merge_trees()

[Elijah Newren]

merge_trees() took a results parameter that would only be written when
opt->call_depth was positive, which is never the case now that
merge_trees_internal() has been split from merge_trees().  Remove the
misleading and unused parameter from merge_trees().

While at it, add some comments explaining how the output of
merge_trees() and merge_recursive() differ.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c |  4 +---
 merge-recursive.c  |  6 +++---
 merge-recursive.h  | 20 ++++++++++++++++----
 sequencer.c        |  4 ++--
 4 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index d5b946dc3a..90e0eaf25e 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -708,7 +708,6 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 * give up or do a real merge, depending on
 			 * whether the merge flag was used.
 			 */
-			struct tree *result;
 			struct tree *work;
 			struct tree *old_tree;
 			struct merge_options o;
@@ -780,8 +779,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			ret = merge_trees(&o,
 					  new_tree,
 					  work,
-					  old_tree,
-					  &result);
+					  old_tree);
 			if (ret < 0)
 				exit(128);
 			ret = reset_tree(new_tree,
diff --git a/merge-recursive.c b/merge-recursive.c
index ded2e1bcfb..89e9a7518c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3628,16 +3628,16 @@ static void merge_finalize(struct merge_options *opt)
 int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
-		struct tree **result)
+		struct tree *common)
 {
 	int clean;
+	struct tree *ignored;
 
 	assert(opt->ancestor != NULL);
 
 	if (merge_start(opt, head))
 		return -1;
-	clean = merge_trees_internal(opt, head, merge, common, result);
+	clean = merge_trees_internal(opt, head, merge, common, &ignored);
 	merge_finalize(opt);
 
 	return clean;
diff --git a/merge-recursive.h b/merge-recursive.h
index f1b6ef38ae..18012fff9d 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -74,19 +74,31 @@ static inline int merge_detect_rename(struct merge_options *o)
 		o->diff_detect_rename >= 0 ? o->diff_detect_rename : 1;
 }
 
-/* merge_trees() but with recursive ancestor consolidation */
+/*
+ * merge_recursive is like merge_trees() but with recursive ancestor
+ * consolidation, and when successful, it creates an actual commit
+ * and writes its address to *result.
+ *
+ * NOTE: empirically, about a decade ago it was determined that with more
+ *       than two merge bases, optimal behavior was found when the
+ *       ancestors were passed in the order of oldest merge base to newest
+ *       one.  Also, ancestors will be consumed (emptied) so make a copy if
+ *       you need it.
+ */
 int merge_recursive(struct merge_options *o,
 		    struct commit *h1,
 		    struct commit *h2,
 		    struct commit_list *ancestors,
 		    struct commit **result);
 
-/* rename-detecting three-way merge, no recursion */
+/*
+ * rename-detecting three-way merge, no recursion; result of merge is written
+ * to opt->repo->index.
+ */
 int merge_trees(struct merge_options *o,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
-		struct tree **result);
+		struct tree *common);
 
 /*
  * "git-merge-recursive" can be fed trees; wrap them into
diff --git a/sequencer.c b/sequencer.c
index 34ebf8ed94..c4ed30f1b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -586,7 +586,7 @@ static int do_recursive_merge(struct repository *r,
 			      struct replay_opts *opts)
 {
 	struct merge_options o;
-	struct tree *result, *next_tree, *base_tree, *head_tree;
+	struct tree *next_tree, *base_tree, *head_tree;
 	int clean;
 	char **xopt;
 	struct lock_file index_lock = LOCK_INIT;
@@ -613,7 +613,7 @@ static int do_recursive_merge(struct repository *r,
 
 	clean = merge_trees(&o,
 			    head_tree,
-			    next_tree, base_tree, &result);
+			    next_tree, base_tree);
 	if (is_rebase_i(opts) && clean <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 11/24] merge-recursive: don't force external callers to do our logging

[Elijah Newren]

Alternatively, you can view this as "make the merge functions behave
more similarly."  merge-recursive has three different entry points:
merge_trees(), merge_recursive(), and merge_recursive_generic().  Two of
these would call diff_warn_rename_limit(), but merge_trees() didn't.
This lead to callers of merge_trees() needing to manually call
diff_warn_rename_limit() themselves.  Move this to the new
merge_finalize() function to make sure that all three entry points run
this function.

Note that there are two external callers of merge_trees(), one in
sequencer.c and one in builtin/checkout.c.  The one in sequencer.c is
cleaned up by this patch and just transfers where the call to
diff_warn_rename_limit() is made; the one in builtin/checkout.c is for
switching to a different commit and in the very rare case where the
warning might be triggered, it would probably be helpful to include
(e.g. if someone is modifying a file that has been renamed in moving to
the other commit, but there are so many renames between the commits that
the limit kicks in and none are detected, it may help to have an
explanation about why they got a delete/modify conflict instead of a
proper content merge in a renamed file).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++----
 sequencer.c       | 1 -
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 89e9a7518c..6cee867d37 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3600,9 +3600,6 @@ static int merge_recursive_internal(struct merge_options *opt,
 	flush_output(opt);
 	if (!opt->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
-	if (show(opt, 2))
-		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
 	return clean;
 }
 
@@ -3622,7 +3619,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 
 static void merge_finalize(struct merge_options *opt)
 {
-	/* Common code for wrapping up merges will be added here later */
+	if (show(opt, 2))
+		diff_warn_rename_limit("merge.renamelimit",
+				       opt->needed_rename_limit, 0);
 }
 
 int merge_trees(struct merge_options *opt,
diff --git a/sequencer.c b/sequencer.c
index c4ed30f1b4..094a4dd03d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -617,7 +617,6 @@ static int do_recursive_merge(struct repository *r,
 	if (is_rebase_i(opts) && clean <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
-	diff_warn_rename_limit("merge.renamelimit", o.needed_rename_limit, 0);
 	if (clean < 0) {
 		rollback_lock_file(&index_lock);
 		return clean;
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 12/24] cache-tree: share code between functions writing an index as a tree

[Elijah Newren]

write_tree_from_memory() appeared to be a merge-recursive special that
basically duplicated write_index_as_tree().  The two have a different
signature, but the bigger difference was just that write_index_as_tree()
would always unconditionally read the index off of disk instead of
working on the current in-memory index.  So:

  * split out common code into write_index_as_tree_internal()

  * rename write_tree_from_memory() to write_inmemory_index_as_tree(),
    make it call write_index_as_tree_internal(), and move it to
    cache-tree.c

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c |  2 +-
 cache-tree.c       | 81 +++++++++++++++++++++++++++++++++-------------
 cache-tree.h       |  3 +-
 merge-recursive.c  | 34 ++-----------------
 merge-recursive.h  |  1 -
 5 files changed, 63 insertions(+), 58 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 90e0eaf25e..19a3a0bae5 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -760,7 +760,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 */
 			init_merge_options(&o, the_repository);
 			o.verbosity = 0;
-			work = write_tree_from_memory(&o);
+			work = write_inmemory_index_as_tree(the_repository);
 
 			ret = reset_tree(new_tree,
 					 opts, 1,
diff --git a/cache-tree.c b/cache-tree.c
index 706ffcf188..00eda3e537 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -608,11 +608,62 @@ static struct cache_tree *cache_tree_find(struct cache_tree *it, const char *pat
 	return it;
 }
 
+int write_index_as_tree_internal(struct object_id *oid, struct index_state *index_state, int cache_tree_valid, int flags, const char *prefix)
+{
+	if (flags & WRITE_TREE_IGNORE_CACHE_TREE) {
+		cache_tree_free(&index_state->cache_tree);
+		cache_tree_valid = 0;
+	}
+
+	if (!index_state->cache_tree)
+		index_state->cache_tree = cache_tree();
+
+	if (!cache_tree_valid && cache_tree_update(index_state, flags) < 0)
+		return WRITE_TREE_UNMERGED_INDEX;
+
+	if (prefix) {
+		struct cache_tree *subtree;
+		subtree = cache_tree_find(index_state->cache_tree, prefix);
+		if (!subtree)
+			return WRITE_TREE_PREFIX_ERROR;
+		oidcpy(oid, &subtree->oid);
+	}
+	else
+		oidcpy(oid, &index_state->cache_tree->oid);
+
+	return 0;
+}
+
+struct tree* write_inmemory_index_as_tree(struct repository *repo) {
+	struct object_id o;
+	int was_valid, ret;
+
+	struct index_state *index_state	= repo->index;
+	was_valid = index_state->cache_tree &&
+		    cache_tree_fully_valid(index_state->cache_tree);
+
+	ret = write_index_as_tree_internal(&o, index_state, was_valid, 0, NULL);
+	if (ret == WRITE_TREE_UNMERGED_INDEX) {
+		int i;
+		fprintf(stderr, "BUG: There are unmerged index entries:\n");
+		for (i = 0; i < index_state->cache_nr; i++) {
+			const struct cache_entry *ce = index_state->cache[i];
+			if (ce_stage(ce))
+				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
+					(int)ce_namelen(ce), ce->name);
+		}
+		BUG("unmerged index entries when writing inmemory index");
+	}
+
+	return lookup_tree(repo, &index_state->cache_tree->oid);
+}
+
+
 int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix)
 {
 	int entries, was_valid;
 	struct lock_file lock_file = LOCK_INIT;
-	int ret = 0;
+	int ret;
 
 	hold_lock_file_for_update(&lock_file, index_path, LOCK_DIE_ON_ERROR);
 
@@ -621,18 +672,14 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 		ret = WRITE_TREE_UNREADABLE_INDEX;
 		goto out;
 	}
-	if (flags & WRITE_TREE_IGNORE_CACHE_TREE)
-		cache_tree_free(&index_state->cache_tree);
 
-	if (!index_state->cache_tree)
-		index_state->cache_tree = cache_tree();
+	was_valid = !(flags & WRITE_TREE_IGNORE_CACHE_TREE) &&
+		    index_state->cache_tree &&
+		    cache_tree_fully_valid(index_state->cache_tree);
 
-	was_valid = cache_tree_fully_valid(index_state->cache_tree);
-	if (!was_valid) {
-		if (cache_tree_update(index_state, flags) < 0) {
-			ret = WRITE_TREE_UNMERGED_INDEX;
-			goto out;
-		}
+	ret = write_index_as_tree_internal(oid, index_state, was_valid, flags,
+					   prefix);
+	if (!ret && !was_valid) {
 		write_locked_index(index_state, &lock_file, COMMIT_LOCK);
 		/* Not being able to write is fine -- we are only interested
 		 * in updating the cache-tree part, and if the next caller
@@ -642,18 +689,6 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 		 */
 	}
 
-	if (prefix) {
-		struct cache_tree *subtree;
-		subtree = cache_tree_find(index_state->cache_tree, prefix);
-		if (!subtree) {
-			ret = WRITE_TREE_PREFIX_ERROR;
-			goto out;
-		}
-		oidcpy(oid, &subtree->oid);
-	}
-	else
-		oidcpy(oid, &index_state->cache_tree->oid);
-
 out:
 	rollback_lock_file(&lock_file);
 	return ret;
diff --git a/cache-tree.h b/cache-tree.h
index 757bbc48bc..8ce1632bc6 100644
--- a/cache-tree.h
+++ b/cache-tree.h
@@ -34,7 +34,7 @@ int cache_tree_fully_valid(struct cache_tree *);
 int cache_tree_update(struct index_state *, int);
 void cache_tree_verify(struct repository *, struct index_state *);
 
-/* bitmasks to write_cache_as_tree flags */
+/* bitmasks to write_index_as_tree flags */
 #define WRITE_TREE_MISSING_OK 1
 #define WRITE_TREE_IGNORE_CACHE_TREE 2
 #define WRITE_TREE_DRY_RUN 4
@@ -46,6 +46,7 @@ void cache_tree_verify(struct repository *, struct index_state *);
 #define WRITE_TREE_UNMERGED_INDEX (-2)
 #define WRITE_TREE_PREFIX_ERROR (-3)
 
+struct tree* write_inmemory_index_as_tree(struct repository *repo);
 int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix);
 void prime_cache_tree(struct repository *, struct index_state *, struct tree *);
 
diff --git a/merge-recursive.c b/merge-recursive.c
index 6cee867d37..1fbae52402 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -412,37 +412,6 @@ static void unpack_trees_finish(struct merge_options *opt)
 	clear_unpack_trees_porcelain(&opt->unpack_opts);
 }
 
-struct tree *write_tree_from_memory(struct merge_options *opt)
-{
-	struct tree *result = NULL;
-	struct index_state *istate = opt->repo->index;
-
-	if (unmerged_index(istate)) {
-		int i;
-		fprintf(stderr, "BUG: There are unmerged index entries:\n");
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
-			if (ce_stage(ce))
-				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
-					(int)ce_namelen(ce), ce->name);
-		}
-		BUG("unmerged index entries in merge-recursive.c");
-	}
-
-	if (!istate->cache_tree)
-		istate->cache_tree = cache_tree();
-
-	if (!cache_tree_fully_valid(istate->cache_tree) &&
-	    cache_tree_update(istate, 0) < 0) {
-		err(opt, _("error building trees"));
-		return NULL;
-	}
-
-	result = lookup_tree(opt->repo, &istate->cache_tree->oid);
-
-	return result;
-}
-
 static int save_files_dirs(const struct object_id *oid,
 			   struct strbuf *base, const char *path,
 			   unsigned int mode, int stage, void *context)
@@ -3472,7 +3441,8 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth && !(*result = write_tree_from_memory(opt)))
+	if (opt->call_depth &&
+	    !(*result = write_inmemory_index_as_tree(opt->repo)))
 		return -1;
 
 	return clean;
diff --git a/merge-recursive.h b/merge-recursive.h
index 18012fff9d..0a3033bdb0 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -113,7 +113,6 @@ int merge_recursive_generic(struct merge_options *o,
 
 void init_merge_options(struct merge_options *o,
 			struct repository *repo);
-struct tree *write_tree_from_memory(struct merge_options *o);
 
 int parse_merge_opt(struct merge_options *out, const char *s);
 
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 13/24] merge-recursive: fix some overly long lines

[Elijah Newren]

No substantive code change, just add some line breaks to fix lines that
have grown in length due to various refactorings.  Most remaining lines
of excessive length in merge-recursive include error messages and it's
not clear that splitting those improves things.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1fbae52402..40f2c6ad55 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -681,7 +681,9 @@ static void add_flattened_path(struct strbuf *out, const char *s)
 			out->buf[i] = '_';
 }
 
-static char *unique_path(struct merge_options *opt, const char *path, const char *branch)
+static char *unique_path(struct merge_options *opt,
+			 const char *path,
+			 const char *branch)
 {
 	struct path_hashmap_entry *entry;
 	struct strbuf newpath = STRBUF_INIT;
@@ -915,7 +917,8 @@ static int update_file_flags(struct merge_options *opt,
 		}
 		if (S_ISREG(contents->mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
-			if (convert_to_working_tree(opt->repo->index, path, buf, size, &strbuf)) {
+			if (convert_to_working_tree(opt->repo->index,
+						    path, buf, size, &strbuf)) {
 				free(buf);
 				size = strbuf.len;
 				buf = strbuf_detach(&strbuf, NULL);
@@ -3393,7 +3396,8 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp, NULL, 512);
+		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
 
@@ -3503,7 +3507,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo, tree, "ancestor");
+		merged_common_ancestors = make_virtual_commit(opt->repo,
+							      tree, "ancestor");
 	}
 
 	for (iter = ca; iter; iter = iter->next) {
@@ -3630,7 +3635,8 @@ int merge_recursive(struct merge_options *opt,
 	return clean;
 }
 
-static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
+static struct commit *get_ref(struct repository *repo,
+			      const struct object_id *oid,
 			      const char *name)
 {
 	struct object *object;
@@ -3665,7 +3671,8 @@ int merge_recursive_generic(struct merge_options *opt,
 		int i;
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i], oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, base_list[i],
+					     oid_to_hex(base_list[i]))))
 				return err(opt, _("Could not parse object '%s'"),
 					   oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 14/24] merge-recursive: use common name for ancestors/common/base_list

[Elijah Newren]

merge_trees(), merge_recursive(), and merge_recursive_generic() in
their function headers used four different names for the merge base or
list of merge bases they were passed:
  * 'common'
  * 'ancestors'
  * 'ca'
  * 'base_list'
They were able to refer to it four different ways instead of only three
by using a different name in the signature for the .c file than the .h
file.  Change all of these to 'merge_base' or 'merge_bases'.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 72 ++++++++++++++++++++++++-----------------------
 merge-recursive.h | 14 ++++-----
 2 files changed, 44 insertions(+), 42 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 40f2c6ad55..d0153ba971 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3357,24 +3357,26 @@ static int process_entry(struct merge_options *opt,
 static int merge_trees_internal(struct merge_options *opt,
 				struct tree *head,
 				struct tree *merge,
-				struct tree *common,
+				struct tree *merge_base,
 				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
 
 	if (opt->subtree_shift) {
-		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
-		common = shift_tree_object(opt->repo, head, common, opt->subtree_shift);
+		merge = shift_tree_object(opt->repo, head, merge,
+					  opt->subtree_shift);
+		merge_base = shift_tree_object(opt->repo, head, merge_base,
+					       opt->subtree_shift);
 	}
 
-	if (oid_eq(&common->object.oid, &merge->object.oid)) {
+	if (oid_eq(&merge_base->object.oid, &merge->object.oid)) {
 		output(opt, 0, _("Already up to date!"));
 		*result = head;
 		return 1;
 	}
 
-	code = unpack_trees_start(opt, common, head, merge);
+	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
 		if (show(opt, 4) || opt->call_depth)
@@ -3402,7 +3404,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		get_files_dirs(opt, merge);
 
 		entries = get_unmerged(opt->repo->index);
-		clean = detect_and_process_renames(opt, common, head, merge,
+		clean = detect_and_process_renames(opt, merge_base, head, merge,
 						   entries, &re_info);
 		record_df_conflict_files(opt, entries);
 		if (clean < 0)
@@ -3470,11 +3472,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
 static int merge_recursive_internal(struct merge_options *opt,
 				    struct commit *h1,
 				    struct commit *h2,
-				    struct commit_list *ca,
+				    struct commit_list *merge_bases,
 				    struct commit **result)
 {
 	struct commit_list *iter;
-	struct commit *merged_common_ancestors;
+	struct commit *merged_merge_bases;
 	struct tree *mrtree;
 	int clean;
 	int num_merge_bases;
@@ -3486,32 +3488,32 @@ static int merge_recursive_internal(struct merge_options *opt,
 		output_commit_title(opt, h2);
 	}
 
-	if (!ca) {
-		ca = get_merge_bases(h1, h2);
-		ca = reverse_commit_list(ca);
+	if (!merge_bases) {
+		merge_bases = get_merge_bases(h1, h2);
+		merge_bases = reverse_commit_list(merge_bases);
 	}
 
 	if (show(opt, 5)) {
-		unsigned cnt = commit_list_count(ca);
+		unsigned cnt = commit_list_count(merge_bases);
 
 		output(opt, 5, Q_("found %u common ancestor:",
 				"found %u common ancestors:", cnt), cnt);
-		for (iter = ca; iter; iter = iter->next)
+		for (iter = merge_bases; iter; iter = iter->next)
 			output_commit_title(opt, iter->item);
 	}
 
-	num_merge_bases = commit_list_count(ca);
-	merged_common_ancestors = pop_commit(&ca);
-	if (merged_common_ancestors == NULL) {
+	num_merge_bases = commit_list_count(merge_bases);
+	merged_merge_bases = pop_commit(&merge_bases);
+	if (merged_merge_bases == NULL) {
 		/* if there is no common ancestor, use an empty tree */
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo,
-							      tree, "ancestor");
+		merged_merge_bases = make_virtual_commit(opt->repo, tree,
+							 "ancestor");
 	}
 
-	for (iter = ca; iter; iter = iter->next) {
+	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
 		opt->call_depth++;
 		/*
@@ -3527,14 +3529,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
-				    NULL, &merged_common_ancestors) < 0)
+		if (merge_recursive_internal(opt, merged_merge_bases, iter->item,
+					     NULL, &merged_merge_bases) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
 		opt->call_depth--;
 
-		if (!merged_common_ancestors)
+		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
@@ -3548,7 +3550,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 		break;
 	case 1:
 		strbuf_add_unique_abbrev(&merge_base_abbrev,
-					 &merged_common_ancestors->object.oid,
+					 &merged_merge_bases->object.oid,
 					 DEFAULT_ABBREV);
 		opt->ancestor = merge_base_abbrev.buf;
 		break;
@@ -3559,7 +3561,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h1),
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
-							  merged_common_ancestors),
+							  merged_merge_bases),
 				     &mrtree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
@@ -3602,7 +3604,7 @@ static void merge_finalize(struct merge_options *opt)
 int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common)
+		struct tree *merge_base)
 {
 	int clean;
 	struct tree *ignored;
@@ -3611,7 +3613,7 @@ int merge_trees(struct merge_options *opt,
 
 	if (merge_start(opt, head))
 		return -1;
-	clean = merge_trees_internal(opt, head, merge, common, &ignored);
+	clean = merge_trees_internal(opt, head, merge, merge_base, &ignored);
 	merge_finalize(opt);
 
 	return clean;
@@ -3620,7 +3622,7 @@ int merge_trees(struct merge_options *opt,
 int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ca,
+		    struct commit_list *merge_bases,
 		    struct commit **result)
 {
 	int clean;
@@ -3629,7 +3631,7 @@ int merge_recursive(struct merge_options *opt,
 
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
-	clean = merge_recursive_internal(opt, h1, h2, ca, result);
+	clean = merge_recursive_internal(opt, h1, h2, merge_bases, result);
 	merge_finalize(opt);
 
 	return clean;
@@ -3657,8 +3659,8 @@ static struct commit *get_ref(struct repository *repo,
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_base_list,
-			    const struct object_id **base_list,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result)
 {
 	int clean;
@@ -3667,14 +3669,14 @@ int merge_recursive_generic(struct merge_options *opt,
 	struct commit *next_commit = get_ref(opt->repo, merge, opt->branch2);
 	struct commit_list *ca = NULL;
 
-	if (base_list) {
+	if (merge_bases) {
 		int i;
-		for (i = 0; i < num_base_list; ++i) {
+		for (i = 0; i < num_merge_bases; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i],
-					     oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, merge_bases[i],
+					     oid_to_hex(merge_bases[i]))))
 				return err(opt, _("Could not parse object '%s'"),
-					   oid_to_hex(base_list[i]));
+					   oid_to_hex(merge_bases[i]));
 			commit_list_insert(base, &ca);
 		}
 	}
diff --git a/merge-recursive.h b/merge-recursive.h
index 0a3033bdb0..6f351098a5 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -81,14 +81,14 @@ static inline int merge_detect_rename(struct merge_options *o)
  *
  * NOTE: empirically, about a decade ago it was determined that with more
  *       than two merge bases, optimal behavior was found when the
- *       ancestors were passed in the order of oldest merge base to newest
- *       one.  Also, ancestors will be consumed (emptied) so make a copy if
- *       you need it.
+ *       merge_bases were passed in the order of oldest commit to newest
+ *       commit.  Also, merge_bases will be consumed (emptied) so make a
+ *       copy if you need it.
  */
 int merge_recursive(struct merge_options *o,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ancestors,
+		    struct commit_list *merge_bases,
 		    struct commit **result);
 
 /*
@@ -98,7 +98,7 @@ int merge_recursive(struct merge_options *o,
 int merge_trees(struct merge_options *o,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common);
+		struct tree *merge_base);
 
 /*
  * "git-merge-recursive" can be fed trees; wrap them into
@@ -107,8 +107,8 @@ int merge_trees(struct merge_options *o,
 int merge_recursive_generic(struct merge_options *o,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_ca,
-			    const struct object_id **ca,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result);
 
 void init_merge_options(struct merge_options *o,
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 15/24] merge-recursive: rename 'mrtree' to 'result_tree', for clarity

[Elijah Newren]

It is not at all clear what 'mr' was supposed to stand for, at least not
to me.  Pick a clearer name for this variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index d0153ba971..bffb5eac7d 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3477,7 +3477,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 {
 	struct commit_list *iter;
 	struct commit *merged_merge_bases;
-	struct tree *mrtree;
+	struct tree *result_tree;
 	int clean;
 	int num_merge_bases;
 	struct strbuf merge_base_abbrev = STRBUF_INIT;
@@ -3562,7 +3562,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
 							  merged_merge_bases),
-				     &mrtree);
+				     &result_tree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
 		flush_output(opt);
@@ -3570,7 +3570,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 	}
 
 	if (opt->call_depth) {
-		*result = make_virtual_commit(opt->repo, mrtree, "merged tree");
+		*result = make_virtual_commit(opt->repo, result_tree,
+					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 16/24] merge-recursive: rename merge_options argument to opt in header

[Elijah Newren]

In commit 259ccb6cc324 ("merge-recursive: rename merge_options argument
from 'o' to 'opt'", 2019-04-05), I renamed a bunch of function
arguments in merge-recursive.c, but forgot to make that same change to
merge-recursive.h.  Make the two match.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.h | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/merge-recursive.h b/merge-recursive.h
index 6f351098a5..2cb3844ad9 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -68,10 +68,10 @@ struct collision_entry {
 	unsigned reported_already:1;
 };
 
-static inline int merge_detect_rename(struct merge_options *o)
+static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return o->merge_detect_rename >= 0 ? o->merge_detect_rename :
-		o->diff_detect_rename >= 0 ? o->diff_detect_rename : 1;
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
 }
 
 /*
@@ -85,7 +85,7 @@ static inline int merge_detect_rename(struct merge_options *o)
  *       commit.  Also, merge_bases will be consumed (emptied) so make a
  *       copy if you need it.
  */
-int merge_recursive(struct merge_options *o,
+int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
 		    struct commit_list *merge_bases,
@@ -95,7 +95,7 @@ int merge_recursive(struct merge_options *o,
  * rename-detecting three-way merge, no recursion; result of merge is written
  * to opt->repo->index.
  */
-int merge_trees(struct merge_options *o,
+int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
 		struct tree *merge_base);
@@ -104,16 +104,16 @@ int merge_trees(struct merge_options *o,
  * "git-merge-recursive" can be fed trees; wrap them into
  * virtual commits and call merge_recursive() proper.
  */
-int merge_recursive_generic(struct merge_options *o,
+int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
 			    int num_merge_bases,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *o,
+void init_merge_options(struct merge_options *opt,
 			struct repository *repo);
 
-int parse_merge_opt(struct merge_options *out, const char *s);
+int parse_merge_opt(struct merge_options *opt, const char *s);
 
 #endif
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 17/24] merge-recursive: move some definitions around to clean up the header

[Elijah Newren]

No substantive code changes (view this with diff --color-moved), but
a few small code cleanups:
  * Move structs and an inline function only used by merge-recursive.c
    into merge-recursive.c
  * Re-order function declarations to be more logical
  * Add or fix some explanatory comments

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 31 +++++++++++++++++
 merge-recursive.h | 87 +++++++++++++++++++++++------------------------
 2 files changed, 73 insertions(+), 45 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index bffb5eac7d..e401114b8f 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -54,6 +54,24 @@ static unsigned int path_hash(const char *path)
 	return ignore_case ? strihash(path) : strhash(path);
 }
 
+/*
+ * For dir_rename_entry, directory names are stored as a full path from the
+ * toplevel of the repository and do not include a trailing '/'.  Also:
+ *
+ *   dir:                original name of directory being renamed
+ *   non_unique_new_dir: if true, could not determine new_dir
+ *   new_dir:            final name of directory being renamed
+ *   possible_new_dirs:  temporary used to help determine new_dir; see comments
+ *                       in get_directory_renames() for details
+ */
+struct dir_rename_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *dir;
+	unsigned non_unique_new_dir:1;
+	struct strbuf new_dir;
+	struct string_list possible_new_dirs;
+};
+
 static struct dir_rename_entry *dir_rename_find_entry(struct hashmap *hashmap,
 						      char *dir)
 {
@@ -92,6 +110,13 @@ static void dir_rename_entry_init(struct dir_rename_entry *entry,
 	string_list_init(&entry->possible_new_dirs, 0);
 }
 
+struct collision_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *target_file;
+	struct string_list source_files;
+	unsigned reported_already:1;
+};
+
 static struct collision_entry *collision_find_entry(struct hashmap *hashmap,
 						    char *target_file)
 {
@@ -358,6 +383,12 @@ static int add_cacheinfo(struct merge_options *opt,
 	return ret;
 }
 
+static inline int merge_detect_rename(struct merge_options *opt)
+{
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+}
+
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 {
 	parse_tree(tree);
diff --git a/merge-recursive.h b/merge-recursive.h
index 2cb3844ad9..0fdae904dd 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -43,47 +43,50 @@ struct merge_options {
 	struct repository *repo;
 };
 
+void init_merge_options(struct merge_options *opt, struct repository *repo);
+
+/* parse the option in s and update the relevant field of opt */
+int parse_merge_opt(struct merge_options *opt, const char *s);
+
 /*
- * For dir_rename_entry, directory names are stored as a full path from the
- * toplevel of the repository and do not include a trailing '/'.  Also:
- *
- *   dir:                original name of directory being renamed
- *   non_unique_new_dir: if true, could not determine new_dir
- *   new_dir:            final name of directory being renamed
- *   possible_new_dirs:  temporary used to help determine new_dir; see comments
- *                       in get_directory_renames() for details
+ * RETURN VALUES: All the merge_* functions below return a value as follows:
+ *   > 0     Merge was clean
+ *   = 0     Merge had conflicts
+ *   < 0     Merge hit an unexpected and unrecoverable problem (e.g. disk
+ *             full) and aborted merge part-way through.
  */
-struct dir_rename_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *dir;
-	unsigned non_unique_new_dir:1;
-	struct strbuf new_dir;
-	struct string_list possible_new_dirs;
-};
-
-struct collision_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *target_file;
-	struct string_list source_files;
-	unsigned reported_already:1;
-};
 
-static inline int merge_detect_rename(struct merge_options *opt)
-{
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
-}
+/*
+ * rename-detecting three-way merge, no recursion.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - No commit is created
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is not updated
+ *   - The working tree is updated with results of the merge
+ */
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *merge_base);
 
 /*
  * merge_recursive is like merge_trees() but with recursive ancestor
- * consolidation, and when successful, it creates an actual commit
- * and writes its address to *result.
+ * consolidation and, if the commit is clean, creation of a commit.
  *
  * NOTE: empirically, about a decade ago it was determined that with more
  *       than two merge bases, optimal behavior was found when the
  *       merge_bases were passed in the order of oldest commit to newest
  *       commit.  Also, merge_bases will be consumed (emptied) so make a
  *       copy if you need it.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - If merge is clean, a commit is created and its address written to *result
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is not updated
+ *   - The working tree is updated with results of the merge
  */
 int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
@@ -92,17 +95,16 @@ int merge_recursive(struct merge_options *opt,
 		    struct commit **result);
 
 /*
- * rename-detecting three-way merge, no recursion; result of merge is written
- * to opt->repo->index.
- */
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *merge_base);
-
-/*
- * "git-merge-recursive" can be fed trees; wrap them into
- * virtual commits and call merge_recursive() proper.
+ * merge_recursive_generic can operate on trees instead of commits, by
+ * wrapping the trees into virtual commits, and calling merge_recursive().
+ * It also writes out the in-memory index to disk if the merge is successful.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - If merge is clean, a commit is created and its address written to *result
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is updated
+ *   - The working tree is updated with results of the merge
  */
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
@@ -111,9 +113,4 @@ int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *opt,
-			struct repository *repo);
-
-int parse_merge_opt(struct merge_options *opt, const char *s);
-
 #endif
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 18/24] merge-recursive: consolidate unnecessary fields in merge_options

[Elijah Newren]

We provided users with the ability to state whether they wanted rename
detection, and to put a limit on how much CPU would be spent.  Both of
these fields had multiple configuration parameters for setting them,
with one being a fallback and the other being an override.  However,
instead of implementing the logic for how to combine the multiple
source locations into the appropriate setting at config loading time,
we loaded and tracked both values and then made the code combine them
every time it wanted to check the overall value.  This had a few
minor drawbacks:
  * it seems more complicated than necessary
  * it runs the risk of people using the independent settings in the
    future and breaking the intent of how the options are used
    together
  * it makes merge_options more complicated than necessary for other
    potential users of the API

Fix these problems by moving the logic for combining the pairs of
options into a single value; make it apply at time-of-config-loading
instead of each-time-of-use.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 27 +++++++++++----------------
 merge-recursive.h |  6 ++----
 2 files changed, 13 insertions(+), 20 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index e401114b8f..b846acf931 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -385,8 +385,7 @@ static int add_cacheinfo(struct merge_options *opt,
 
 static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+	return (opt->detect_renames != -1) ? opt->detect_renames : 1;
 }
 
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
@@ -1883,9 +1882,7 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	 */
 	if (opts.detect_rename > DIFF_DETECT_RENAME)
 		opts.detect_rename = DIFF_DETECT_RENAME;
-	opts.rename_limit = opt->merge_rename_limit >= 0 ? opt->merge_rename_limit :
-			    opt->diff_rename_limit >= 0 ? opt->diff_rename_limit :
-			    1000;
+	opts.rename_limit = (opt->rename_limit != -1) ? opt->rename_limit : 1000;
 	opts.rename_score = opt->rename_score;
 	opts.show_rename_progress = opt->show_rename_progress;
 	opts.output_format = DIFF_FORMAT_NO_OUTPUT;
@@ -3732,14 +3729,14 @@ static void merge_recursive_config(struct merge_options *opt)
 {
 	char *value = NULL;
 	git_config_get_int("merge.verbosity", &opt->verbosity);
-	git_config_get_int("diff.renamelimit", &opt->diff_rename_limit);
-	git_config_get_int("merge.renamelimit", &opt->merge_rename_limit);
+	git_config_get_int("diff.renamelimit", &opt->rename_limit);
+	git_config_get_int("merge.renamelimit", &opt->rename_limit);
 	if (!git_config_get_string("diff.renames", &value)) {
-		opt->diff_detect_rename = git_config_rename("diff.renames", value);
+		opt->detect_renames = git_config_rename("diff.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.renames", &value)) {
-		opt->merge_detect_rename = git_config_rename("merge.renames", value);
+		opt->detect_renames = git_config_rename("merge.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.directoryrenames", &value)) {
@@ -3765,11 +3762,9 @@ void init_merge_options(struct merge_options *opt,
 	opt->repo = repo;
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->diff_rename_limit = -1;
-	opt->merge_rename_limit = -1;
+	opt->rename_limit = -1;
 	opt->renormalize = 0;
-	opt->diff_detect_rename = -1;
-	opt->merge_detect_rename = -1;
+	opt->detect_renames = -1;
 	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
@@ -3821,16 +3816,16 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	else if (!strcmp(s, "no-renormalize"))
 		opt->renormalize = 0;
 	else if (!strcmp(s, "no-renames"))
-		opt->merge_detect_rename = 0;
+		opt->detect_renames = 0;
 	else if (!strcmp(s, "find-renames")) {
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 		opt->rename_score = 0;
 	}
 	else if (skip_prefix(s, "find-renames=", &arg) ||
 		 skip_prefix(s, "rename-threshold=", &arg)) {
 		if ((opt->rename_score = parse_rename_score(&arg)) == -1 || *arg != 0)
 			return -1;
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 	}
 	/*
 	 * Please update $__git_merge_strategy_options in
diff --git a/merge-recursive.h b/merge-recursive.h
index 0fdae904dd..f4bdfbc897 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -27,10 +27,8 @@ struct merge_options {
 		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
 		MERGE_DIRECTORY_RENAMES_TRUE = 2
 	} detect_directory_renames;
-	int diff_detect_rename;
-	int merge_detect_rename;
-	int diff_rename_limit;
-	int merge_rename_limit;
+	int detect_renames;
+	int rename_limit;
 	int rename_score;
 	int needed_rename_limit;
 	int show_rename_progress;
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 19/24] merge-recursive: comment and reorder the merge_options fields

[Elijah Newren]

The merge_options struct had lots of fields, making it a little
imposing, but the options naturally fall into multiple different groups.
Grouping similar options and adding a comment or two makes it easier to
read, easier for new folks to figure out which options are related, and
thus easier for them to find the options they need.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 16 +++++++++++-----
 merge-recursive.h | 40 ++++++++++++++++++++++++++--------------
 2 files changed, 37 insertions(+), 19 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index b846acf931..db2c544d0b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3759,21 +3759,27 @@ void init_merge_options(struct merge_options *opt,
 {
 	const char *merge_verbosity;
 	memset(opt, 0, sizeof(struct merge_options));
+
 	opt->repo = repo;
+
+	opt->detect_renames = -1;
+	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
+	opt->rename_limit = -1;
+
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->rename_limit = -1;
+	strbuf_init(&opt->obuf, 0);
+
 	opt->renormalize = 0;
-	opt->detect_renames = -1;
-	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
+
+	string_list_init(&opt->df_conflict_file_set, 1);
+
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
 		opt->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (opt->verbosity >= 5)
 		opt->buffer_output = 0;
-	strbuf_init(&opt->obuf, 0);
-	string_list_init(&opt->df_conflict_file_set, 1);
 }
 
 int parse_merge_opt(struct merge_options *opt, const char *s)
diff --git a/merge-recursive.h b/merge-recursive.h
index f4bdfbc897..9e040608fe 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -9,36 +9,48 @@ struct commit;
 struct repository;
 
 struct merge_options {
+	struct repository *repo;
+
+	/* ref names used in console messages and conflict markers */
 	const char *ancestor;
 	const char *branch1;
 	const char *branch2;
-	enum {
-		MERGE_RECURSIVE_NORMAL = 0,
-		MERGE_RECURSIVE_OURS,
-		MERGE_RECURSIVE_THEIRS
-	} recursive_variant;
-	const char *subtree_shift;
-	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
-	unsigned renormalize : 1;
-	long xdl_opts;
-	int verbosity;
+
+	/* rename related options */
+	int detect_renames;
 	enum {
 		MERGE_DIRECTORY_RENAMES_NONE = 0,
 		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
 		MERGE_DIRECTORY_RENAMES_TRUE = 2
 	} detect_directory_renames;
-	int detect_renames;
 	int rename_limit;
 	int rename_score;
-	int needed_rename_limit;
 	int show_rename_progress;
+
+	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
+	long xdl_opts;
+	enum {
+		MERGE_RECURSIVE_NORMAL = 0,
+		MERGE_RECURSIVE_OURS,
+		MERGE_RECURSIVE_THEIRS
+	} recursive_variant;
+
+	/* console output related options */
+	int verbosity;
+	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
+	struct strbuf obuf;     /* output buffer */
+
+	/* miscellaneous control options */
+	const char *subtree_shift;
+	unsigned renormalize : 1;
+
+	/* internal fields used by the implementation (do NOT set these) */
 	int call_depth;
-	struct strbuf obuf;
+	int needed_rename_limit;
 	struct hashmap current_file_dir_set;
 	struct string_list df_conflict_file_set;
 	struct unpack_trees_options unpack_opts;
 	struct index_state orig_index;
-	struct repository *repo;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 20/24] merge-recursive: avoid losing output and leaking memory holding that output

[Elijah Newren]

If opt->buffer_output is less than 2, then merge_trees(),
merge_recursive(), and merge_recursive_generic() are all supposed to
flush the opt->obuf output buffer to stdout and release any memory it
holds.  merge_trees() did not do this.  Move the logic that handles this
for merge_recursive_internal() to merge_finalize() so that all three
methods handle this requirement.

Note that this bug didn't cause any problems currently, because there
are only two callers of merge_trees() right now (a git grep for
'merge_trees(' is misleading because builtin/merge-tree.c also defines a
'merge_tree' function that is unrelated), and only one of those is
called with buffer_output less than 2 (builtin/checkout.c), but it set
opt->verbosity to 0, for which there is only currently one non-error
message that would be shown: "Already up to date!".  However, that one
message can only occur when the merge is utterly trivial (the merge base
tree exactly matches the merge tree), and builtin/checkout.c already
attempts a trivial merge via unpack_trees() before falling back to
merge_trees().

Also, if opt->buffer_output is 2, then the caller is responsible to
handle showing any output in opt->obuf and for free'ing it.  This
requirement might be easy to overlook, so add a comment to
merge-recursive.h pointing it out.  (There are currently two callers
that set buffer_output to 2, both in sequencer.c, and both of which
handle this correctly.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 6 +++---
 merge-recursive.h | 3 ++-
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index db2c544d0b..b4334d0506 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3603,9 +3603,6 @@ static int merge_recursive_internal(struct merge_options *opt,
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-	flush_output(opt);
-	if (!opt->call_depth && opt->buffer_output < 2)
-		strbuf_release(&opt->obuf);
 	return clean;
 }
 
@@ -3625,6 +3622,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 
 static void merge_finalize(struct merge_options *opt)
 {
+	flush_output(opt);
+	if (!opt->call_depth && opt->buffer_output < 2)
+		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       opt->needed_rename_limit, 0);
diff --git a/merge-recursive.h b/merge-recursive.h
index 9e040608fe..933d6e7642 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -38,7 +38,8 @@ struct merge_options {
 	/* console output related options */
 	int verbosity;
 	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
-	struct strbuf obuf;     /* output buffer */
+	struct strbuf obuf;     /* output buffer; if buffer_output == 2, caller
+				 * must handle and call strbuf_release */
 
 	/* miscellaneous control options */
 	const char *subtree_shift;
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 21/24] merge-recursive: split internal fields into a separate struct

[Elijah Newren]

merge_options has several internal fields that should not be set or read
by external callers.  This just complicates the API.  Move them into an
opaque merge_options_internal struct that is defined only in
merge-recursive.c and keep these out of merge-recursive.h.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 186 ++++++++++++++++++++++++----------------------
 merge-recursive.h |  17 ++---
 2 files changed, 106 insertions(+), 97 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index b4334d0506..840b09f1dc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -29,6 +29,15 @@
 #include "revision.h"
 #include "commit-reach.h"
 
+struct merge_options_internal {
+	int call_depth;
+	int needed_rename_limit;
+	struct hashmap current_file_dir_set;
+	struct string_list df_conflict_file_set;
+	struct unpack_trees_options unpack_opts;
+	struct index_state orig_index;
+};
+
 struct path_hashmap_entry {
 	struct hashmap_entry e;
 	char path[FLEX_ARRAY];
@@ -309,7 +318,8 @@ static inline void setup_rename_conflict_info(enum rename_type rename_type,
 
 static int show(struct merge_options *opt, int v)
 {
-	return (!opt->call_depth && opt->verbosity >= v) || opt->verbosity >= 5;
+	return (!opt->priv->call_depth && opt->verbosity >= v) ||
+		opt->verbosity >= 5;
 }
 
 __attribute__((format (printf, 3, 4)))
@@ -320,7 +330,7 @@ static void output(struct merge_options *opt, int v, const char *fmt, ...)
 	if (!show(opt, v))
 		return;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 
 	va_start(ap, fmt);
 	strbuf_vaddf(&opt->obuf, fmt, ap);
@@ -335,7 +345,7 @@ static void output_commit_title(struct merge_options *opt, struct commit *commit
 {
 	struct merge_remote_desc *desc;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 	desc = merge_remote_util(commit);
 	if (desc)
 		strbuf_addf(&opt->obuf, "virtual %s\n", desc->name);
@@ -403,43 +413,43 @@ static int unpack_trees_start(struct merge_options *opt,
 	struct tree_desc t[3];
 	struct index_state tmp_index = { NULL };
 
-	memset(&opt->unpack_opts, 0, sizeof(opt->unpack_opts));
-	if (opt->call_depth)
-		opt->unpack_opts.index_only = 1;
+	memset(&opt->priv->unpack_opts, 0, sizeof(opt->priv->unpack_opts));
+	if (opt->priv->call_depth)
+		opt->priv->unpack_opts.index_only = 1;
 	else
-		opt->unpack_opts.update = 1;
-	opt->unpack_opts.merge = 1;
-	opt->unpack_opts.head_idx = 2;
-	opt->unpack_opts.fn = threeway_merge;
-	opt->unpack_opts.src_index = opt->repo->index;
-	opt->unpack_opts.dst_index = &tmp_index;
-	opt->unpack_opts.aggressive = !merge_detect_rename(opt);
-	setup_unpack_trees_porcelain(&opt->unpack_opts, "merge");
+		opt->priv->unpack_opts.update = 1;
+	opt->priv->unpack_opts.merge = 1;
+	opt->priv->unpack_opts.head_idx = 2;
+	opt->priv->unpack_opts.fn = threeway_merge;
+	opt->priv->unpack_opts.src_index = opt->repo->index;
+	opt->priv->unpack_opts.dst_index = &tmp_index;
+	opt->priv->unpack_opts.aggressive = !merge_detect_rename(opt);
+	setup_unpack_trees_porcelain(&opt->priv->unpack_opts, "merge");
 
 	init_tree_desc_from_tree(t+0, common);
 	init_tree_desc_from_tree(t+1, head);
 	init_tree_desc_from_tree(t+2, merge);
 
-	rc = unpack_trees(3, t, &opt->unpack_opts);
+	rc = unpack_trees(3, t, &opt->priv->unpack_opts);
 	cache_tree_free(&opt->repo->index->cache_tree);
 
 	/*
-	 * Update opt->repo->index to match the new results, AFTER saving a copy
-	 * in opt->orig_index.  Update src_index to point to the saved copy.
-	 * (verify_uptodate() checks src_index, and the original index is
-	 * the one that had the necessary modification timestamps.)
+	 * Update opt->repo->index to match the new results, AFTER saving a
+	 * copy in opt->priv->orig_index.  Update src_index to point to the
+	 * saved copy.  (verify_uptodate() checks src_index, and the original
+	 * index is the one that had the necessary modification timestamps.)
 	 */
-	opt->orig_index = *opt->repo->index;
+	opt->priv->orig_index = *opt->repo->index;
 	*opt->repo->index = tmp_index;
-	opt->unpack_opts.src_index = &opt->orig_index;
+	opt->priv->unpack_opts.src_index = &opt->priv->orig_index;
 
 	return rc;
 }
 
 static void unpack_trees_finish(struct merge_options *opt)
 {
-	discard_index(&opt->orig_index);
-	clear_unpack_trees_porcelain(&opt->unpack_opts);
+	discard_index(&opt->priv->orig_index);
+	clear_unpack_trees_porcelain(&opt->priv->unpack_opts);
 }
 
 static int save_files_dirs(const struct object_id *oid,
@@ -454,7 +464,7 @@ static int save_files_dirs(const struct object_id *oid,
 
 	FLEX_ALLOC_MEM(entry, path, base->buf, base->len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 
 	strbuf_setlen(base, baselen);
 	return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);
@@ -585,7 +595,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	 * If we're merging merge-bases, we don't want to bother with
 	 * any working directory changes.
 	 */
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return;
 
 	/* Ensure D/F conflicts are adjacent in the entries list. */
@@ -597,7 +607,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	df_sorted_entries.cmp = string_list_df_name_compare;
 	string_list_sort(&df_sorted_entries);
 
-	string_list_clear(&opt->df_conflict_file_set, 1);
+	string_list_clear(&opt->priv->df_conflict_file_set, 1);
 	for (i = 0; i < df_sorted_entries.nr; i++) {
 		const char *path = df_sorted_entries.items[i].string;
 		int len = strlen(path);
@@ -613,7 +623,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 		    len > last_len &&
 		    memcmp(path, last_file, last_len) == 0 &&
 		    path[last_len] == '/') {
-			string_list_insert(&opt->df_conflict_file_set, last_file);
+			string_list_insert(&opt->priv->df_conflict_file_set, last_file);
 		}
 
 		/*
@@ -680,8 +690,8 @@ static void update_entry(struct stage_data *entry,
 static int remove_file(struct merge_options *opt, int clean,
 		       const char *path, int no_wd)
 {
-	int update_cache = opt->call_depth || clean;
-	int update_working_directory = !opt->call_depth && !no_wd;
+	int update_cache = opt->priv->call_depth || clean;
+	int update_working_directory = !opt->priv->call_depth && !no_wd;
 
 	if (update_cache) {
 		if (remove_file_from_index(opt->repo->index, path))
@@ -724,16 +734,16 @@ static char *unique_path(struct merge_options *opt,
 	add_flattened_path(&newpath, branch);
 
 	base_len = newpath.len;
-	while (hashmap_get_from_hash(&opt->current_file_dir_set,
+	while (hashmap_get_from_hash(&opt->priv->current_file_dir_set,
 				     path_hash(newpath.buf), newpath.buf) ||
-	       (!opt->call_depth && file_exists(newpath.buf))) {
+	       (!opt->priv->call_depth && file_exists(newpath.buf))) {
 		strbuf_setlen(&newpath, base_len);
 		strbuf_addf(&newpath, "_%d", suffix++);
 	}
 
 	FLEX_ALLOC_MEM(entry, path, newpath.buf, newpath.len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 	return strbuf_detach(&newpath, NULL);
 }
 
@@ -775,7 +785,7 @@ static int dir_in_way(struct index_state *istate, const char *path,
 static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 				   const struct diff_filespec *blob)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 	struct cache_entry *ce;
 
 	if (0 > pos)
@@ -783,7 +793,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 		return 0;
 
 	/* See if the file we were tracking before matches */
-	ce = opt->orig_index.cache[pos];
+	ce = opt->priv->orig_index.cache[pos];
 	return (oid_eq(&ce->oid, &blob->oid) && ce->ce_mode == blob->mode);
 }
 
@@ -792,7 +802,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
  */
 static int was_tracked(struct merge_options *opt, const char *path)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 
 	if (0 <= pos)
 		/* we were tracking this path before the merge */
@@ -849,12 +859,12 @@ static int was_dirty(struct merge_options *opt, const char *path)
 	struct cache_entry *ce;
 	int dirty = 1;
 
-	if (opt->call_depth || !was_tracked(opt, path))
+	if (opt->priv->call_depth || !was_tracked(opt, path))
 		return !dirty;
 
-	ce = index_file_exists(opt->unpack_opts.src_index,
+	ce = index_file_exists(opt->priv->unpack_opts.src_index,
 			       path, strlen(path), ignore_case);
-	dirty = verify_uptodate(ce, &opt->unpack_opts) != 0;
+	dirty = verify_uptodate(ce, &opt->priv->unpack_opts) != 0;
 	return dirty;
 }
 
@@ -864,8 +874,8 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 	const char *msg = _("failed to create path '%s'%s");
 
 	/* Unlink any D/F conflict files that are in the way */
-	for (i = 0; i < opt->df_conflict_file_set.nr; i++) {
-		const char *df_path = opt->df_conflict_file_set.items[i].string;
+	for (i = 0; i < opt->priv->df_conflict_file_set.nr; i++) {
+		const char *df_path = opt->priv->df_conflict_file_set.items[i].string;
 		size_t pathlen = strlen(path);
 		size_t df_pathlen = strlen(df_path);
 		if (df_pathlen < pathlen &&
@@ -875,7 +885,7 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 			       _("Removing %s to make room for subdirectory\n"),
 			       df_path);
 			unlink(df_path);
-			unsorted_string_list_delete_item(&opt->df_conflict_file_set,
+			unsorted_string_list_delete_item(&opt->priv->df_conflict_file_set,
 							 i, 0);
 			break;
 		}
@@ -916,7 +926,7 @@ static int update_file_flags(struct merge_options *opt,
 {
 	int ret = 0;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		update_wd = 0;
 
 	if (update_wd) {
@@ -1001,7 +1011,7 @@ static int update_file(struct merge_options *opt,
 		       const char *path)
 {
 	return update_file_flags(opt, contents, path,
-				 opt->call_depth || clean, !opt->call_depth);
+				 opt->priv->call_depth || clean, !opt->priv->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1030,7 +1040,7 @@ static int merge_3way(struct merge_options *opt,
 	ll_opts.extra_marker_size = extra_marker_size;
 	ll_opts.xdl_opts = opt->xdl_opts;
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		ll_opts.virtual_ancestor = 1;
 		ll_opts.variant = 0;
 	} else {
@@ -1161,7 +1171,7 @@ static int merge_submodule(struct merge_options *opt,
 	struct object_array merges;
 
 	int i;
-	int search = !opt->call_depth;
+	int search = !opt->priv->call_depth;
 
 	/* store a in result in case we fail */
 	oidcpy(result, a);
@@ -1383,7 +1393,7 @@ static int handle_rename_via_dir(struct merge_options *opt,
 			       MERGE_DIRECTORY_RENAMES_CONFLICT);
 	assert(ren->dir_rename_original_dest);
 
-	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
+	if (!opt->priv->call_depth && would_lose_untracked(opt, dest->path)) {
 		mark_conflicted = 1;
 		file_path = unique_path(opt, dest->path, ren->branch);
 		output(opt, 1, _("Error: Refusing to lose untracked file at %s; "
@@ -1426,12 +1436,12 @@ static int handle_change_delete(struct merge_options *opt,
 	const char *update_path = path;
 	int ret = 0;
 
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0) ||
-	    (!opt->call_depth && would_lose_untracked(opt, path))) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0) ||
+	    (!opt->priv->call_depth && would_lose_untracked(opt, path))) {
 		update_path = alt_path = unique_path(opt, path, change_branch);
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * We cannot arbitrarily accept either a_sha or b_sha as
 		 * correct; since there is no true "middle point" between
@@ -1506,14 +1516,14 @@ static int handle_rename_delete(struct merge_options *opt,
 				     opt->branch2 : opt->branch1);
 
 	if (handle_change_delete(opt,
-				 opt->call_depth ? orig->path : dest->path,
-				 opt->call_depth ? NULL : orig->path,
+				 opt->priv->call_depth ? orig->path : dest->path,
+				 opt->priv->call_depth ? NULL : orig->path,
 				 orig, dest,
 				 rename_branch, delete_branch,
 				 _("rename"), _("renamed")))
 		return -1;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return remove_file_from_index(opt->repo->index, dest->path);
 	else
 		return update_stages(opt, dest->path, NULL,
@@ -1550,7 +1560,7 @@ static int handle_file_collision(struct merge_options *opt,
 	/*
 	 * In the recursive case, we just opt to undo renames
 	 */
-	if (opt->call_depth && (prev_path1 || prev_path2)) {
+	if (opt->priv->call_depth && (prev_path1 || prev_path2)) {
 		/* Put first file (a->oid, a->mode) in its original spot */
 		if (prev_path1) {
 			if (update_file(opt, 1, a, prev_path1))
@@ -1579,10 +1589,10 @@ static int handle_file_collision(struct merge_options *opt,
 	/* Remove rename sources if rename/add or rename/rename(2to1) */
 	if (prev_path1)
 		remove_file(opt, 1, prev_path1,
-			    opt->call_depth || would_lose_untracked(opt, prev_path1));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path1));
 	if (prev_path2)
 		remove_file(opt, 1, prev_path2,
-			    opt->call_depth || would_lose_untracked(opt, prev_path2));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path2));
 
 	/*
 	 * Remove the collision path, if it wouldn't cause dirty contents
@@ -1624,12 +1634,12 @@ static int handle_file_collision(struct merge_options *opt,
 	null.mode = 0;
 
 	if (merge_mode_and_contents(opt, &null, a, b, collide_path,
-				    branch1, branch2, opt->call_depth * 2, &mfi))
+				    branch1, branch2, opt->priv->call_depth * 2, &mfi))
 		return -1;
 	mfi.clean &= !alt_path;
 	if (update_file(opt, mfi.clean, &mfi.blob, update_path))
 		return -1;
-	if (!mfi.clean && !opt->call_depth &&
+	if (!mfi.clean && !opt->priv->call_depth &&
 	    update_stages(opt, collide_path, NULL, a, b))
 		return -1;
 	free(alt_path);
@@ -1669,7 +1679,7 @@ static int handle_rename_add(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[other_stage],
 				    prev_path_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi))
+				    1 + opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(prev_path_desc);
 
@@ -1687,7 +1697,7 @@ static char *find_path_for_conflict(struct merge_options *opt,
 				    const char *branch2)
 {
 	char *new_path = NULL;
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0)) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0)) {
 		new_path = unique_path(opt, path, branch1);
 		output(opt, 1, _("%s is a directory in %s adding "
 			       "as %s instead"),
@@ -1718,17 +1728,17 @@ static int handle_rename_rename_1to2(struct merge_options *opt,
 	       "rename \"%s\"->\"%s\" in \"%s\"%s"),
 	       o->path, a->path, ci->ren1->branch,
 	       o->path, b->path, ci->ren2->branch,
-	       opt->call_depth ? _(" (left unresolved)") : "");
+	       opt->priv->call_depth ? _(" (left unresolved)") : "");
 
 	path_desc = xstrfmt("%s and %s, both renamed from %s",
 			    a->path, b->path, o->path);
 	if (merge_mode_and_contents(opt, o, a, b, path_desc,
 				    ci->ren1->branch, ci->ren2->branch,
-				    opt->call_depth * 2, &mfi))
+				    opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(path_desc);
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1843,12 +1853,12 @@ static int handle_rename_rename_2to1(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[ostage1],
 				    path_side_1_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c1) ||
+				    1 + opt->priv->call_depth * 2, &mfi_c1) ||
 	    merge_mode_and_contents(opt, b,
 				    &ci->ren2->src_entry->stages[ostage2],
 				    c2, path_side_2_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c2))
+				    1 + opt->priv->call_depth * 2, &mfi_c2))
 		return -1;
 	free(path_side_1_desc);
 	free(path_side_2_desc);
@@ -1889,8 +1899,8 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	diff_setup_done(&opts);
 	diff_tree_oid(&o_tree->object.oid, &tree->object.oid, "", &opts);
 	diffcore_std(&opts);
-	if (opts.needed_rename_limit > opt->needed_rename_limit)
-		opt->needed_rename_limit = opts.needed_rename_limit;
+	if (opts.needed_rename_limit > opt->priv->needed_rename_limit)
+		opt->priv->needed_rename_limit = opts.needed_rename_limit;
 
 	ret = xmalloc(sizeof(*ret));
 	*ret = diff_queued_diff;
@@ -2865,7 +2875,7 @@ static int detect_and_process_renames(struct merge_options *opt,
 
 	if ((opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE) ||
 	    (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_CONFLICT &&
-	     !opt->call_depth)) {
+	     !opt->priv->call_depth)) {
 		dir_re_head = get_directory_renames(head_pairs);
 		dir_re_merge = get_directory_renames(merge_pairs);
 
@@ -3022,13 +3032,13 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		reason = _("add/add");
 
 	assert(o->path && a->path && b->path);
-	if (ci && dir_in_way(opt->repo->index, path, !opt->call_depth,
+	if (ci && dir_in_way(opt->repo->index, path, !opt->priv->call_depth,
 			     S_ISGITLINK(ci->ren1->pair->two->mode)))
 		df_conflict_remains = 1;
 
 	if (merge_mode_and_contents(opt, o, a, b, path,
 				    opt->branch1, opt->branch2,
-				    opt->call_depth * 2, mfi))
+				    opt->priv->call_depth * 2, mfi))
 		return -1;
 
 	/*
@@ -3044,7 +3054,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 		output(opt, 3, _("Skipped %s (merged same as existing)"), path);
 		if (add_cacheinfo(opt, &mfi->blob, path,
-				  0, (!opt->call_depth && !is_dirty), 0))
+				  0, (!opt->priv->call_depth && !is_dirty), 0))
 			return -1;
 		/*
 		 * However, add_cacheinfo() will delete the old cache entry
@@ -3052,8 +3062,8 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		 * flag to avoid making the file appear as if it were
 		 * deleted by the user.
 		 */
-		pos = index_name_pos(&opt->orig_index, path, strlen(path));
-		ce = opt->orig_index.cache[pos];
+		pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
+		ce = opt->priv->orig_index.cache[pos];
 		if (ce_skip_worktree(ce)) {
 			pos = index_name_pos(opt->repo->index, path, strlen(path));
 			ce = opt->repo->index->cache[pos];
@@ -3074,7 +3084,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 	if (df_conflict_remains || is_dirty) {
 		char *new_path;
-		if (opt->call_depth) {
+		if (opt->priv->call_depth) {
 			remove_file_from_index(opt->repo->index, path);
 		} else {
 			if (!mfi->clean) {
@@ -3333,7 +3343,7 @@ static int process_entry(struct merge_options *opt,
 			conf = _("directory/file");
 		}
 		if (dir_in_way(opt->repo->index, path,
-			       !opt->call_depth && !S_ISGITLINK(a->mode),
+			       !opt->priv->call_depth && !S_ISGITLINK(a->mode),
 			       0)) {
 			char *new_path = unique_path(opt, path, add_branch);
 			clean_merge = 0;
@@ -3342,7 +3352,7 @@ static int process_entry(struct merge_options *opt,
 			       conf, path, other_branch, path, new_path);
 			if (update_file(opt, 0, contents, new_path))
 				clean_merge = -1;
-			else if (opt->call_depth)
+			else if (opt->priv->call_depth)
 				remove_file_from_index(opt->repo->index, path);
 			free(new_path);
 		} else {
@@ -3407,7 +3417,7 @@ static int merge_trees_internal(struct merge_options *opt,
 	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
-		if (show(opt, 4) || opt->call_depth)
+		if (show(opt, 4) || opt->priv->call_depth)
 			err(opt, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
@@ -3426,7 +3436,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+		hashmap_init(&opt->priv->current_file_dir_set, path_hashmap_cmp,
 			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
@@ -3463,7 +3473,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		string_list_clear(entries, 1);
 		free(entries);
 
-		hashmap_free(&opt->current_file_dir_set, 1);
+		hashmap_free(&opt->priv->current_file_dir_set, 1);
 
 		if (clean < 0) {
 			unpack_trees_finish(opt);
@@ -3475,7 +3485,7 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth &&
+	if (opt->priv->call_depth &&
 	    !(*result = write_inmemory_index_as_tree(opt->repo)))
 		return -1;
 
@@ -3543,7 +3553,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 
 	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
-		opt->call_depth++;
+		opt->priv->call_depth++;
 		/*
 		 * When the merge fails, the result contains files
 		 * with conflict markers. The cleanness flag is
@@ -3562,14 +3572,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
-		opt->call_depth--;
+		opt->priv->call_depth--;
 
 		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
 	discard_index(opt->repo->index);
-	if (!opt->call_depth)
+	if (!opt->priv->call_depth)
 		repo_read_index(opt->repo);
 
 	switch (num_merge_bases) {
@@ -3597,7 +3607,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 		return clean;
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		*result = make_virtual_commit(opt->repo, result_tree,
 					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
@@ -3617,17 +3627,21 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 		return -1;
 	}
 
+	opt->priv = xcalloc(1, sizeof(*opt->priv));
+	string_list_init(&opt->priv->df_conflict_file_set, 1);
 	return 0;
 }
 
 static void merge_finalize(struct merge_options *opt)
 {
 	flush_output(opt);
-	if (!opt->call_depth && opt->buffer_output < 2)
+	if (!opt->priv->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0);
+	free(opt->priv);
+	opt->priv = NULL;
 }
 
 int merge_trees(struct merge_options *opt,
@@ -3772,8 +3786,6 @@ void init_merge_options(struct merge_options *opt,
 
 	opt->renormalize = 0;
 
-	string_list_init(&opt->df_conflict_file_set, 1);
-
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
diff --git a/merge-recursive.h b/merge-recursive.h
index 933d6e7642..58a4c5238a 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -1,13 +1,15 @@
 #ifndef MERGE_RECURSIVE_H
 #define MERGE_RECURSIVE_H
 
-#include "string-list.h"
-#include "unpack-trees.h"
+#include "strbuf.h"
 
 struct commit;
-
+struct commit_list;
+struct object_id;
 struct repository;
+struct tree;
 
+struct merge_options_internal;
 struct merge_options {
 	struct repository *repo;
 
@@ -45,13 +47,8 @@ struct merge_options {
 	const char *subtree_shift;
 	unsigned renormalize : 1;
 
-	/* internal fields used by the implementation (do NOT set these) */
-	int call_depth;
-	int needed_rename_limit;
-	struct hashmap current_file_dir_set;
-	struct string_list df_conflict_file_set;
-	struct unpack_trees_options unpack_opts;
-	struct index_state orig_index;
+	/* internal fields used by the implementation */
+	struct merge_options_internal *priv;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 22/24] merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*

[Elijah Newren]

I want to implement the same outward facing API as found within
merge-recursive.h in a different merge strategy.  However, that makes
names like MERGE_RECURSIVE_{NORMAL,OURS,THEIRS} look a little funny;
rename to MERGE_VARIANT_{NORMAL,OURS,THEIRS}.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 14 +++++++-------
 merge-recursive.h |  6 +++---
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 840b09f1dc..647b1f25c3 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1045,10 +1045,10 @@ static int merge_3way(struct merge_options *opt,
 		ll_opts.variant = 0;
 	} else {
 		switch (opt->recursive_variant) {
-		case MERGE_RECURSIVE_OURS:
+		case MERGE_VARIANT_OURS:
 			ll_opts.variant = XDL_MERGE_FAVOR_OURS;
 			break;
-		case MERGE_RECURSIVE_THEIRS:
+		case MERGE_VARIANT_THEIRS:
 			ll_opts.variant = XDL_MERGE_FAVOR_THEIRS;
 			break;
 		default:
@@ -1355,15 +1355,15 @@ static int merge_mode_and_contents(struct merge_options *opt,
 							&b->oid);
 		} else if (S_ISLNK(a->mode)) {
 			switch (opt->recursive_variant) {
-			case MERGE_RECURSIVE_NORMAL:
+			case MERGE_VARIANT_NORMAL:
 				oidcpy(&result->blob.oid, &a->oid);
 				if (!oid_eq(&a->oid, &b->oid))
 					result->clean = 0;
 				break;
-			case MERGE_RECURSIVE_OURS:
+			case MERGE_VARIANT_OURS:
 				oidcpy(&result->blob.oid, &a->oid);
 				break;
-			case MERGE_RECURSIVE_THEIRS:
+			case MERGE_VARIANT_THEIRS:
 				oidcpy(&result->blob.oid, &b->oid);
 				break;
 			}
@@ -3801,9 +3801,9 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	if (!s || !*s)
 		return -1;
 	if (!strcmp(s, "ours"))
-		opt->recursive_variant = MERGE_RECURSIVE_OURS;
+		opt->recursive_variant = MERGE_VARIANT_OURS;
 	else if (!strcmp(s, "theirs"))
-		opt->recursive_variant = MERGE_RECURSIVE_THEIRS;
+		opt->recursive_variant = MERGE_VARIANT_THEIRS;
 	else if (!strcmp(s, "subtree"))
 		opt->subtree_shift = "";
 	else if (skip_prefix(s, "subtree=", &arg))
diff --git a/merge-recursive.h b/merge-recursive.h
index 58a4c5238a..978847e672 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -32,9 +32,9 @@ struct merge_options {
 	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
 	long xdl_opts;
 	enum {
-		MERGE_RECURSIVE_NORMAL = 0,
-		MERGE_RECURSIVE_OURS,
-		MERGE_RECURSIVE_THEIRS
+		MERGE_VARIANT_NORMAL = 0,
+		MERGE_VARIANT_OURS,
+		MERGE_VARIANT_THEIRS
 	} recursive_variant;
 
 	/* console output related options */
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 23/24] merge-recursive: add sanity checks for relevant merge_options

[Elijah Newren]

There are lots of options that callers can set, yet most have a limited
range of valid values, some options are meant for output (e.g.
opt->obuf, which is expected to start empty), and callers are expected
to not set opt->priv.  Add several sanity checks to ensure callers
provide sane values.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 23 +++++++++++++++++++++++
 merge-recursive.h |  2 +-
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 647b1f25c3..bc0da608c4 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3620,6 +3620,29 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 {
 	struct strbuf sb = STRBUF_INIT;
 
+	/* Sanity checks on opt */
+	assert(opt->repo);
+
+	assert(opt->branch1 && opt->branch2);
+
+	assert(opt->detect_renames >= -1 &&
+	       opt->detect_renames <= DIFF_DETECT_COPY);
+	assert(opt->detect_directory_renames >= MERGE_DIRECTORY_RENAMES_NONE &&
+	       opt->detect_directory_renames <= MERGE_DIRECTORY_RENAMES_TRUE);
+	assert(opt->rename_limit >= -1);
+	assert(opt->rename_score >= 0 && opt->rename_score <= MAX_SCORE);
+
+	assert(opt->xdl_opts >= 0);
+	assert(opt->recursive_variant >= MERGE_VARIANT_NORMAL &&
+	       opt->recursive_variant <= MERGE_VARIANT_THEIRS);
+
+	assert(opt->verbosity >= 0 && opt->verbosity <= 5);
+	assert(opt->buffer_output >= 0 && opt->buffer_output <= 2);
+	assert(opt->obuf.len == 0);
+
+	assert(opt->priv == NULL);
+
+	/* Sanity check on repo state; index must match head */
 	if (repo_index_has_changes(opt->repo, head, &sb)) {
 		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
 		    sb.buf);
diff --git a/merge-recursive.h b/merge-recursive.h
index 978847e672..d201ee80fb 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -27,7 +27,7 @@ struct merge_options {
 	} detect_directory_renames;
 	int rename_limit;
 	int rename_score;
-	int show_rename_progress;
+	int show_rename_progress : 1;
 
 	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
 	long xdl_opts;
-- 
2.23.0.rc2.32.g2123e9e4e4

[PATCH v3 24/24] merge-recursive: alphabetize include list

[Elijah Newren]

Other than cache.h which needs to appear first, and merge-recursive.h
which I want to be second so that we are more likely to notice if
merge-recursive.h has any missing includes, the rest of the list is
long and easier to look through if it's alphabetical.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index bc0da608c4..d3dc3d8a49 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -4,30 +4,31 @@
  * The thieves were Alex Riesen and Johannes Schindelin, in June/July 2006
  */
 #include "cache.h"
-#include "config.h"
+#include "merge-recursive.h"
+
 #include "advice.h"
-#include "lockfile.h"
-#include "cache-tree.h"
-#include "object-store.h"
-#include "repository.h"
-#include "commit.h"
+#include "alloc.h"
+#include "attr.h"
 #include "blob.h"
 #include "builtin.h"
-#include "tree-walk.h"
+#include "cache-tree.h"
+#include "commit.h"
+#include "commit-reach.h"
+#include "config.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "dir.h"
+#include "ll-merge.h"
+#include "lockfile.h"
+#include "object-store.h"
+#include "repository.h"
+#include "revision.h"
+#include "string-list.h"
+#include "submodule.h"
 #include "tag.h"
-#include "alloc.h"
+#include "tree-walk.h"
 #include "unpack-trees.h"
-#include "string-list.h"
 #include "xdiff-interface.h"
-#include "ll-merge.h"
-#include "attr.h"
-#include "merge-recursive.h"
-#include "dir.h"
-#include "submodule.h"
-#include "revision.h"
-#include "commit-reach.h"
 
 struct merge_options_internal {
 	int call_depth;
-- 
2.23.0.rc2.32.g2123e9e4e4

Re: [PATCH v3 23/24] merge-recursive: add sanity checks for relevant merge_options

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> There are lots of options that callers can set, yet most have a limited
> range of valid values, some options are meant for output (e.g.
> opt->obuf, which is expected to start empty), and callers are expected
> to not set opt->priv.  Add several sanity checks to ensure callers
> provide sane values.
> ...

The change to the struct does not seem to have much with the above
rationale.

> diff --git a/merge-recursive.h b/merge-recursive.h
> index 978847e672..d201ee80fb 100644
> --- a/merge-recursive.h
> +++ b/merge-recursive.h
> @@ -27,7 +27,7 @@ struct merge_options {
>  	} detect_directory_renames;
>  	int rename_limit;
>  	int rename_score;
> -	int show_rename_progress;
> +	int show_rename_progress : 1;

And a one-bit wide bitfield that is signed is always an iffy idea.

>  
>  	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
>  	long xdl_opts;

Re: [PATCH v3 23/24] merge-recursive: add sanity checks for relevant merge_options

[Junio C Hamano]

> diff --git a/merge-recursive.c b/merge-recursive.c
> index 647b1f25c3..bc0da608c4 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3620,6 +3620,29 @@ static int merge_start(struct merge_options *opt, struct tree *head)
>  ...
> +	assert(opt->buffer_output >= 0 && opt->buffer_output <= 2);

The field is unsigned, so >=0 side triggers "-Werror=type-limits" warning.

Material for squashing I have collected so far...

 cache-tree.c      | 2 +-
 merge-recursive.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/cache-tree.c b/cache-tree.c
index 00eda3e537..ef8c9f5e04 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -608,7 +608,7 @@ static struct cache_tree *cache_tree_find(struct cache_tree *it, const char *pat
 	return it;
 }
 
-int write_index_as_tree_internal(struct object_id *oid, struct index_state *index_state, int cache_tree_valid, int flags, const char *prefix)
+static int write_index_as_tree_internal(struct object_id *oid, struct index_state *index_state, int cache_tree_valid, int flags, const char *prefix)
 {
 	if (flags & WRITE_TREE_IGNORE_CACHE_TREE) {
 		cache_tree_free(&index_state->cache_tree);
diff --git a/merge-recursive.c b/merge-recursive.c
index d3dc3d8a49..3d126dcc48 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3638,7 +3638,7 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 	       opt->recursive_variant <= MERGE_VARIANT_THEIRS);
 
 	assert(opt->verbosity >= 0 && opt->verbosity <= 5);
-	assert(opt->buffer_output >= 0 && opt->buffer_output <= 2);
+	assert(opt->buffer_output <= 2);
 	assert(opt->obuf.len == 0);
 
 	assert(opt->priv == NULL);

Re: [PATCH v3 03/24] merge-recursive: enforce opt->ancestor != NULL when calling merge_trees()

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> We always want our conflict hunks to be labelled so that users can know
> where each came from.  The previous commit fixed the one caller in the
> codebase which was not setting opt->ancestor (and thus not providing a
> label for the "merge base" conflict hunk in diff3-style conflict
> markers); add an assertion to prevent future codepaths from also
> overlooking this requirement.

Makes sense.

Re: [PATCH v3 21/24] merge-recursive: split internal fields into a separate struct

[SZEDER Gábor]

On Thu, Aug 15, 2019 at 02:40:50PM -0700, Elijah Newren wrote:
> diff --git a/merge-recursive.c b/merge-recursive.c

>  static void merge_finalize(struct merge_options *opt)
>  {
>  	flush_output(opt);
> -	if (!opt->call_depth && opt->buffer_output < 2)
> +	if (!opt->priv->call_depth && opt->buffer_output < 2)
>  		strbuf_release(&opt->obuf);
>  	if (show(opt, 2))
>  		diff_warn_rename_limit("merge.renamelimit",
> -				       opt->needed_rename_limit, 0);
> +				       opt->priv->needed_rename_limit, 0);
> +	free(opt->priv);
> +	opt->priv = NULL;

Coccinelle suggests FREE_AND_NULL(opt->priv) here.

Re: [PATCH v3 04/24] merge-recursive: provide a better label for diff3 common ancestor

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> @@ -3507,6 +3507,11 @@ int merge_recursive(struct merge_options *opt,
>  	struct commit *merged_common_ancestors;
>  	struct tree *mrtree;
>  	int clean;
> +	int num_merge_bases;
> +	struct strbuf merge_base_abbrev = STRBUF_INIT;
> +
> +	if (!opt->call_depth)
> +		assert(opt->ancestor == NULL);

Hmph.  Do we have anything to say on this field when call_depth is
not zero?  Is it OK for opt->ancestor to be sometimes NULL and non
NULL some other times?

> @@ -3528,6 +3533,7 @@ int merge_recursive(struct merge_options *opt,
>  			output_commit_title(opt, iter->item);
>  	}
>  
> +	num_merge_bases = commit_list_count(ca);

Criss-cross merge with very large number of merge bases is rare, so
it is OK to count them all, even though we only care about "is it
zero, is it one, or is it two or more?"

I suspect this does not have to count, though, if we really wanted
to avoid counting.

>  	merged_common_ancestors = pop_commit(&ca);
>  	if (merged_common_ancestors == NULL) {
>  		/* if there is no common ancestor, use an empty tree */

Here is the case where we can already decide the ancestor name for
the later merge_trees() should be "empty tree".

And if merged_common_ancestors is not NULL, ca may have run out (in
which case, we only have a single merge base), or ca still has
another merge base (in which case, we have two or more).  So, if you
add
		ancestor_name = "empty tree";
	} else if (ca) {
		ancestor_name = "merged common ancestors";
	} else {
		ancestor_name = abbrev_name(merged_common_ancestors);
	}

to that if() statement above, that should be sufficient, no?

opt is used for inner merge in the for() loop, so you would probably
need another "char *" variable without contaminating opt->ancestor_name
at this point, and then assign the value in the temporary to the
opt->ancestor field where the original always assigned "merged
common ancestors".

> @@ -3568,10 +3574,23 @@ int merge_recursive(struct merge_options *opt,
>  	if (!opt->call_depth)
>  		repo_read_index(opt->repo);
>  
> -	opt->ancestor = "merged common ancestors";
> +	switch (num_merge_bases) {
> +	case 0:
> +		opt->ancestor = "<empty tree>";

Also, I do not see a reason why you want angle bra-ket pair around
"empty tree".  You are already using "merged common ancestors"
literal phrase without any special marker syntax.

Thanks.

Re: [PATCH v3 12/24] cache-tree: share code between functions writing an index as a tree

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> write_tree_from_memory() appeared to be a merge-recursive special that
> basically duplicated write_index_as_tree().  The two have a different
> signature, but the bigger difference was just that write_index_as_tree()
> would always unconditionally read the index off of disk instead of
> working on the current in-memory index.  So:
>
>   * split out common code into write_index_as_tree_internal()
>
>   * rename write_tree_from_memory() to write_inmemory_index_as_tree(),

Somewhat minor, but I find "inmemory_index" hard to see while
scanning the patch.  Perhaps call it "in_core_index" instead?

I originally started the above with "Very minor, ...", but as this
is exposed to the public in a header file, the name matters a bit
more than that.

Re: [PATCH v3 23/24] merge-recursive: add sanity checks for relevant merge_options

[Elijah Newren]

On Fri, Aug 16, 2019 at 12:52 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > There are lots of options that callers can set, yet most have a limited
> > range of valid values, some options are meant for output (e.g.
> > opt->obuf, which is expected to start empty), and callers are expected
> > to not set opt->priv.  Add several sanity checks to ensure callers
> > provide sane values.
> > ...
>
> The change to the struct does not seem to have much with the above
> rationale.

If the only possible values are 0 and 1, I can either add assertions
to check that at run time, or make the compiler check it for us by
confining its value to a single bit.  A compile-time check seems more
robust...

> > diff --git a/merge-recursive.h b/merge-recursive.h
> > index 978847e672..d201ee80fb 100644
> > --- a/merge-recursive.h
> > +++ b/merge-recursive.h
> > @@ -27,7 +27,7 @@ struct merge_options {
> >       } detect_directory_renames;
> >       int rename_limit;
> >       int rename_score;
> > -     int show_rename_progress;
> > +     int show_rename_progress : 1;
>
> And a one-bit wide bitfield that is signed is always an iffy idea.

...assuming of course I don't mess it up like this, not noticing that
it's an int that needs to be changed to an unsigned.  I'll fix this
up.

But the fact that you flagged the struct change -- would you prefer
some commit message explanation of how it's related, or was it more
the case that you felt it was a different kind of change and wanted it
split out into a separate patch?  I'm suspecting the former but am not
quite sure.

Re: [PATCH v3 23/24] merge-recursive: add sanity checks for relevant merge_options

[Elijah Newren]

On Fri, Aug 16, 2019 at 12:59 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> > diff --git a/merge-recursive.c b/merge-recursive.c
> > index 647b1f25c3..bc0da608c4 100644
> > --- a/merge-recursive.c
> > +++ b/merge-recursive.c
> > @@ -3620,6 +3620,29 @@ static int merge_start(struct merge_options *opt, struct tree *head)
> >  ...
> > +     assert(opt->buffer_output >= 0 && opt->buffer_output <= 2);
>
> The field is unsigned, so >=0 side triggers "-Werror=type-limits" warning.
>
> Material for squashing I have collected so far...
<snip>

Thanks, will include all of this in my next round.

Re: [PATCH v3 18/24] merge-recursive: consolidate unnecessary fields in merge_options

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

>  static inline int merge_detect_rename(struct merge_options *opt)
>  {
> -	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
> -		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
> +	return (opt->detect_renames != -1) ? opt->detect_renames : 1;
>  }

Every time I see "is it not negative?" (or more generally "is it in
this range?") converted to "is it not this exact value?", it makes
me feel uneasy.

> -	opts.rename_limit = opt->merge_rename_limit >= 0 ? opt->merge_rename_limit :
> -			    opt->diff_rename_limit >= 0 ? opt->diff_rename_limit :
> -			    1000;
> +	opts.rename_limit = (opt->rename_limit != -1) ? opt->rename_limit : 1000;

Likewise.  I have no objection to merging two rename-limit to a
single field (and two detect-renames to a single field).

> @@ -3732,14 +3729,14 @@ static void merge_recursive_config(struct merge_options *opt)
>  {
>  	char *value = NULL;
>  	git_config_get_int("merge.verbosity", &opt->verbosity);
> -	git_config_get_int("diff.renamelimit", &opt->diff_rename_limit);
> -	git_config_get_int("merge.renamelimit", &opt->merge_rename_limit);
> +	git_config_get_int("diff.renamelimit", &opt->rename_limit);
> +	git_config_get_int("merge.renamelimit", &opt->rename_limit);

Hmph.  If merge.renameLimit is there, that would overwrite whatever
we get by reading from diff.renameLimit, so the two fields with
runtime precedence order can easily be replaced by these two calls.

Nice.

If you have "[diff] renamelimit = -2" in your $GIT_DIR/config, would
we change behaviour due to the earlier conversion that has nothing
to do with the theme of this step (i.e. consolidate two variables
into one)?

> @@ -3765,11 +3762,9 @@ void init_merge_options(struct merge_options *opt,
>  	opt->repo = repo;
>  	opt->verbosity = 2;
>  	opt->buffer_output = 1;
> -	opt->diff_rename_limit = -1;
> -	opt->merge_rename_limit = -1;
> +	opt->rename_limit = -1;
>  	opt->renormalize = 0;
> -	opt->diff_detect_rename = -1;
> -	opt->merge_detect_rename = -1;
> +	opt->detect_renames = -1;
>  	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
>  	merge_recursive_config(opt);
>  	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
> @@ -3821,16 +3816,16 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
>  	else if (!strcmp(s, "no-renormalize"))
>  		opt->renormalize = 0;
>  	else if (!strcmp(s, "no-renames"))
> -		opt->merge_detect_rename = 0;
> +		opt->detect_renames = 0;
>  	else if (!strcmp(s, "find-renames")) {
> -		opt->merge_detect_rename = 1;
> +		opt->detect_renames = 1;
>  		opt->rename_score = 0;
>  	}
>  	else if (skip_prefix(s, "find-renames=", &arg) ||
>  		 skip_prefix(s, "rename-threshold=", &arg)) {
>  		if ((opt->rename_score = parse_rename_score(&arg)) == -1 || *arg != 0)
>  			return -1;
> -		opt->merge_detect_rename = 1;
> +		opt->detect_renames = 1;
>  	}
>  	/*
>  	 * Please update $__git_merge_strategy_options in
> diff --git a/merge-recursive.h b/merge-recursive.h
> index 0fdae904dd..f4bdfbc897 100644
> --- a/merge-recursive.h
> +++ b/merge-recursive.h
> @@ -27,10 +27,8 @@ struct merge_options {
>  		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
>  		MERGE_DIRECTORY_RENAMES_TRUE = 2
>  	} detect_directory_renames;
> -	int diff_detect_rename;
> -	int merge_detect_rename;
> -	int diff_rename_limit;
> -	int merge_rename_limit;
> +	int detect_renames;
> +	int rename_limit;
>  	int rename_score;
>  	int needed_rename_limit;
>  	int show_rename_progress;

Re: [PATCH v3 21/24] merge-recursive: split internal fields into a separate struct

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> merge_options has several internal fields that should not be set or read
> by external callers.  This just complicates the API.  Move them into an
> opaque merge_options_internal struct that is defined only in
> merge-recursive.c and keep these out of merge-recursive.h.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c | 186 ++++++++++++++++++++++++----------------------
>  merge-recursive.h |  17 ++---
>  2 files changed, 106 insertions(+), 97 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index b4334d0506..840b09f1dc 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3617,17 +3627,21 @@ static int merge_start(struct merge_options *opt, struct tree *he> ...
> -				       opt->needed_rename_limit, 0);
> +				       opt->priv->needed_rename_limit, 0);
> +	free(opt->priv);
> +	opt->priv = NULL;

This gets hit by Cocci.

Re: [PATCH v3 04/24] merge-recursive: provide a better label for diff3 common ancestor

[Elijah Newren]

On Fri, Aug 16, 2019 at 2:33 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > @@ -3507,6 +3507,11 @@ int merge_recursive(struct merge_options *opt,
> >       struct commit *merged_common_ancestors;
> >       struct tree *mrtree;
> >       int clean;
> > +     int num_merge_bases;
> > +     struct strbuf merge_base_abbrev = STRBUF_INIT;
> > +
> > +     if (!opt->call_depth)
> > +             assert(opt->ancestor == NULL);
>
> Hmph.  Do we have anything to say on this field when call_depth is
> not zero?  Is it OK for opt->ancestor to be sometimes NULL and non
> NULL some other times?

I was specifically trying to add a check for external callers of
merge_recursive() to make sure they called it correctly.  Since
merge_recursive() sets opt->ancestor before calling itself
recursively, I had to hide the assertion behind an if-check, namely on
call_depth.

We could add an assertion that opt->ancestor != NULL when
opt->call_depth > 0, but it seemed odd to document pre-conditions for
how merge_recursive() calls itself.  Anyway, this code block actually
becomes a bit cleaner later in the series when I create  separate
merge_recursive() and merge_recursive_internal() functions, as the
assertion can just go into merge_recursive() and not be protected by
the opt->call_depth check.

> > @@ -3528,6 +3533,7 @@ int merge_recursive(struct merge_options *opt,
> >                       output_commit_title(opt, iter->item);
> >       }
> >
> > +     num_merge_bases = commit_list_count(ca);
>
> Criss-cross merge with very large number of merge bases is rare, so
> it is OK to count them all, even though we only care about "is it
> zero, is it one, or is it two or more?"
>
> I suspect this does not have to count, though, if we really wanted
> to avoid counting.
>
> >       merged_common_ancestors = pop_commit(&ca);
> >       if (merged_common_ancestors == NULL) {
> >               /* if there is no common ancestor, use an empty tree */
>
> Here is the case where we can already decide the ancestor name for
> the later merge_trees() should be "empty tree".
>
> And if merged_common_ancestors is not NULL, ca may have run out (in
> which case, we only have a single merge base), or ca still has
> another merge base (in which case, we have two or more).  So, if you
> add
>                 ancestor_name = "empty tree";
>         } else if (ca) {
>                 ancestor_name = "merged common ancestors";
>         } else {
>                 ancestor_name = abbrev_name(merged_common_ancestors);
>         }
>
> to that if() statement above, that should be sufficient, no?
>
> opt is used for inner merge in the for() loop, so you would probably
> need another "char *" variable without contaminating opt->ancestor_name
> at this point, and then assign the value in the temporary to the
> opt->ancestor field where the original always assigned "merged
> common ancestors".

Sure, I can make these changes.

> > @@ -3568,10 +3574,23 @@ int merge_recursive(struct merge_options *opt,
> >       if (!opt->call_depth)
> >               repo_read_index(opt->repo);
> >
> > -     opt->ancestor = "merged common ancestors";
> > +     switch (num_merge_bases) {
> > +     case 0:
> > +             opt->ancestor = "<empty tree>";
>
> Also, I do not see a reason why you want angle bra-ket pair around
> "empty tree".  You are already using "merged common ancestors"
> literal phrase without any special marker syntax.

Oh good point; will drop.

Re: [PATCH v3 12/24] cache-tree: share code between functions writing an index as a tree

[Elijah Newren]

On Fri, Aug 16, 2019 at 3:01 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > write_tree_from_memory() appeared to be a merge-recursive special that
> > basically duplicated write_index_as_tree().  The two have a different
> > signature, but the bigger difference was just that write_index_as_tree()
> > would always unconditionally read the index off of disk instead of
> > working on the current in-memory index.  So:
> >
> >   * split out common code into write_index_as_tree_internal()
> >
> >   * rename write_tree_from_memory() to write_inmemory_index_as_tree(),
>
> Somewhat minor, but I find "inmemory_index" hard to see while
> scanning the patch.  Perhaps call it "in_core_index" instead?
>
> I originally started the above with "Very minor, ...", but as this
> is exposed to the public in a header file, the name matters a bit
> more than that.

Sounds good; will change.

Re: [PATCH v3 18/24] merge-recursive: consolidate unnecessary fields in merge_options

[Elijah Newren]

On Fri, Aug 16, 2019 at 3:14 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> >  static inline int merge_detect_rename(struct merge_options *opt)
> >  {
> > -     return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
> > -             opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
> > +     return (opt->detect_renames != -1) ? opt->detect_renames : 1;
> >  }
>
> Every time I see "is it not negative?" (or more generally "is it in
> this range?") converted to "is it not this exact value?", it makes
> me feel uneasy.
>
> > -     opts.rename_limit = opt->merge_rename_limit >= 0 ? opt->merge_rename_limit :
> > -                         opt->diff_rename_limit >= 0 ? opt->diff_rename_limit :
> > -                         1000;
> > +     opts.rename_limit = (opt->rename_limit != -1) ? opt->rename_limit : 1000;
>
> Likewise.  I have no objection to merging two rename-limit to a
> single field (and two detect-renames to a single field).
>
> > @@ -3732,14 +3729,14 @@ static void merge_recursive_config(struct merge_options *opt)
> >  {
> >       char *value = NULL;
> >       git_config_get_int("merge.verbosity", &opt->verbosity);
> > -     git_config_get_int("diff.renamelimit", &opt->diff_rename_limit);
> > -     git_config_get_int("merge.renamelimit", &opt->merge_rename_limit);
> > +     git_config_get_int("diff.renamelimit", &opt->rename_limit);
> > +     git_config_get_int("merge.renamelimit", &opt->rename_limit);
>
> Hmph.  If merge.renameLimit is there, that would overwrite whatever
> we get by reading from diff.renameLimit, so the two fields with
> runtime precedence order can easily be replaced by these two calls.
>
> Nice.
>
> If you have "[diff] renamelimit = -2" in your $GIT_DIR/config, would
> we change behaviour due to the earlier conversion that has nothing
> to do with the theme of this step (i.e. consolidate two variables
> into one)?

At the end of this series, the "merge-recursive: add sanity checks for
relevant merge_options" commit adds some assertions that would fail if
someone passed such a value, regardless of whether this patch was
included or not.  (Are we worried about people having such a config
value and should we support it?  It goes against the documented
values, but I guess someone could have set it anyway even if it seems
odd to set a value that says, "give me whatever the default is.")

If we tried with this specific commit, though, then: diffcore-rename.c
checks for rename_limit <= 0 and sets the value to 32767 in that case,
so it'd have the effect of extending the default merge-related rename
limit.  As far as detect_rename, diff.c treats that as a boolean in
most places (e.g. "if (options->detect_rename)"), and specifically
compares against DIFF_DETECT_COPY in places where copy detection are
relevant.  So, detect_rename would behave the same.

All of that said, I'm happy to restore the is-not-negative checks.

Re: [PATCH v3 21/24] merge-recursive: split internal fields into a separate struct

[Elijah Newren]

On Fri, Aug 16, 2019 at 2:19 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>
> On Thu, Aug 15, 2019 at 02:40:50PM -0700, Elijah Newren wrote:
> > diff --git a/merge-recursive.c b/merge-recursive.c
>
> >  static void merge_finalize(struct merge_options *opt)
> >  {
> >       flush_output(opt);
> > -     if (!opt->call_depth && opt->buffer_output < 2)
> > +     if (!opt->priv->call_depth && opt->buffer_output < 2)
> >               strbuf_release(&opt->obuf);
> >       if (show(opt, 2))
> >               diff_warn_rename_limit("merge.renamelimit",
> > -                                    opt->needed_rename_limit, 0);
> > +                                    opt->priv->needed_rename_limit, 0);
> > +     free(opt->priv);
> > +     opt->priv = NULL;
>
> Coccinelle suggests FREE_AND_NULL(opt->priv) here.

Thanks; will fix.

Re: [PATCH v3 23/24] merge-recursive: add sanity checks for relevant merge_options

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> If the only possible values are 0 and 1, I can either add assertions
> to check that at run time, or make the compiler check it for us by
> confining its value to a single bit.  A compile-time check seems more
> robust...

Sure, as long as they can catch assignments (e.g. ".field = 2",
or more interestingly ".field = .field + 1" in a loop, etc.) and
increments or decrements and flag them.

> But the fact that you flagged the struct change -- would you prefer
> some commit message explanation of how it's related, or was it more
> the case that you felt it was a different kind of change and wanted it
> split out into a separate patch?  I'm suspecting the former but am not
> quite sure.

I do not see it as related at all, so either split it out into a
separate patch, or just drop it (and have a runtime check as
everybody else in this step), would be the sensible alternatives.  I
think the latter is easier to reason about but it may be just me.

Re: [PATCH v3 18/24] merge-recursive: consolidate unnecessary fields in merge_options

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> At the end of this series, the "merge-recursive: add sanity checks for
> relevant merge_options" commit adds some assertions that would fail if
> someone passed such a value, regardless of whether this patch was
> included or not.  (Are we worried about people having such a config
> value and should we support it?  It goes against the documented
> values, but I guess someone could have set it anyway even if it seems
> odd to set a value that says, "give me whatever the default is.")

I am somewhat worried about changing the rule on the users without
telling them (and without having a good reason to enforce a tighter
version retroactively).

If we are formally forbidding "[diff] renameLimit = -1" and other
out-of-bound values, (1) assert() will most often turn into noop in
non-debug builds, so the last step of this series may not be such a
good sanity check anyway, (2) "if (condition) BUG();" may be better,
but that's for catching programming errors that made the condition
hold and configuration the end-user had should not trigger it.

So if we want to really catch bogus end-user-supplied values (and we
really do---but the definition of "bogus" may be debatable), the
place to do so is not the "sanity-check at the end", but where we
call get_config_int() and friends.

[PATCH v4 00/24] Clean up merge API

[Elijah Newren]

Before writing a replacement merge strategy for recursive, I decided to
first cleanup the merge API -- streamlining merge-recursive.h and making
it more readable.  This includes some minor fixes found along the way.

Changes since v3 (full range-diff below):
  * Addressed feedback from Junio and SZEDER:
    * [Patch 4] Avoided looping over commit_list in determining
                ancestor_name
    * [Patch 12] s/inmemory/in_core/ in new cache-tree function name
    * [Patch 18] restored the '>= 0' comparision
    * [Patch 21] Use FREE_AND_NULL
    * [Patch 23] Use range checks for show_rename_progress (don't use
                 a bitfield), and don't compare an unsigned using >= 0.

Notes:
  * Patches 7, 9, 13, 14, and 15 only changed in the range-diff due to
    context changes from other patches

Derrick Stolee (1):
  merge-recursive: introduce an enum for detect_directory_renames values

Elijah Newren (23):
  merge-recursive: be consistent with assert
  checkout: provide better conflict hunk description with detached HEAD
  merge-recursive: enforce opt->ancestor != NULL when calling
    merge_trees()
  merge-recursive: provide a better label for diff3 common ancestor
  merge-recursive: future-proof update_file_flags() against memory leaks
  merge-recursive: remove another implicit dependency on the_repository
  Ensure index matches head before invoking merge machinery, round N
  merge-recursive: exit early if index != head
  merge-recursive: remove useless parameter in merge_trees()
  merge-recursive: don't force external callers to do our logging
  cache-tree: share code between functions writing an index as a tree
  merge-recursive: fix some overly long lines
  merge-recursive: use common name for ancestors/common/base_list
  merge-recursive: rename 'mrtree' to 'result_tree', for clarity
  merge-recursive: rename merge_options argument to opt in header
  merge-recursive: move some definitions around to clean up the header
  merge-recursive: consolidate unnecessary fields in merge_options
  merge-recursive: comment and reorder the merge_options fields
  merge-recursive: avoid losing output and leaking memory holding that
    output
  merge-recursive: split internal fields into a separate struct
  merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
  merge-recursive: add sanity checks for relevant merge_options
  merge-recursive: alphabetize include list

 builtin/am.c                      |   2 +-
 builtin/checkout.c                |  14 +-
 builtin/merge-recursive.c         |   4 +
 builtin/stash.c                   |   2 +
 cache-tree.c                      |  85 +++--
 cache-tree.h                      |   3 +-
 merge-recursive.c                 | 566 ++++++++++++++++++------------
 merge-recursive.h                 | 164 +++++----
 sequencer.c                       |   5 +-
 t/t3030-merge-recursive.sh        |   9 +-
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 189 ++++++++++
 12 files changed, 717 insertions(+), 334 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

Range-diff:
 1:  21f1e04dc9 =  1:  7f99c2401e merge-recursive: be consistent with assert
 2:  ac24702773 =  2:  b305348f45 checkout: provide better conflict hunk description with detached HEAD
 3:  fd14ed9490 =  3:  5b9e32a356 merge-recursive: enforce opt->ancestor != NULL when calling merge_trees()
 4:  540a1d17d7 !  4:  566a55cf76 merge-recursive: provide a better label for diff3 common ancestor
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      	struct commit *merged_common_ancestors;
      	struct tree *mrtree;
      	int clean;
    -+	int num_merge_bases;
    ++	const char *ancestor_name;
     +	struct strbuf merge_base_abbrev = STRBUF_INIT;
     +
     +	if (!opt->call_depth)
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      	if (show(opt, 4)) {
      		output(opt, 4, _("Merging:"));
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    - 			output_commit_title(opt, iter->item);
    + 
    + 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
    + 		merged_common_ancestors = make_virtual_commit(opt->repo, tree, "ancestor");
    ++		ancestor_name = "empty tree";
    ++	} else if (ca) {
    ++		ancestor_name = "merged common ancestors";
    ++	} else {
    ++		strbuf_add_unique_abbrev(&merge_base_abbrev,
    ++					 &merged_common_ancestors->object.oid,
    ++					 DEFAULT_ABBREV);
    ++		ancestor_name = merge_base_abbrev.buf;
      	}
      
    -+	num_merge_bases = commit_list_count(ca);
    - 	merged_common_ancestors = pop_commit(&ca);
    - 	if (merged_common_ancestors == NULL) {
    - 		/* if there is no common ancestor, use an empty tree */
    + 	for (iter = ca; iter; iter = iter->next) {
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      	if (!opt->call_depth)
      		repo_read_index(opt->repo);
      
     -	opt->ancestor = "merged common ancestors";
    -+	switch (num_merge_bases) {
    -+	case 0:
    -+		opt->ancestor = "<empty tree>";
    -+		break;
    -+	case 1:
    -+		strbuf_add_unique_abbrev(&merge_base_abbrev,
    -+					 &merged_common_ancestors->object.oid,
    -+					 DEFAULT_ABBREV);
    -+		opt->ancestor = merge_base_abbrev.buf;
    -+		break;
    -+	default:
    -+		opt->ancestor = "merged common ancestors";
    -+	}
    ++	opt->ancestor = ancestor_name;
      	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
      			    get_commit_tree(merged_common_ancestors),
      			    &mrtree);
    @@ t/t6047-diff3-conflict-markers.sh (new)
     +
     +		test_must_fail git -c merge.conflictstyle=diff3 merge --allow-unrelated-histories -s recursive R^0 &&
     +
    -+		grep "|||||| <empty tree>" content
    ++		grep "|||||| empty tree" content
     +	)
     +'
     +
 5:  19ff6a9503 =  5:  9c0bab2a0b merge-recursive: introduce an enum for detect_directory_renames values
 6:  5e44146da1 =  6:  2cd1a60a16 merge-recursive: future-proof update_file_flags() against memory leaks
 7:  df210eb029 !  7:  27a339781f merge-recursive: remove another implicit dependency on the_repository
    @@ Commit message
     
      ## merge-recursive.c ##
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    - 	default:
    - 		opt->ancestor = "merged common ancestors";
    - 	}
    + 		repo_read_index(opt->repo);
    + 
    + 	opt->ancestor = ancestor_name;
     -	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
     -			    get_commit_tree(merged_common_ancestors),
     +	clean = merge_trees(opt,
 8:  74dd7b8f59 =  8:  133c470371 Ensure index matches head before invoking merge machinery, round N
 9:  f04eba4184 !  9:  f43a229ae6 merge-recursive: exit early if index != head
    @@ merge-recursive.c: static struct commit_list *reverse_commit_list(struct commit_
      	struct commit_list *iter;
      	struct commit *merged_common_ancestors;
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    - 	int num_merge_bases;
    + 	const char *ancestor_name;
      	struct strbuf merge_base_abbrev = STRBUF_INIT;
      
     -	if (!opt->call_depth)
    @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      			return -1;
      		opt->branch1 = saved_b1;
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
    - 	default:
    - 		opt->ancestor = "merged common ancestors";
    - 	}
    + 		repo_read_index(opt->repo);
    + 
    + 	opt->ancestor = ancestor_name;
     -	clean = merge_trees(opt,
     -			    repo_get_commit_tree(opt->repo, h1),
     -			    repo_get_commit_tree(opt->repo, h2),
10:  8688d40739 = 10:  678dfbf507 merge-recursive: remove useless parameter in merge_trees()
11:  a92d460707 = 11:  7b28da5637 merge-recursive: don't force external callers to do our logging
12:  dec0ea7409 ! 12:  c1ae5cc1c4 cache-tree: share code between functions writing an index as a tree
    @@ builtin/checkout.c: static int merge_working_tree(const struct checkout_opts *op
      			init_merge_options(&o, the_repository);
      			o.verbosity = 0;
     -			work = write_tree_from_memory(&o);
    -+			work = write_inmemory_index_as_tree(the_repository);
    ++			work = write_in_core_index_as_tree(the_repository);
      
      			ret = reset_tree(new_tree,
      					 opts, 1,
    @@ cache-tree.c: static struct cache_tree *cache_tree_find(struct cache_tree *it, c
      	return it;
      }
      
    -+int write_index_as_tree_internal(struct object_id *oid, struct index_state *index_state, int cache_tree_valid, int flags, const char *prefix)
    ++static int write_index_as_tree_internal(struct object_id *oid,
    ++					struct index_state *index_state,
    ++					int cache_tree_valid,
    ++					int flags,
    ++					const char *prefix)
     +{
     +	if (flags & WRITE_TREE_IGNORE_CACHE_TREE) {
     +		cache_tree_free(&index_state->cache_tree);
    @@ cache-tree.c: static struct cache_tree *cache_tree_find(struct cache_tree *it, c
     +	return 0;
     +}
     +
    -+struct tree* write_inmemory_index_as_tree(struct repository *repo) {
    ++struct tree* write_in_core_index_as_tree(struct repository *repo) {
     +	struct object_id o;
     +	int was_valid, ret;
     +
    @@ cache-tree.h: void cache_tree_verify(struct repository *, struct index_state *);
      #define WRITE_TREE_UNMERGED_INDEX (-2)
      #define WRITE_TREE_PREFIX_ERROR (-3)
      
    -+struct tree* write_inmemory_index_as_tree(struct repository *repo);
    ++struct tree* write_in_core_index_as_tree(struct repository *repo);
      int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix);
      void prime_cache_tree(struct repository *, struct index_state *, struct tree *);
      
    @@ merge-recursive.c: static int merge_trees_internal(struct merge_options *opt,
      
     -	if (opt->call_depth && !(*result = write_tree_from_memory(opt)))
     +	if (opt->call_depth &&
    -+	    !(*result = write_inmemory_index_as_tree(opt->repo)))
    ++	    !(*result = write_in_core_index_as_tree(opt->repo)))
      		return -1;
      
      	return clean;
13:  b51f3d1924 ! 13:  2e9fb223d9 merge-recursive: fix some overly long lines
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     -		merged_common_ancestors = make_virtual_commit(opt->repo, tree, "ancestor");
     +		merged_common_ancestors = make_virtual_commit(opt->repo,
     +							      tree, "ancestor");
    - 	}
    - 
    - 	for (iter = ca; iter; iter = iter->next) {
    + 		ancestor_name = "empty tree";
    + 	} else if (ca) {
    + 		ancestor_name = "merged common ancestors";
     @@ merge-recursive.c: int merge_recursive(struct merge_options *opt,
      	return clean;
      }
14:  a069cc4cca ! 14:  230d903012 merge-recursive: use common name for ancestors/common/base_list
    @@ merge-recursive.c: static struct commit_list *reverse_commit_list(struct commit_
     +	struct commit *merged_merge_bases;
      	struct tree *mrtree;
      	int clean;
    - 	int num_merge_bases;
    + 	const char *ancestor_name;
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      		output_commit_title(opt, h2);
      	}
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
      			output_commit_title(opt, iter->item);
      	}
      
    --	num_merge_bases = commit_list_count(ca);
     -	merged_common_ancestors = pop_commit(&ca);
     -	if (merged_common_ancestors == NULL) {
    -+	num_merge_bases = commit_list_count(merge_bases);
     +	merged_merge_bases = pop_commit(&merge_bases);
     +	if (merged_merge_bases == NULL) {
      		/* if there is no common ancestor, use an empty tree */
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     -							      tree, "ancestor");
     +		merged_merge_bases = make_virtual_commit(opt->repo, tree,
     +							 "ancestor");
    + 		ancestor_name = "empty tree";
    +-	} else if (ca) {
    ++	} else if (merge_bases) {
    + 		ancestor_name = "merged common ancestors";
    + 	} else {
    + 		strbuf_add_unique_abbrev(&merge_base_abbrev,
    +-					 &merged_common_ancestors->object.oid,
    ++					 &merged_merge_bases->object.oid,
    + 					 DEFAULT_ABBREV);
    + 		ancestor_name = merge_base_abbrev.buf;
      	}
      
     -	for (iter = ca; iter; iter = iter->next) {
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
      			return err(opt, _("merge returned no commit"));
      	}
      
    -@@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
    - 		break;
    - 	case 1:
    - 		strbuf_add_unique_abbrev(&merge_base_abbrev,
    --					 &merged_common_ancestors->object.oid,
    -+					 &merged_merge_bases->object.oid,
    - 					 DEFAULT_ABBREV);
    - 		opt->ancestor = merge_base_abbrev.buf;
    - 		break;
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      				     repo_get_commit_tree(opt->repo, h1),
      				     repo_get_commit_tree(opt->repo, h2),
15:  93a3ce6b88 ! 15:  28c20a2d08 merge-recursive: rename 'mrtree' to 'result_tree', for clarity
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     -	struct tree *mrtree;
     +	struct tree *result_tree;
      	int clean;
    - 	int num_merge_bases;
    + 	const char *ancestor_name;
      	struct strbuf merge_base_abbrev = STRBUF_INIT;
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      				     repo_get_commit_tree(opt->repo, h2),
16:  1b1df10c11 = 16:  50e2e51cf6 merge-recursive: rename merge_options argument to opt in header
17:  1526977a85 = 17:  b8cd111bd2 merge-recursive: move some definitions around to clean up the header
18:  c90f2f15cd ! 18:  9bb8a7f162 merge-recursive: consolidate unnecessary fields in merge_options
    @@ merge-recursive.c: static int add_cacheinfo(struct merge_options *opt,
      {
     -	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
     -		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
    -+	return (opt->detect_renames != -1) ? opt->detect_renames : 1;
    ++	return (opt->detect_renames >= 0) ? opt->detect_renames : 1;
      }
      
      static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
    @@ merge-recursive.c: static struct diff_queue_struct *get_diffpairs(struct merge_o
     -	opts.rename_limit = opt->merge_rename_limit >= 0 ? opt->merge_rename_limit :
     -			    opt->diff_rename_limit >= 0 ? opt->diff_rename_limit :
     -			    1000;
    -+	opts.rename_limit = (opt->rename_limit != -1) ? opt->rename_limit : 1000;
    ++	opts.rename_limit = (opt->rename_limit >= 0) ? opt->rename_limit : 1000;
      	opts.rename_score = opt->rename_score;
      	opts.show_rename_progress = opt->show_rename_progress;
      	opts.output_format = DIFF_FORMAT_NO_OUTPUT;
19:  6d930dba72 = 19:  2140490796 merge-recursive: comment and reorder the merge_options fields
20:  ec3e15f6a8 = 20:  02478e2088 merge-recursive: avoid losing output and leaking memory holding that output
21:  7edfac7048 ! 21:  2781ca78d7 merge-recursive: split internal fields into a separate struct
    @@ merge-recursive.c: static int merge_trees_internal(struct merge_options *opt,
      
     -	if (opt->call_depth &&
     +	if (opt->priv->call_depth &&
    - 	    !(*result = write_inmemory_index_as_tree(opt->repo)))
    + 	    !(*result = write_in_core_index_as_tree(opt->repo)))
      		return -1;
      
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
    @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt
     +	if (!opt->priv->call_depth)
      		repo_read_index(opt->repo);
      
    - 	switch (num_merge_bases) {
    + 	opt->ancestor = ancestor_name;
     @@ merge-recursive.c: static int merge_recursive_internal(struct merge_options *opt,
      		return clean;
      	}
    @@ merge-recursive.c: static int merge_start(struct merge_options *opt, struct tree
      		diff_warn_rename_limit("merge.renamelimit",
     -				       opt->needed_rename_limit, 0);
     +				       opt->priv->needed_rename_limit, 0);
    -+	free(opt->priv);
    -+	opt->priv = NULL;
    ++	FREE_AND_NULL(opt->priv);
      }
      
      int merge_trees(struct merge_options *opt,
22:  9a381873c2 = 22:  5a52aa786e merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
23:  c6bc8a196f ! 23:  b7e6c2436a merge-recursive: add sanity checks for relevant merge_options
    @@ merge-recursive.c: static int merge_start(struct merge_options *opt, struct tree
     +	       opt->detect_directory_renames <= MERGE_DIRECTORY_RENAMES_TRUE);
     +	assert(opt->rename_limit >= -1);
     +	assert(opt->rename_score >= 0 && opt->rename_score <= MAX_SCORE);
    ++	assert(opt->show_rename_progress >= 0 && opt->show_rename_progress <= 1);
     +
     +	assert(opt->xdl_opts >= 0);
     +	assert(opt->recursive_variant >= MERGE_VARIANT_NORMAL &&
     +	       opt->recursive_variant <= MERGE_VARIANT_THEIRS);
     +
     +	assert(opt->verbosity >= 0 && opt->verbosity <= 5);
    -+	assert(opt->buffer_output >= 0 && opt->buffer_output <= 2);
    ++	assert(opt->buffer_output <= 2);
     +	assert(opt->obuf.len == 0);
     +
     +	assert(opt->priv == NULL);
    @@ merge-recursive.c: static int merge_start(struct merge_options *opt, struct tree
      	if (repo_index_has_changes(opt->repo, head, &sb)) {
      		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
      		    sb.buf);
    -
    - ## merge-recursive.h ##
    -@@ merge-recursive.h: struct merge_options {
    - 	} detect_directory_renames;
    - 	int rename_limit;
    - 	int rename_score;
    --	int show_rename_progress;
    -+	int show_rename_progress : 1;
    - 
    - 	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
    - 	long xdl_opts;
24:  2123e9e4e4 = 24:  6ac9e42a3e merge-recursive: alphabetize include list
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 01/24] merge-recursive: be consistent with assert

[Elijah Newren]

In commit 8daec1df03de ("merge-recursive: switch from (oid,mode) pairs
to a diff_filespec", 2019-04-05), an assertion on a->path && b->path
was added for code readability to document that these both needed to be
non-NULL at this point in the code.  However, the subsequent lines also
read o->path, so it should be included in the assert.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 6b812d67e3..1d960fa64b 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1043,7 +1043,7 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path);
+	assert(a->path && b->path && o->path);
 	if (strcmp(a->path, b->path) ||
 	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
 		base_name = opt->ancestor == NULL ? NULL :
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 02/24] checkout: provide better conflict hunk description with detached HEAD

[Elijah Newren]

When running 'git checkout -m' and using diff3 style conflict markers,
we want all the conflict hunks (left-side, "common" or "merge base", and
right-side) to have label markers letting the user know where each came
from.  The "common" hunk label (o.ancestor) came from
old_branch_info->name, but that is NULL when HEAD is detached, which
resulted in a blank label.  Check for that case and provide an
abbreviated commit hash instead.

(Incidentally, this was the only case in the git codebase where
merge_trees() was called with opt->ancestor being NULL.  A subsequent
commit will prevent similar problems by enforcing that merge_trees()
always be called with opt->ancestor != NULL.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 6123f732a2..d5b946dc3a 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -713,6 +713,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			struct tree *old_tree;
 			struct merge_options o;
 			struct strbuf sb = STRBUF_INIT;
+			struct strbuf old_commit_shortname = STRBUF_INIT;
 
 			if (!opts->merge)
 				return 1;
@@ -768,6 +769,12 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			if (ret)
 				return ret;
 			o.ancestor = old_branch_info->name;
+			if (old_branch_info->name == NULL) {
+				strbuf_add_unique_abbrev(&old_commit_shortname,
+							 &old_branch_info->commit->object.oid,
+							 DEFAULT_ABBREV);
+				o.ancestor = old_commit_shortname.buf;
+			}
 			o.branch1 = new_branch_info->name;
 			o.branch2 = "local";
 			ret = merge_trees(&o,
@@ -781,6 +788,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 					 opts, 0,
 					 writeout_error);
 			strbuf_release(&o.obuf);
+			strbuf_release(&old_commit_shortname);
 			if (ret)
 				return ret;
 		}
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 03/24] merge-recursive: enforce opt->ancestor != NULL when calling merge_trees()

[Elijah Newren]

We always want our conflict hunks to be labelled so that users can know
where each came from.  The previous commit fixed the one caller in the
codebase which was not setting opt->ancestor (and thus not providing a
label for the "merge base" conflict hunk in diff3-style conflict
markers); add an assertion to prevent future codepaths from also
overlooking this requirement.

Enforcing this requirement also allows us to simplify the code for
labelling the conflict hunks by no longer checking if the ancestor label
is NULL.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1d960fa64b..a67ea4957a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1019,7 +1019,7 @@ static int merge_3way(struct merge_options *opt,
 {
 	mmfile_t orig, src1, src2;
 	struct ll_merge_options ll_opts = {0};
-	char *base_name, *name1, *name2;
+	char *base, *name1, *name2;
 	int merge_status;
 
 	ll_opts.renormalize = opt->renormalize;
@@ -1043,16 +1043,13 @@ static int merge_3way(struct merge_options *opt,
 		}
 	}
 
-	assert(a->path && b->path && o->path);
-	if (strcmp(a->path, b->path) ||
-	    (opt->ancestor != NULL && strcmp(a->path, o->path) != 0)) {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s:%s", opt->ancestor, o->path);
+	assert(a->path && b->path && o->path && opt->ancestor);
+	if (strcmp(a->path, b->path) || strcmp(a->path, o->path) != 0) {
+		base  = mkpathdup("%s:%s", opt->ancestor, o->path);
 		name1 = mkpathdup("%s:%s", branch1, a->path);
 		name2 = mkpathdup("%s:%s", branch2, b->path);
 	} else {
-		base_name = opt->ancestor == NULL ? NULL :
-			mkpathdup("%s", opt->ancestor);
+		base  = mkpathdup("%s", opt->ancestor);
 		name1 = mkpathdup("%s", branch1);
 		name2 = mkpathdup("%s", branch2);
 	}
@@ -1061,11 +1058,11 @@ static int merge_3way(struct merge_options *opt,
 	read_mmblob(&src1, &a->oid);
 	read_mmblob(&src2, &b->oid);
 
-	merge_status = ll_merge(result_buf, a->path, &orig, base_name,
+	merge_status = ll_merge(result_buf, a->path, &orig, base,
 				&src1, name1, &src2, name2,
 				opt->repo->index, &ll_opts);
 
-	free(base_name);
+	free(base);
 	free(name1);
 	free(name2);
 	free(orig.ptr);
@@ -3390,6 +3387,8 @@ int merge_trees(struct merge_options *opt,
 	int code, clean;
 	struct strbuf sb = STRBUF_INIT;
 
+	assert(opt->ancestor != NULL);
+
 	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
 		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
 		    sb.buf);
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 04/24] merge-recursive: provide a better label for diff3 common ancestor

[Elijah Newren]

In commit 7ca56aa07619 ("merge-recursive: add a label for ancestor",
2010-03-20), a label was added for the '||||||' line to make it have
the more informative heading '|||||| merged common ancestors', with
the statement:

    It would be nicer to use a more informative label.  Perhaps someone
    will provide one some day.

This chosen label was perfectly reasonable when recursiveness kicks in,
i.e. when there are multiple merge bases.  (I can't think of a better
label in such cases.)  But it is actually somewhat misleading when there
is a unique merge base or no merge base.  Change this based on the
number of merge bases:
    >=2: "merged common ancestors"
    1:   <abbreviated commit hash>
    0:   "<empty tree>"

Tests have also been added to check that we get the right ancestor name
for each of the three cases.

Also, since merge_recursive() and merge_trees() have polar opposite
pre-conditions for opt->ancestor, document merge_recursive()'s
pre-condition with an assertion.  (An assertion was added to
merge_trees() already a few commits ago.)  The differences in
pre-conditions stem from two factors: (1) merge_trees() does not recurse
and thus does not have multiple sub-merges to worry about -- each of
which would require a different value for opt->ancestor, (2)
merge_trees() is only passed trees rather than commits and thus cannot
internally guess as good of a label.  Thus, while external callers of
merge_trees() are required to provide a non-NULL opt->ancestor,
merge_recursive() expects to set this value itself.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c                 |  16 ++-
 t/t6036-recursive-corner-cases.sh |   8 +-
 t/t6047-diff3-conflict-markers.sh | 189 ++++++++++++++++++++++++++++++
 3 files changed, 209 insertions(+), 4 deletions(-)
 create mode 100755 t/t6047-diff3-conflict-markers.sh

diff --git a/merge-recursive.c b/merge-recursive.c
index a67ea4957a..e6b84db2ef 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3507,6 +3507,11 @@ int merge_recursive(struct merge_options *opt,
 	struct commit *merged_common_ancestors;
 	struct tree *mrtree;
 	int clean;
+	const char *ancestor_name;
+	struct strbuf merge_base_abbrev = STRBUF_INIT;
+
+	if (!opt->call_depth)
+		assert(opt->ancestor == NULL);
 
 	if (show(opt, 4)) {
 		output(opt, 4, _("Merging:"));
@@ -3535,6 +3540,14 @@ int merge_recursive(struct merge_options *opt,
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
 		merged_common_ancestors = make_virtual_commit(opt->repo, tree, "ancestor");
+		ancestor_name = "empty tree";
+	} else if (ca) {
+		ancestor_name = "merged common ancestors";
+	} else {
+		strbuf_add_unique_abbrev(&merge_base_abbrev,
+					 &merged_common_ancestors->object.oid,
+					 DEFAULT_ABBREV);
+		ancestor_name = merge_base_abbrev.buf;
 	}
 
 	for (iter = ca; iter; iter = iter->next) {
@@ -3568,10 +3581,11 @@ int merge_recursive(struct merge_options *opt,
 	if (!opt->call_depth)
 		repo_read_index(opt->repo);
 
-	opt->ancestor = "merged common ancestors";
+	opt->ancestor = ancestor_name;
 	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
 			    get_commit_tree(merged_common_ancestors),
 			    &mrtree);
+	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
diff --git a/t/t6036-recursive-corner-cases.sh b/t/t6036-recursive-corner-cases.sh
index d23b948f27..7fddcc8c73 100755
--- a/t/t6036-recursive-corner-cases.sh
+++ b/t/t6036-recursive-corner-cases.sh
@@ -1562,6 +1562,7 @@ test_expect_success 'check nested conflicts' '
 		cd nested_conflicts &&
 
 		git clean -f &&
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L2^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1582,7 +1583,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:a >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1594,7 +1595,7 @@ test_expect_success 'check nested conflicts' '
 		git cat-file -p R1:b >theirs &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			ours  \
 			base  \
@@ -1732,6 +1733,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 	(
 		cd virtual_merge_base_has_nested_conflicts &&
 
+		MASTER=$(git rev-parse --short master) &&
 		git checkout L3^0 &&
 
 		# Merge must fail; there is a conflict
@@ -1760,7 +1762,7 @@ test_expect_success 'check virtual merge base with nested conflicts' '
 		cp left merged-once &&
 		test_must_fail git merge-file --diff3 \
 			-L "Temporary merge branch 1" \
-			-L "merged common ancestors"  \
+			-L "$MASTER"  \
 			-L "Temporary merge branch 2" \
 			merged-once \
 			base        \
diff --git a/t/t6047-diff3-conflict-markers.sh b/t/t6047-diff3-conflict-markers.sh
new file mode 100755
index 0000000000..3fb68e0aae
--- /dev/null
+++ b/t/t6047-diff3-conflict-markers.sh
@@ -0,0 +1,189 @@
+#!/bin/sh
+
+test_description='recursive merge diff3 style conflict markers'
+
+. ./test-lib.sh
+
+# Setup:
+#          L1
+#            \
+#             ?
+#            /
+#          R1
+#
+# Where:
+#   L1 and R1 both have a file named 'content' but have no common history
+#
+
+test_expect_success 'setup no merge base' '
+	test_create_repo no_merge_base &&
+	(
+		cd no_merge_base &&
+
+		git checkout -b L &&
+		test_commit A content A &&
+
+		git checkout --orphan R &&
+		test_commit B content B
+	)
+'
+
+test_expect_success 'check no merge base' '
+	(
+		cd no_merge_base &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge --allow-unrelated-histories -s recursive R^0 &&
+
+		grep "|||||| empty tree" content
+	)
+'
+
+# Setup:
+#          L1
+#         /  \
+#   master    ?
+#         \  /
+#          R1
+#
+# Where:
+#   L1 and R1 have modified the same file ('content') in conflicting ways
+#
+
+test_expect_success 'setup unique merge base' '
+	test_create_repo unique_merge_base &&
+	(
+		cd unique_merge_base &&
+
+		test_commit base content "1
+2
+3
+4
+5
+" &&
+
+		git branch L &&
+		git branch R &&
+
+		git checkout L &&
+		test_commit L content "1
+2
+3
+4
+5
+7" &&
+
+		git checkout R &&
+		git rm content &&
+		test_commit R renamed "1
+2
+3
+4
+5
+six"
+	)
+'
+
+test_expect_success 'check unique merge base' '
+	(
+		cd unique_merge_base &&
+
+		git checkout L^0 &&
+		MASTER=$(git rev-parse --short master) &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| $MASTER:content" renamed
+	)
+'
+
+# Setup:
+#          L1---L2--L3
+#         /  \ /      \
+#   master    X1       ?
+#         \  / \      /
+#          R1---R2--R3
+#
+# Where:
+#   commits L1 and R1 have modified the same file in non-conflicting ways
+#   X1 is an auto-generated merge-base used when merging L1 and R1
+#   commits L2 and R2 are merges of R1 and L1 into L1 and R1, respectively
+#   commits L3 and R3 both modify 'content' in conflicting ways
+#
+
+test_expect_success 'setup multiple merge bases' '
+	test_create_repo multiple_merge_bases &&
+	(
+		cd multiple_merge_bases &&
+
+		test_commit initial content "1
+2
+3
+4
+5" &&
+
+		git branch L &&
+		git branch R &&
+
+		# Create L1
+		git checkout L &&
+		test_commit L1 content "0
+1
+2
+3
+4
+5" &&
+
+		# Create R1
+		git checkout R &&
+		test_commit R1 content "1
+2
+3
+4
+5
+6" &&
+
+		# Create L2
+		git checkout L &&
+		git merge R1 &&
+
+		# Create R2
+		git checkout R &&
+		git merge L1 &&
+
+		# Create L3
+		git checkout L &&
+		test_commit L3 content "0
+1
+2
+3
+4
+5
+A" &&
+
+		# Create R3
+		git checkout R &&
+		git rm content &&
+		test_commit R3 renamed "0
+2
+3
+4
+5
+six"
+	)
+'
+
+test_expect_success 'check multiple merge bases' '
+	(
+		cd multiple_merge_bases &&
+
+		git checkout L^0 &&
+
+		test_must_fail git -c merge.conflictstyle=diff3 merge -s recursive R^0 &&
+
+		grep "|||||| merged common ancestors:content" renamed
+	)
+'
+
+test_done
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 05/24] merge-recursive: introduce an enum for detect_directory_renames values

[Elijah Newren]

From: Derrick Stolee <dstolee@microsoft.com>

Improve code readability by introducing an enum to replace the
not-quite-boolean values taken on by detect_directory_renames.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/am.c      |  2 +-
 merge-recursive.c | 24 +++++++++++++++---------
 merge-recursive.h |  6 +++++-
 3 files changed, 21 insertions(+), 11 deletions(-)

diff --git a/builtin/am.c b/builtin/am.c
index 1aea657a7f..037e828efe 100644
--- a/builtin/am.c
+++ b/builtin/am.c
@@ -1538,7 +1538,7 @@ static int fall_back_threeway(const struct am_state *state, const char *index_pa
 	o.branch1 = "HEAD";
 	their_tree_name = xstrfmt("%.*s", linelen(state->msg), state->msg);
 	o.branch2 = their_tree_name;
-	o.detect_directory_renames = 0;
+	o.detect_directory_renames = MERGE_DIRECTORY_RENAMES_NONE;
 
 	if (state->quiet)
 		o.verbosity = 0;
diff --git a/merge-recursive.c b/merge-recursive.c
index e6b84db2ef..9622781612 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1375,7 +1375,8 @@ static int handle_rename_via_dir(struct merge_options *opt,
 	const struct rename *ren = ci->ren1;
 	const struct diff_filespec *dest = ren->pair->two;
 	char *file_path = dest->path;
-	int mark_conflicted = (opt->detect_directory_renames == 1);
+	int mark_conflicted = (opt->detect_directory_renames ==
+			       MERGE_DIRECTORY_RENAMES_CONFLICT);
 	assert(ren->dir_rename_original_dest);
 
 	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
@@ -2860,8 +2861,9 @@ static int detect_and_process_renames(struct merge_options *opt,
 	head_pairs = get_diffpairs(opt, common, head);
 	merge_pairs = get_diffpairs(opt, common, merge);
 
-	if ((opt->detect_directory_renames == 2) ||
-	    (opt->detect_directory_renames == 1 && !opt->call_depth)) {
+	if ((opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE) ||
+	    (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_CONFLICT &&
+	     !opt->call_depth)) {
 		dir_re_head = get_directory_renames(head_pairs);
 		dir_re_merge = get_directory_renames(merge_pairs);
 
@@ -3119,7 +3121,8 @@ static int handle_rename_normal(struct merge_options *opt,
 	clean = handle_content_merge(&mfi, opt, path, was_dirty(opt, path),
 				     o, a, b, ci);
 
-	if (clean && opt->detect_directory_renames == 1 &&
+	if (clean &&
+	    opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_CONFLICT &&
 	    ren->dir_rename_original_dest) {
 		if (update_stages(opt, path,
 				  NULL,
@@ -3164,12 +3167,12 @@ static int warn_about_dir_renamed_entries(struct merge_options *opt,
 		return clean;
 
 	/* Sanity checks */
-	assert(opt->detect_directory_renames > 0);
+	assert(opt->detect_directory_renames > MERGE_DIRECTORY_RENAMES_NONE);
 	assert(ren->dir_rename_original_type == 'A' ||
 	       ren->dir_rename_original_type == 'R');
 
 	/* Check whether to treat directory renames as a conflict */
-	clean = (opt->detect_directory_renames == 2);
+	clean = (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE);
 
 	is_add = (ren->dir_rename_original_type == 'A');
 	if (ren->dir_rename_original_type == 'A' && clean) {
@@ -3679,9 +3682,12 @@ static void merge_recursive_config(struct merge_options *opt)
 	if (!git_config_get_string("merge.directoryrenames", &value)) {
 		int boolval = git_parse_maybe_bool(value);
 		if (0 <= boolval) {
-			opt->detect_directory_renames = boolval ? 2 : 0;
+			opt->detect_directory_renames = boolval ?
+				MERGE_DIRECTORY_RENAMES_TRUE :
+				MERGE_DIRECTORY_RENAMES_NONE;
 		} else if (!strcasecmp(value, "conflict")) {
-			opt->detect_directory_renames = 1;
+			opt->detect_directory_renames =
+				MERGE_DIRECTORY_RENAMES_CONFLICT;
 		} /* avoid erroring on values from future versions of git */
 		free(value);
 	}
@@ -3701,7 +3707,7 @@ void init_merge_options(struct merge_options *opt,
 	opt->renormalize = 0;
 	opt->diff_detect_rename = -1;
 	opt->merge_detect_rename = -1;
-	opt->detect_directory_renames = 1;
+	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
diff --git a/merge-recursive.h b/merge-recursive.h
index c2b7bb65c6..f1b6ef38ae 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -22,7 +22,11 @@ struct merge_options {
 	unsigned renormalize : 1;
 	long xdl_opts;
 	int verbosity;
-	int detect_directory_renames;
+	enum {
+		MERGE_DIRECTORY_RENAMES_NONE = 0,
+		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
+		MERGE_DIRECTORY_RENAMES_TRUE = 2
+	} detect_directory_renames;
 	int diff_detect_rename;
 	int merge_detect_rename;
 	int diff_rename_limit;
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 06/24] merge-recursive: future-proof update_file_flags() against memory leaks

[Elijah Newren]

There is a 'free_buf' label to which all but one of the error paths in
update_file_flags() jump; that error case involves a NULL buf and is
thus not a memory leak.  However, make that error case execute the same
deallocation code anyway so that if anyone adds any additional memory
allocations or deallocations, then all error paths correctly deallocate
resources.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 9622781612..1d4df952e5 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -934,9 +934,11 @@ static int update_file_flags(struct merge_options *opt,
 		}
 
 		buf = read_object_file(&contents->oid, &type, &size);
-		if (!buf)
-			return err(opt, _("cannot read object %s '%s'"),
-				   oid_to_hex(&contents->oid), path);
+		if (!buf) {
+			ret = err(opt, _("cannot read object %s '%s'"),
+				  oid_to_hex(&contents->oid), path);
+			goto free_buf;
+		}
 		if (type != OBJ_BLOB) {
 			ret = err(opt, _("blob expected for %s '%s'"),
 				  oid_to_hex(&contents->oid), path);
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 07/24] merge-recursive: remove another implicit dependency on the_repository

[Elijah Newren]

Commit d7cf3a96e9a0 ("merge-recursive.c: remove implicit dependency on
the_repository", 2019-01-12) and follow-ups like commit 34e7771bc644
("Use the right 'struct repository' instead of the_repository",
2019-06-27), removed most implicit uses of the_repository.  Convert
calls to get_commit_tree() to instead use repo_get_commit_tree() to get
rid of another.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1d4df952e5..88a33e6e72 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3587,8 +3587,11 @@ int merge_recursive(struct merge_options *opt,
 		repo_read_index(opt->repo);
 
 	opt->ancestor = ancestor_name;
-	clean = merge_trees(opt, get_commit_tree(h1), get_commit_tree(h2),
-			    get_commit_tree(merged_common_ancestors),
+	clean = merge_trees(opt,
+			    repo_get_commit_tree(opt->repo, h1),
+			    repo_get_commit_tree(opt->repo, h2),
+			    repo_get_commit_tree(opt->repo,
+						 merged_common_ancestors),
 			    &mrtree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 08/24] Ensure index matches head before invoking merge machinery, round N

[Elijah Newren]

This is the bug that just won't die; there always seems to be another
form of it somewhere.  See the commit message of 55f39cf7551b ("merge:
fix misleading pre-merge check documentation", 2018-06-30) for a more
detailed explanation), but in short:

<quick summary>

builtin/merge.c contains this important requirement for merge
strategies:

    ...the index must be in sync with the head commit.  The strategies are
    responsible to ensure this.

This condition is important to enforce because there are two likely
failure cases when the index isn't in sync with the head commit:

  * we silently throw away changes the user had staged before the merge

  * we accidentally (and silently) include changes in the merge that
    were not part of either of the branches/trees being merged

Discarding users' work and mis-merging are both bad outcomes, especially
when done silently, so naturally this rule was stated sternly -- but,
unfortunately totally ignored in practice unless and until actual bugs
were found.  But, fear not: the bugs from this were fixed in commit
  ee6566e8d70d ("[PATCH] Rewrite read-tree", 2005-09-05)
through a rewrite of read-tree (again, commit 55f39cf7551b has a more
detailed explanation of how this affected merge).  And it was fixed
again in commit
  160252f81626 ("git-merge-ours: make sure our index matches HEAD", 2005-11-03)
...and it was fixed again in commit
  3ec62ad9ffba ("merge-octopus: abort if index does not match HEAD", 2016-04-09)
...and again in commit
  65170c07d466 ("merge-recursive: avoid incorporating uncommitted changes in a merge", 2017-12-21)
...and again in commit
  eddd1a411d93 ("merge-recursive: enforce rule that index matches head before merging", 2018-06-30)

...with multiple testcases added to the testsuite that could be
enumerated in even more commits.

Then, finally, in a patch in the same series as the last fix above, the
documentation about this requirement was fixed in commit 55f39cf7551b
("merge: fix misleading pre-merge check documentation", 2018-06-30), and
we all lived happily ever after...

</quick summary>

Unfortunately, "ever after" apparently denotes a limited time and it
expired today.  The merge-recursive rule to enforce that index matches
head was at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially HUGE amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Trying to enforce that all of merge_trees(), merge_recursive(), and
merge_recursive_generic() checked the index == head condition earlier
resulted in a bunch of broken tests.  It turns out that
merge_recursive() has code to drop and reload the cache while recursing
to create intermediate virtual merge bases, but unfortunately that code
runs even when no recursion is necessary.  This unconditional dropping
and reloading of the cache masked a few bugs:

  * builtin/merge-recursive.c: didn't even bother loading the index.

  * builtin/stash.c: feels like a fake 'builtin' because it repeatedly
    invokes git subprocesses all over the place, mixed with other
    operations.  In particular, invoking "git reset" will reset the
    index on disk, but the parent process that invoked it won't
    automatically have its in-memory index updated.

  * t3030-merge-recursive.h: this test has always been broken in that it
    didn't make sure to make index match head before running.  But, it
    didn't care about the index or even the merge result, just the
    verbose output while running.  While commit eddd1a411d93
    ("merge-recursive: enforce rule that index matches head before
    merging", 2018-06-30) should have uncovered this broken test, it
    used a test_must_fail wrapper around the merge-recursive call
    because it was known that the merge resulted in a rename/rename
    conflict.  Thus, that fix only made this test fail for a different
    reason, and since the index == head check didn't happen until after
    coming all the way back out of the recursion, the testcase had
    enough information to pass the one check that it did perform.

So, load the index in builtin/merge-recursive.c, reload the in-memory
index in builtin/stash.c, and modify the t3030 testcase to correctly
setup the index and make sure that the test fails in the expected way
(meaning it reports a rename/rename conflict).  This makes sure that
all callers actually make the index match head.  The next commit will
then enforce the condition that index matches head earlier so this
problem doesn't return in the future.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-recursive.c  | 4 ++++
 builtin/stash.c            | 2 ++
 t/t3030-merge-recursive.sh | 9 ++++++++-
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/builtin/merge-recursive.c b/builtin/merge-recursive.c
index 5b910e351e..a4bfd8fc51 100644
--- a/builtin/merge-recursive.c
+++ b/builtin/merge-recursive.c
@@ -1,3 +1,4 @@
+#include "cache.h"
 #include "builtin.h"
 #include "commit.h"
 #include "tag.h"
@@ -63,6 +64,9 @@ int cmd_merge_recursive(int argc, const char **argv, const char *prefix)
 	if (argc - i != 3) /* "--" "<head>" "<remote>" */
 		die(_("not handling anything other than two heads merge."));
 
+	if (repo_read_index_unmerged(the_repository))
+		die_resolve_conflict("merge");
+
 	o.branch1 = argv[++i];
 	o.branch2 = argv[++i];
 
diff --git a/builtin/stash.c b/builtin/stash.c
index b5a301f24d..4aa47785f9 100644
--- a/builtin/stash.c
+++ b/builtin/stash.c
@@ -427,6 +427,8 @@ static int do_apply_stash(const char *prefix, struct stash_info *info,
 				return error(_("could not save index tree"));
 
 			reset_head();
+			discard_cache();
+			read_cache();
 		}
 	}
 
diff --git a/t/t3030-merge-recursive.sh b/t/t3030-merge-recursive.sh
index ff641b348a..a37bcc58a0 100755
--- a/t/t3030-merge-recursive.sh
+++ b/t/t3030-merge-recursive.sh
@@ -667,15 +667,22 @@ test_expect_success 'merging with triple rename across D/F conflict' '
 test_expect_success 'merge-recursive remembers the names of all base trees' '
 	git reset --hard HEAD &&
 
+	# make the index match $c1 so that merge-recursive below does not
+	# fail early
+	git diff --binary HEAD $c1 -- | git apply --cached &&
+
 	# more trees than static slots used by oid_to_hex()
 	for commit in $c0 $c2 $c4 $c5 $c6 $c7
 	do
 		git rev-parse "$commit^{tree}"
 	done >trees &&
 
-	# ignore the return code -- it only fails because the input is weird
+	# ignore the return code; it only fails because the input is weird...
 	test_must_fail git -c merge.verbosity=5 merge-recursive $(cat trees) -- $c1 $c3 >out &&
 
+	# ...but make sure it fails in the expected way
+	test_i18ngrep CONFLICT.*rename/rename out &&
+
 	# merge-recursive prints in reverse order, but we do not care
 	sort <trees >expect &&
 	sed -n "s/^virtual //p" out | sort >actual &&
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 09/24] merge-recursive: exit early if index != head

[Elijah Newren]

We had a rule to enforce that the index matches head, but it was found
at the beginning of merge_trees() and would only trigger when
opt->call_depth was 0.  Since merge_recursive() doesn't call
merge_trees() until after returning from recursing, this meant that the
check wasn't triggered by merge_recursive() until it had first finished
all the intermediate merges to create virtual merge bases.  That is a
potentially huge amount of computation (and writing of intermediate
merge results into the .git/objects directory) before it errors out and
says, in effect, "Sorry, I can't do any merging because you have some
local changes that would be overwritten."

Further, not enforcing this requirement earlier allowed other bugs (such
as an unintentional unconditional dropping and reloading of the index in
merge_recursive() even when no recursion was necessary), to mask bugs in
other callers (which were fixed in the commit prior to this one).

Make sure we do the index == head check at the beginning of the merge,
and error out immediately if it fails.  While we're at it, fix a small
leak in the show-the-error codepath.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 101 +++++++++++++++++++++++++++++++++-------------
 1 file changed, 72 insertions(+), 29 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 88a33e6e72..2a254d5563 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3382,23 +3382,14 @@ static int process_entry(struct merge_options *opt,
 	return clean_merge;
 }
 
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *common,
-		struct tree **result)
+static int merge_trees_internal(struct merge_options *opt,
+				struct tree *head,
+				struct tree *merge,
+				struct tree *common,
+				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
-	struct strbuf sb = STRBUF_INIT;
-
-	assert(opt->ancestor != NULL);
-
-	if (!opt->call_depth && repo_index_has_changes(opt->repo, head, &sb)) {
-		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
-		    sb.buf);
-		return -1;
-	}
 
 	if (opt->subtree_shift) {
 		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
@@ -3502,11 +3493,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
  * Merge the commits h1 and h2, return the resulting virtual
  * commit object and a flag indicating the cleanness of the merge.
  */
-int merge_recursive(struct merge_options *opt,
-		    struct commit *h1,
-		    struct commit *h2,
-		    struct commit_list *ca,
-		    struct commit **result)
+static int merge_recursive_internal(struct merge_options *opt,
+				    struct commit *h1,
+				    struct commit *h2,
+				    struct commit_list *ca,
+				    struct commit **result)
 {
 	struct commit_list *iter;
 	struct commit *merged_common_ancestors;
@@ -3515,9 +3506,6 @@ int merge_recursive(struct merge_options *opt,
 	const char *ancestor_name;
 	struct strbuf merge_base_abbrev = STRBUF_INIT;
 
-	if (!opt->call_depth)
-		assert(opt->ancestor == NULL);
-
 	if (show(opt, 4)) {
 		output(opt, 4, _("Merging:"));
 		output_commit_title(opt, h1);
@@ -3571,7 +3559,7 @@ int merge_recursive(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive(opt, merged_common_ancestors, iter->item,
+		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
 				    NULL, &merged_common_ancestors) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
@@ -3587,12 +3575,12 @@ int merge_recursive(struct merge_options *opt,
 		repo_read_index(opt->repo);
 
 	opt->ancestor = ancestor_name;
-	clean = merge_trees(opt,
-			    repo_get_commit_tree(opt->repo, h1),
-			    repo_get_commit_tree(opt->repo, h2),
-			    repo_get_commit_tree(opt->repo,
-						 merged_common_ancestors),
-			    &mrtree);
+	clean = merge_trees_internal(opt,
+				     repo_get_commit_tree(opt->repo, h1),
+				     repo_get_commit_tree(opt->repo, h2),
+				     repo_get_commit_tree(opt->repo,
+							  merged_common_ancestors),
+				     &mrtree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
 		flush_output(opt);
@@ -3613,6 +3601,61 @@ int merge_recursive(struct merge_options *opt,
 	return clean;
 }
 
+static int merge_start(struct merge_options *opt, struct tree *head)
+{
+	struct strbuf sb = STRBUF_INIT;
+
+	if (repo_index_has_changes(opt->repo, head, &sb)) {
+		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
+		    sb.buf);
+		strbuf_release(&sb);
+		return -1;
+	}
+
+	return 0;
+}
+
+static void merge_finalize(struct merge_options *opt)
+{
+	/* Common code for wrapping up merges will be added here later */
+}
+
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *common,
+		struct tree **result)
+{
+	int clean;
+
+	assert(opt->ancestor != NULL);
+
+	if (merge_start(opt, head))
+		return -1;
+	clean = merge_trees_internal(opt, head, merge, common, result);
+	merge_finalize(opt);
+
+	return clean;
+}
+
+int merge_recursive(struct merge_options *opt,
+		    struct commit *h1,
+		    struct commit *h2,
+		    struct commit_list *ca,
+		    struct commit **result)
+{
+	int clean;
+
+	assert(opt->ancestor == NULL);
+
+	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
+		return -1;
+	clean = merge_recursive_internal(opt, h1, h2, ca, result);
+	merge_finalize(opt);
+
+	return clean;
+}
+
 static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
 			      const char *name)
 {
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 10/24] merge-recursive: remove useless parameter in merge_trees()

[Elijah Newren]

merge_trees() took a results parameter that would only be written when
opt->call_depth was positive, which is never the case now that
merge_trees_internal() has been split from merge_trees().  Remove the
misleading and unused parameter from merge_trees().

While at it, add some comments explaining how the output of
merge_trees() and merge_recursive() differ.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c |  4 +---
 merge-recursive.c  |  6 +++---
 merge-recursive.h  | 20 ++++++++++++++++----
 sequencer.c        |  4 ++--
 4 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index d5b946dc3a..90e0eaf25e 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -708,7 +708,6 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 * give up or do a real merge, depending on
 			 * whether the merge flag was used.
 			 */
-			struct tree *result;
 			struct tree *work;
 			struct tree *old_tree;
 			struct merge_options o;
@@ -780,8 +779,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			ret = merge_trees(&o,
 					  new_tree,
 					  work,
-					  old_tree,
-					  &result);
+					  old_tree);
 			if (ret < 0)
 				exit(128);
 			ret = reset_tree(new_tree,
diff --git a/merge-recursive.c b/merge-recursive.c
index 2a254d5563..4ce783dbfa 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3623,16 +3623,16 @@ static void merge_finalize(struct merge_options *opt)
 int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
-		struct tree **result)
+		struct tree *common)
 {
 	int clean;
+	struct tree *ignored;
 
 	assert(opt->ancestor != NULL);
 
 	if (merge_start(opt, head))
 		return -1;
-	clean = merge_trees_internal(opt, head, merge, common, result);
+	clean = merge_trees_internal(opt, head, merge, common, &ignored);
 	merge_finalize(opt);
 
 	return clean;
diff --git a/merge-recursive.h b/merge-recursive.h
index f1b6ef38ae..18012fff9d 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -74,19 +74,31 @@ static inline int merge_detect_rename(struct merge_options *o)
 		o->diff_detect_rename >= 0 ? o->diff_detect_rename : 1;
 }
 
-/* merge_trees() but with recursive ancestor consolidation */
+/*
+ * merge_recursive is like merge_trees() but with recursive ancestor
+ * consolidation, and when successful, it creates an actual commit
+ * and writes its address to *result.
+ *
+ * NOTE: empirically, about a decade ago it was determined that with more
+ *       than two merge bases, optimal behavior was found when the
+ *       ancestors were passed in the order of oldest merge base to newest
+ *       one.  Also, ancestors will be consumed (emptied) so make a copy if
+ *       you need it.
+ */
 int merge_recursive(struct merge_options *o,
 		    struct commit *h1,
 		    struct commit *h2,
 		    struct commit_list *ancestors,
 		    struct commit **result);
 
-/* rename-detecting three-way merge, no recursion */
+/*
+ * rename-detecting three-way merge, no recursion; result of merge is written
+ * to opt->repo->index.
+ */
 int merge_trees(struct merge_options *o,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common,
-		struct tree **result);
+		struct tree *common);
 
 /*
  * "git-merge-recursive" can be fed trees; wrap them into
diff --git a/sequencer.c b/sequencer.c
index 34ebf8ed94..c4ed30f1b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -586,7 +586,7 @@ static int do_recursive_merge(struct repository *r,
 			      struct replay_opts *opts)
 {
 	struct merge_options o;
-	struct tree *result, *next_tree, *base_tree, *head_tree;
+	struct tree *next_tree, *base_tree, *head_tree;
 	int clean;
 	char **xopt;
 	struct lock_file index_lock = LOCK_INIT;
@@ -613,7 +613,7 @@ static int do_recursive_merge(struct repository *r,
 
 	clean = merge_trees(&o,
 			    head_tree,
-			    next_tree, base_tree, &result);
+			    next_tree, base_tree);
 	if (is_rebase_i(opts) && clean <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 11/24] merge-recursive: don't force external callers to do our logging

[Elijah Newren]

Alternatively, you can view this as "make the merge functions behave
more similarly."  merge-recursive has three different entry points:
merge_trees(), merge_recursive(), and merge_recursive_generic().  Two of
these would call diff_warn_rename_limit(), but merge_trees() didn't.
This lead to callers of merge_trees() needing to manually call
diff_warn_rename_limit() themselves.  Move this to the new
merge_finalize() function to make sure that all three entry points run
this function.

Note that there are two external callers of merge_trees(), one in
sequencer.c and one in builtin/checkout.c.  The one in sequencer.c is
cleaned up by this patch and just transfers where the call to
diff_warn_rename_limit() is made; the one in builtin/checkout.c is for
switching to a different commit and in the very rare case where the
warning might be triggered, it would probably be helpful to include
(e.g. if someone is modifying a file that has been renamed in moving to
the other commit, but there are so many renames between the commits that
the limit kicks in and none are detected, it may help to have an
explanation about why they got a delete/modify conflict instead of a
proper content merge in a renamed file).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 +++----
 sequencer.c       | 1 -
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 4ce783dbfa..fda67dd371 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3595,9 +3595,6 @@ static int merge_recursive_internal(struct merge_options *opt,
 	flush_output(opt);
 	if (!opt->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
-	if (show(opt, 2))
-		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
 	return clean;
 }
 
@@ -3617,7 +3614,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 
 static void merge_finalize(struct merge_options *opt)
 {
-	/* Common code for wrapping up merges will be added here later */
+	if (show(opt, 2))
+		diff_warn_rename_limit("merge.renamelimit",
+				       opt->needed_rename_limit, 0);
 }
 
 int merge_trees(struct merge_options *opt,
diff --git a/sequencer.c b/sequencer.c
index c4ed30f1b4..094a4dd03d 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -617,7 +617,6 @@ static int do_recursive_merge(struct repository *r,
 	if (is_rebase_i(opts) && clean <= 0)
 		fputs(o.obuf.buf, stdout);
 	strbuf_release(&o.obuf);
-	diff_warn_rename_limit("merge.renamelimit", o.needed_rename_limit, 0);
 	if (clean < 0) {
 		rollback_lock_file(&index_lock);
 		return clean;
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 12/24] cache-tree: share code between functions writing an index as a tree

[Elijah Newren]

write_tree_from_memory() appeared to be a merge-recursive special that
basically duplicated write_index_as_tree().  The two have a different
signature, but the bigger difference was just that write_index_as_tree()
would always unconditionally read the index off of disk instead of
working on the current in-memory index.  So:

  * split out common code into write_index_as_tree_internal()

  * rename write_tree_from_memory() to write_inmemory_index_as_tree(),
    make it call write_index_as_tree_internal(), and move it to
    cache-tree.c

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/checkout.c |  2 +-
 cache-tree.c       | 85 +++++++++++++++++++++++++++++++++-------------
 cache-tree.h       |  3 +-
 merge-recursive.c  | 34 ++-----------------
 merge-recursive.h  |  1 -
 5 files changed, 67 insertions(+), 58 deletions(-)

diff --git a/builtin/checkout.c b/builtin/checkout.c
index 90e0eaf25e..5e41fc1c01 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -760,7 +760,7 @@ static int merge_working_tree(const struct checkout_opts *opts,
 			 */
 			init_merge_options(&o, the_repository);
 			o.verbosity = 0;
-			work = write_tree_from_memory(&o);
+			work = write_in_core_index_as_tree(the_repository);
 
 			ret = reset_tree(new_tree,
 					 opts, 1,
diff --git a/cache-tree.c b/cache-tree.c
index 706ffcf188..fbb5252521 100644
--- a/cache-tree.c
+++ b/cache-tree.c
@@ -608,11 +608,66 @@ static struct cache_tree *cache_tree_find(struct cache_tree *it, const char *pat
 	return it;
 }
 
+static int write_index_as_tree_internal(struct object_id *oid,
+					struct index_state *index_state,
+					int cache_tree_valid,
+					int flags,
+					const char *prefix)
+{
+	if (flags & WRITE_TREE_IGNORE_CACHE_TREE) {
+		cache_tree_free(&index_state->cache_tree);
+		cache_tree_valid = 0;
+	}
+
+	if (!index_state->cache_tree)
+		index_state->cache_tree = cache_tree();
+
+	if (!cache_tree_valid && cache_tree_update(index_state, flags) < 0)
+		return WRITE_TREE_UNMERGED_INDEX;
+
+	if (prefix) {
+		struct cache_tree *subtree;
+		subtree = cache_tree_find(index_state->cache_tree, prefix);
+		if (!subtree)
+			return WRITE_TREE_PREFIX_ERROR;
+		oidcpy(oid, &subtree->oid);
+	}
+	else
+		oidcpy(oid, &index_state->cache_tree->oid);
+
+	return 0;
+}
+
+struct tree* write_in_core_index_as_tree(struct repository *repo) {
+	struct object_id o;
+	int was_valid, ret;
+
+	struct index_state *index_state	= repo->index;
+	was_valid = index_state->cache_tree &&
+		    cache_tree_fully_valid(index_state->cache_tree);
+
+	ret = write_index_as_tree_internal(&o, index_state, was_valid, 0, NULL);
+	if (ret == WRITE_TREE_UNMERGED_INDEX) {
+		int i;
+		fprintf(stderr, "BUG: There are unmerged index entries:\n");
+		for (i = 0; i < index_state->cache_nr; i++) {
+			const struct cache_entry *ce = index_state->cache[i];
+			if (ce_stage(ce))
+				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
+					(int)ce_namelen(ce), ce->name);
+		}
+		BUG("unmerged index entries when writing inmemory index");
+	}
+
+	return lookup_tree(repo, &index_state->cache_tree->oid);
+}
+
+
 int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix)
 {
 	int entries, was_valid;
 	struct lock_file lock_file = LOCK_INIT;
-	int ret = 0;
+	int ret;
 
 	hold_lock_file_for_update(&lock_file, index_path, LOCK_DIE_ON_ERROR);
 
@@ -621,18 +676,14 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 		ret = WRITE_TREE_UNREADABLE_INDEX;
 		goto out;
 	}
-	if (flags & WRITE_TREE_IGNORE_CACHE_TREE)
-		cache_tree_free(&index_state->cache_tree);
 
-	if (!index_state->cache_tree)
-		index_state->cache_tree = cache_tree();
+	was_valid = !(flags & WRITE_TREE_IGNORE_CACHE_TREE) &&
+		    index_state->cache_tree &&
+		    cache_tree_fully_valid(index_state->cache_tree);
 
-	was_valid = cache_tree_fully_valid(index_state->cache_tree);
-	if (!was_valid) {
-		if (cache_tree_update(index_state, flags) < 0) {
-			ret = WRITE_TREE_UNMERGED_INDEX;
-			goto out;
-		}
+	ret = write_index_as_tree_internal(oid, index_state, was_valid, flags,
+					   prefix);
+	if (!ret && !was_valid) {
 		write_locked_index(index_state, &lock_file, COMMIT_LOCK);
 		/* Not being able to write is fine -- we are only interested
 		 * in updating the cache-tree part, and if the next caller
@@ -642,18 +693,6 @@ int write_index_as_tree(struct object_id *oid, struct index_state *index_state,
 		 */
 	}
 
-	if (prefix) {
-		struct cache_tree *subtree;
-		subtree = cache_tree_find(index_state->cache_tree, prefix);
-		if (!subtree) {
-			ret = WRITE_TREE_PREFIX_ERROR;
-			goto out;
-		}
-		oidcpy(oid, &subtree->oid);
-	}
-	else
-		oidcpy(oid, &index_state->cache_tree->oid);
-
 out:
 	rollback_lock_file(&lock_file);
 	return ret;
diff --git a/cache-tree.h b/cache-tree.h
index 757bbc48bc..639bfa5340 100644
--- a/cache-tree.h
+++ b/cache-tree.h
@@ -34,7 +34,7 @@ int cache_tree_fully_valid(struct cache_tree *);
 int cache_tree_update(struct index_state *, int);
 void cache_tree_verify(struct repository *, struct index_state *);
 
-/* bitmasks to write_cache_as_tree flags */
+/* bitmasks to write_index_as_tree flags */
 #define WRITE_TREE_MISSING_OK 1
 #define WRITE_TREE_IGNORE_CACHE_TREE 2
 #define WRITE_TREE_DRY_RUN 4
@@ -46,6 +46,7 @@ void cache_tree_verify(struct repository *, struct index_state *);
 #define WRITE_TREE_UNMERGED_INDEX (-2)
 #define WRITE_TREE_PREFIX_ERROR (-3)
 
+struct tree* write_in_core_index_as_tree(struct repository *repo);
 int write_index_as_tree(struct object_id *oid, struct index_state *index_state, const char *index_path, int flags, const char *prefix);
 void prime_cache_tree(struct repository *, struct index_state *, struct tree *);
 
diff --git a/merge-recursive.c b/merge-recursive.c
index fda67dd371..ae509357f7 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -412,37 +412,6 @@ static void unpack_trees_finish(struct merge_options *opt)
 	clear_unpack_trees_porcelain(&opt->unpack_opts);
 }
 
-struct tree *write_tree_from_memory(struct merge_options *opt)
-{
-	struct tree *result = NULL;
-	struct index_state *istate = opt->repo->index;
-
-	if (unmerged_index(istate)) {
-		int i;
-		fprintf(stderr, "BUG: There are unmerged index entries:\n");
-		for (i = 0; i < istate->cache_nr; i++) {
-			const struct cache_entry *ce = istate->cache[i];
-			if (ce_stage(ce))
-				fprintf(stderr, "BUG: %d %.*s\n", ce_stage(ce),
-					(int)ce_namelen(ce), ce->name);
-		}
-		BUG("unmerged index entries in merge-recursive.c");
-	}
-
-	if (!istate->cache_tree)
-		istate->cache_tree = cache_tree();
-
-	if (!cache_tree_fully_valid(istate->cache_tree) &&
-	    cache_tree_update(istate, 0) < 0) {
-		err(opt, _("error building trees"));
-		return NULL;
-	}
-
-	result = lookup_tree(opt->repo, &istate->cache_tree->oid);
-
-	return result;
-}
-
 static int save_files_dirs(const struct object_id *oid,
 			   struct strbuf *base, const char *path,
 			   unsigned int mode, int stage, void *context)
@@ -3472,7 +3441,8 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth && !(*result = write_tree_from_memory(opt)))
+	if (opt->call_depth &&
+	    !(*result = write_in_core_index_as_tree(opt->repo)))
 		return -1;
 
 	return clean;
diff --git a/merge-recursive.h b/merge-recursive.h
index 18012fff9d..0a3033bdb0 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -113,7 +113,6 @@ int merge_recursive_generic(struct merge_options *o,
 
 void init_merge_options(struct merge_options *o,
 			struct repository *repo);
-struct tree *write_tree_from_memory(struct merge_options *o);
 
 int parse_merge_opt(struct merge_options *out, const char *s);
 
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 13/24] merge-recursive: fix some overly long lines

[Elijah Newren]

No substantive code change, just add some line breaks to fix lines that
have grown in length due to various refactorings.  Most remaining lines
of excessive length in merge-recursive include error messages and it's
not clear that splitting those improves things.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index ae509357f7..720678c21d 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -681,7 +681,9 @@ static void add_flattened_path(struct strbuf *out, const char *s)
 			out->buf[i] = '_';
 }
 
-static char *unique_path(struct merge_options *opt, const char *path, const char *branch)
+static char *unique_path(struct merge_options *opt,
+			 const char *path,
+			 const char *branch)
 {
 	struct path_hashmap_entry *entry;
 	struct strbuf newpath = STRBUF_INIT;
@@ -915,7 +917,8 @@ static int update_file_flags(struct merge_options *opt,
 		}
 		if (S_ISREG(contents->mode)) {
 			struct strbuf strbuf = STRBUF_INIT;
-			if (convert_to_working_tree(opt->repo->index, path, buf, size, &strbuf)) {
+			if (convert_to_working_tree(opt->repo->index,
+						    path, buf, size, &strbuf)) {
 				free(buf);
 				size = strbuf.len;
 				buf = strbuf_detach(&strbuf, NULL);
@@ -3393,7 +3396,8 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp, NULL, 512);
+		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
 
@@ -3502,7 +3506,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo, tree, "ancestor");
+		merged_common_ancestors = make_virtual_commit(opt->repo,
+							      tree, "ancestor");
 		ancestor_name = "empty tree";
 	} else if (ca) {
 		ancestor_name = "merged common ancestors";
@@ -3625,7 +3630,8 @@ int merge_recursive(struct merge_options *opt,
 	return clean;
 }
 
-static struct commit *get_ref(struct repository *repo, const struct object_id *oid,
+static struct commit *get_ref(struct repository *repo,
+			      const struct object_id *oid,
 			      const char *name)
 {
 	struct object *object;
@@ -3660,7 +3666,8 @@ int merge_recursive_generic(struct merge_options *opt,
 		int i;
 		for (i = 0; i < num_base_list; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i], oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, base_list[i],
+					     oid_to_hex(base_list[i]))))
 				return err(opt, _("Could not parse object '%s'"),
 					   oid_to_hex(base_list[i]));
 			commit_list_insert(base, &ca);
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 14/24] merge-recursive: use common name for ancestors/common/base_list

[Elijah Newren]

merge_trees(), merge_recursive(), and merge_recursive_generic() in
their function headers used four different names for the merge base or
list of merge bases they were passed:
  * 'common'
  * 'ancestors'
  * 'ca'
  * 'base_list'
They were able to refer to it four different ways instead of only three
by using a different name in the signature for the .c file than the .h
file.  Change all of these to 'merge_base' or 'merge_bases'.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 72 ++++++++++++++++++++++++-----------------------
 merge-recursive.h | 14 ++++-----
 2 files changed, 44 insertions(+), 42 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 720678c21d..32871132a6 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3357,24 +3357,26 @@ static int process_entry(struct merge_options *opt,
 static int merge_trees_internal(struct merge_options *opt,
 				struct tree *head,
 				struct tree *merge,
-				struct tree *common,
+				struct tree *merge_base,
 				struct tree **result)
 {
 	struct index_state *istate = opt->repo->index;
 	int code, clean;
 
 	if (opt->subtree_shift) {
-		merge = shift_tree_object(opt->repo, head, merge, opt->subtree_shift);
-		common = shift_tree_object(opt->repo, head, common, opt->subtree_shift);
+		merge = shift_tree_object(opt->repo, head, merge,
+					  opt->subtree_shift);
+		merge_base = shift_tree_object(opt->repo, head, merge_base,
+					       opt->subtree_shift);
 	}
 
-	if (oid_eq(&common->object.oid, &merge->object.oid)) {
+	if (oid_eq(&merge_base->object.oid, &merge->object.oid)) {
 		output(opt, 0, _("Already up to date!"));
 		*result = head;
 		return 1;
 	}
 
-	code = unpack_trees_start(opt, common, head, merge);
+	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
 		if (show(opt, 4) || opt->call_depth)
@@ -3402,7 +3404,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		get_files_dirs(opt, merge);
 
 		entries = get_unmerged(opt->repo->index);
-		clean = detect_and_process_renames(opt, common, head, merge,
+		clean = detect_and_process_renames(opt, merge_base, head, merge,
 						   entries, &re_info);
 		record_df_conflict_files(opt, entries);
 		if (clean < 0)
@@ -3470,11 +3472,11 @@ static struct commit_list *reverse_commit_list(struct commit_list *list)
 static int merge_recursive_internal(struct merge_options *opt,
 				    struct commit *h1,
 				    struct commit *h2,
-				    struct commit_list *ca,
+				    struct commit_list *merge_bases,
 				    struct commit **result)
 {
 	struct commit_list *iter;
-	struct commit *merged_common_ancestors;
+	struct commit *merged_merge_bases;
 	struct tree *mrtree;
 	int clean;
 	const char *ancestor_name;
@@ -3486,39 +3488,39 @@ static int merge_recursive_internal(struct merge_options *opt,
 		output_commit_title(opt, h2);
 	}
 
-	if (!ca) {
-		ca = get_merge_bases(h1, h2);
-		ca = reverse_commit_list(ca);
+	if (!merge_bases) {
+		merge_bases = get_merge_bases(h1, h2);
+		merge_bases = reverse_commit_list(merge_bases);
 	}
 
 	if (show(opt, 5)) {
-		unsigned cnt = commit_list_count(ca);
+		unsigned cnt = commit_list_count(merge_bases);
 
 		output(opt, 5, Q_("found %u common ancestor:",
 				"found %u common ancestors:", cnt), cnt);
-		for (iter = ca; iter; iter = iter->next)
+		for (iter = merge_bases; iter; iter = iter->next)
 			output_commit_title(opt, iter->item);
 	}
 
-	merged_common_ancestors = pop_commit(&ca);
-	if (merged_common_ancestors == NULL) {
+	merged_merge_bases = pop_commit(&merge_bases);
+	if (merged_merge_bases == NULL) {
 		/* if there is no common ancestor, use an empty tree */
 		struct tree *tree;
 
 		tree = lookup_tree(opt->repo, opt->repo->hash_algo->empty_tree);
-		merged_common_ancestors = make_virtual_commit(opt->repo,
-							      tree, "ancestor");
+		merged_merge_bases = make_virtual_commit(opt->repo, tree,
+							 "ancestor");
 		ancestor_name = "empty tree";
-	} else if (ca) {
+	} else if (merge_bases) {
 		ancestor_name = "merged common ancestors";
 	} else {
 		strbuf_add_unique_abbrev(&merge_base_abbrev,
-					 &merged_common_ancestors->object.oid,
+					 &merged_merge_bases->object.oid,
 					 DEFAULT_ABBREV);
 		ancestor_name = merge_base_abbrev.buf;
 	}
 
-	for (iter = ca; iter; iter = iter->next) {
+	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
 		opt->call_depth++;
 		/*
@@ -3534,14 +3536,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 		saved_b2 = opt->branch2;
 		opt->branch1 = "Temporary merge branch 1";
 		opt->branch2 = "Temporary merge branch 2";
-		if (merge_recursive_internal(opt, merged_common_ancestors, iter->item,
-				    NULL, &merged_common_ancestors) < 0)
+		if (merge_recursive_internal(opt, merged_merge_bases, iter->item,
+					     NULL, &merged_merge_bases) < 0)
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
 		opt->call_depth--;
 
-		if (!merged_common_ancestors)
+		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
@@ -3554,7 +3556,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h1),
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
-							  merged_common_ancestors),
+							  merged_merge_bases),
 				     &mrtree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
@@ -3597,7 +3599,7 @@ static void merge_finalize(struct merge_options *opt)
 int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common)
+		struct tree *merge_base)
 {
 	int clean;
 	struct tree *ignored;
@@ -3606,7 +3608,7 @@ int merge_trees(struct merge_options *opt,
 
 	if (merge_start(opt, head))
 		return -1;
-	clean = merge_trees_internal(opt, head, merge, common, &ignored);
+	clean = merge_trees_internal(opt, head, merge, merge_base, &ignored);
 	merge_finalize(opt);
 
 	return clean;
@@ -3615,7 +3617,7 @@ int merge_trees(struct merge_options *opt,
 int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ca,
+		    struct commit_list *merge_bases,
 		    struct commit **result)
 {
 	int clean;
@@ -3624,7 +3626,7 @@ int merge_recursive(struct merge_options *opt,
 
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
-	clean = merge_recursive_internal(opt, h1, h2, ca, result);
+	clean = merge_recursive_internal(opt, h1, h2, merge_bases, result);
 	merge_finalize(opt);
 
 	return clean;
@@ -3652,8 +3654,8 @@ static struct commit *get_ref(struct repository *repo,
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_base_list,
-			    const struct object_id **base_list,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result)
 {
 	int clean;
@@ -3662,14 +3664,14 @@ int merge_recursive_generic(struct merge_options *opt,
 	struct commit *next_commit = get_ref(opt->repo, merge, opt->branch2);
 	struct commit_list *ca = NULL;
 
-	if (base_list) {
+	if (merge_bases) {
 		int i;
-		for (i = 0; i < num_base_list; ++i) {
+		for (i = 0; i < num_merge_bases; ++i) {
 			struct commit *base;
-			if (!(base = get_ref(opt->repo, base_list[i],
-					     oid_to_hex(base_list[i]))))
+			if (!(base = get_ref(opt->repo, merge_bases[i],
+					     oid_to_hex(merge_bases[i]))))
 				return err(opt, _("Could not parse object '%s'"),
-					   oid_to_hex(base_list[i]));
+					   oid_to_hex(merge_bases[i]));
 			commit_list_insert(base, &ca);
 		}
 	}
diff --git a/merge-recursive.h b/merge-recursive.h
index 0a3033bdb0..6f351098a5 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -81,14 +81,14 @@ static inline int merge_detect_rename(struct merge_options *o)
  *
  * NOTE: empirically, about a decade ago it was determined that with more
  *       than two merge bases, optimal behavior was found when the
- *       ancestors were passed in the order of oldest merge base to newest
- *       one.  Also, ancestors will be consumed (emptied) so make a copy if
- *       you need it.
+ *       merge_bases were passed in the order of oldest commit to newest
+ *       commit.  Also, merge_bases will be consumed (emptied) so make a
+ *       copy if you need it.
  */
 int merge_recursive(struct merge_options *o,
 		    struct commit *h1,
 		    struct commit *h2,
-		    struct commit_list *ancestors,
+		    struct commit_list *merge_bases,
 		    struct commit **result);
 
 /*
@@ -98,7 +98,7 @@ int merge_recursive(struct merge_options *o,
 int merge_trees(struct merge_options *o,
 		struct tree *head,
 		struct tree *merge,
-		struct tree *common);
+		struct tree *merge_base);
 
 /*
  * "git-merge-recursive" can be fed trees; wrap them into
@@ -107,8 +107,8 @@ int merge_trees(struct merge_options *o,
 int merge_recursive_generic(struct merge_options *o,
 			    const struct object_id *head,
 			    const struct object_id *merge,
-			    int num_ca,
-			    const struct object_id **ca,
+			    int num_merge_bases,
+			    const struct object_id **merge_bases,
 			    struct commit **result);
 
 void init_merge_options(struct merge_options *o,
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 15/24] merge-recursive: rename 'mrtree' to 'result_tree', for clarity

[Elijah Newren]

It is not at all clear what 'mr' was supposed to stand for, at least not
to me.  Pick a clearer name for this variable.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 32871132a6..1823a87706 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3477,7 +3477,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 {
 	struct commit_list *iter;
 	struct commit *merged_merge_bases;
-	struct tree *mrtree;
+	struct tree *result_tree;
 	int clean;
 	const char *ancestor_name;
 	struct strbuf merge_base_abbrev = STRBUF_INIT;
@@ -3557,7 +3557,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 				     repo_get_commit_tree(opt->repo, h2),
 				     repo_get_commit_tree(opt->repo,
 							  merged_merge_bases),
-				     &mrtree);
+				     &result_tree);
 	strbuf_release(&merge_base_abbrev);
 	if (clean < 0) {
 		flush_output(opt);
@@ -3565,7 +3565,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 	}
 
 	if (opt->call_depth) {
-		*result = make_virtual_commit(opt->repo, mrtree, "merged tree");
+		*result = make_virtual_commit(opt->repo, result_tree,
+					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 16/24] merge-recursive: rename merge_options argument to opt in header

[Elijah Newren]

In commit 259ccb6cc324 ("merge-recursive: rename merge_options argument
from 'o' to 'opt'", 2019-04-05), I renamed a bunch of function
arguments in merge-recursive.c, but forgot to make that same change to
merge-recursive.h.  Make the two match.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.h | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/merge-recursive.h b/merge-recursive.h
index 6f351098a5..2cb3844ad9 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -68,10 +68,10 @@ struct collision_entry {
 	unsigned reported_already:1;
 };
 
-static inline int merge_detect_rename(struct merge_options *o)
+static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return o->merge_detect_rename >= 0 ? o->merge_detect_rename :
-		o->diff_detect_rename >= 0 ? o->diff_detect_rename : 1;
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
 }
 
 /*
@@ -85,7 +85,7 @@ static inline int merge_detect_rename(struct merge_options *o)
  *       commit.  Also, merge_bases will be consumed (emptied) so make a
  *       copy if you need it.
  */
-int merge_recursive(struct merge_options *o,
+int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
 		    struct commit *h2,
 		    struct commit_list *merge_bases,
@@ -95,7 +95,7 @@ int merge_recursive(struct merge_options *o,
  * rename-detecting three-way merge, no recursion; result of merge is written
  * to opt->repo->index.
  */
-int merge_trees(struct merge_options *o,
+int merge_trees(struct merge_options *opt,
 		struct tree *head,
 		struct tree *merge,
 		struct tree *merge_base);
@@ -104,16 +104,16 @@ int merge_trees(struct merge_options *o,
  * "git-merge-recursive" can be fed trees; wrap them into
  * virtual commits and call merge_recursive() proper.
  */
-int merge_recursive_generic(struct merge_options *o,
+int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
 			    const struct object_id *merge,
 			    int num_merge_bases,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *o,
+void init_merge_options(struct merge_options *opt,
 			struct repository *repo);
 
-int parse_merge_opt(struct merge_options *out, const char *s);
+int parse_merge_opt(struct merge_options *opt, const char *s);
 
 #endif
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 17/24] merge-recursive: move some definitions around to clean up the header

[Elijah Newren]

No substantive code changes (view this with diff --color-moved), but
a few small code cleanups:
  * Move structs and an inline function only used by merge-recursive.c
    into merge-recursive.c
  * Re-order function declarations to be more logical
  * Add or fix some explanatory comments

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 31 +++++++++++++++++
 merge-recursive.h | 87 +++++++++++++++++++++++------------------------
 2 files changed, 73 insertions(+), 45 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1823a87706..9807b24c65 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -54,6 +54,24 @@ static unsigned int path_hash(const char *path)
 	return ignore_case ? strihash(path) : strhash(path);
 }
 
+/*
+ * For dir_rename_entry, directory names are stored as a full path from the
+ * toplevel of the repository and do not include a trailing '/'.  Also:
+ *
+ *   dir:                original name of directory being renamed
+ *   non_unique_new_dir: if true, could not determine new_dir
+ *   new_dir:            final name of directory being renamed
+ *   possible_new_dirs:  temporary used to help determine new_dir; see comments
+ *                       in get_directory_renames() for details
+ */
+struct dir_rename_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *dir;
+	unsigned non_unique_new_dir:1;
+	struct strbuf new_dir;
+	struct string_list possible_new_dirs;
+};
+
 static struct dir_rename_entry *dir_rename_find_entry(struct hashmap *hashmap,
 						      char *dir)
 {
@@ -92,6 +110,13 @@ static void dir_rename_entry_init(struct dir_rename_entry *entry,
 	string_list_init(&entry->possible_new_dirs, 0);
 }
 
+struct collision_entry {
+	struct hashmap_entry ent; /* must be the first member! */
+	char *target_file;
+	struct string_list source_files;
+	unsigned reported_already:1;
+};
+
 static struct collision_entry *collision_find_entry(struct hashmap *hashmap,
 						    char *target_file)
 {
@@ -358,6 +383,12 @@ static int add_cacheinfo(struct merge_options *opt,
 	return ret;
 }
 
+static inline int merge_detect_rename(struct merge_options *opt)
+{
+	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
+		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+}
+
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
 {
 	parse_tree(tree);
diff --git a/merge-recursive.h b/merge-recursive.h
index 2cb3844ad9..0fdae904dd 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -43,47 +43,50 @@ struct merge_options {
 	struct repository *repo;
 };
 
+void init_merge_options(struct merge_options *opt, struct repository *repo);
+
+/* parse the option in s and update the relevant field of opt */
+int parse_merge_opt(struct merge_options *opt, const char *s);
+
 /*
- * For dir_rename_entry, directory names are stored as a full path from the
- * toplevel of the repository and do not include a trailing '/'.  Also:
- *
- *   dir:                original name of directory being renamed
- *   non_unique_new_dir: if true, could not determine new_dir
- *   new_dir:            final name of directory being renamed
- *   possible_new_dirs:  temporary used to help determine new_dir; see comments
- *                       in get_directory_renames() for details
+ * RETURN VALUES: All the merge_* functions below return a value as follows:
+ *   > 0     Merge was clean
+ *   = 0     Merge had conflicts
+ *   < 0     Merge hit an unexpected and unrecoverable problem (e.g. disk
+ *             full) and aborted merge part-way through.
  */
-struct dir_rename_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *dir;
-	unsigned non_unique_new_dir:1;
-	struct strbuf new_dir;
-	struct string_list possible_new_dirs;
-};
-
-struct collision_entry {
-	struct hashmap_entry ent; /* must be the first member! */
-	char *target_file;
-	struct string_list source_files;
-	unsigned reported_already:1;
-};
 
-static inline int merge_detect_rename(struct merge_options *opt)
-{
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
-}
+/*
+ * rename-detecting three-way merge, no recursion.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - No commit is created
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is not updated
+ *   - The working tree is updated with results of the merge
+ */
+int merge_trees(struct merge_options *opt,
+		struct tree *head,
+		struct tree *merge,
+		struct tree *merge_base);
 
 /*
  * merge_recursive is like merge_trees() but with recursive ancestor
- * consolidation, and when successful, it creates an actual commit
- * and writes its address to *result.
+ * consolidation and, if the commit is clean, creation of a commit.
  *
  * NOTE: empirically, about a decade ago it was determined that with more
  *       than two merge bases, optimal behavior was found when the
  *       merge_bases were passed in the order of oldest commit to newest
  *       commit.  Also, merge_bases will be consumed (emptied) so make a
  *       copy if you need it.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - If merge is clean, a commit is created and its address written to *result
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is not updated
+ *   - The working tree is updated with results of the merge
  */
 int merge_recursive(struct merge_options *opt,
 		    struct commit *h1,
@@ -92,17 +95,16 @@ int merge_recursive(struct merge_options *opt,
 		    struct commit **result);
 
 /*
- * rename-detecting three-way merge, no recursion; result of merge is written
- * to opt->repo->index.
- */
-int merge_trees(struct merge_options *opt,
-		struct tree *head,
-		struct tree *merge,
-		struct tree *merge_base);
-
-/*
- * "git-merge-recursive" can be fed trees; wrap them into
- * virtual commits and call merge_recursive() proper.
+ * merge_recursive_generic can operate on trees instead of commits, by
+ * wrapping the trees into virtual commits, and calling merge_recursive().
+ * It also writes out the in-memory index to disk if the merge is successful.
+ *
+ * Outputs:
+ *   - See RETURN VALUES above
+ *   - If merge is clean, a commit is created and its address written to *result
+ *   - opt->repo->index has the new index
+ *   - $GIT_INDEX_FILE is updated
+ *   - The working tree is updated with results of the merge
  */
 int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id *head,
@@ -111,9 +113,4 @@ int merge_recursive_generic(struct merge_options *opt,
 			    const struct object_id **merge_bases,
 			    struct commit **result);
 
-void init_merge_options(struct merge_options *opt,
-			struct repository *repo);
-
-int parse_merge_opt(struct merge_options *opt, const char *s);
-
 #endif
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 18/24] merge-recursive: consolidate unnecessary fields in merge_options

[Elijah Newren]

We provided users with the ability to state whether they wanted rename
detection, and to put a limit on how much CPU would be spent.  Both of
these fields had multiple configuration parameters for setting them,
with one being a fallback and the other being an override.  However,
instead of implementing the logic for how to combine the multiple
source locations into the appropriate setting at config loading time,
we loaded and tracked both values and then made the code combine them
every time it wanted to check the overall value.  This had a few
minor drawbacks:
  * it seems more complicated than necessary
  * it runs the risk of people using the independent settings in the
    future and breaking the intent of how the options are used
    together
  * it makes merge_options more complicated than necessary for other
    potential users of the API

Fix these problems by moving the logic for combining the pairs of
options into a single value; make it apply at time-of-config-loading
instead of each-time-of-use.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 27 +++++++++++----------------
 merge-recursive.h |  6 ++----
 2 files changed, 13 insertions(+), 20 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 9807b24c65..0f0b952c04 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -385,8 +385,7 @@ static int add_cacheinfo(struct merge_options *opt,
 
 static inline int merge_detect_rename(struct merge_options *opt)
 {
-	return opt->merge_detect_rename >= 0 ? opt->merge_detect_rename :
-		opt->diff_detect_rename >= 0 ? opt->diff_detect_rename : 1;
+	return (opt->detect_renames >= 0) ? opt->detect_renames : 1;
 }
 
 static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree)
@@ -1883,9 +1882,7 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	 */
 	if (opts.detect_rename > DIFF_DETECT_RENAME)
 		opts.detect_rename = DIFF_DETECT_RENAME;
-	opts.rename_limit = opt->merge_rename_limit >= 0 ? opt->merge_rename_limit :
-			    opt->diff_rename_limit >= 0 ? opt->diff_rename_limit :
-			    1000;
+	opts.rename_limit = (opt->rename_limit >= 0) ? opt->rename_limit : 1000;
 	opts.rename_score = opt->rename_score;
 	opts.show_rename_progress = opt->show_rename_progress;
 	opts.output_format = DIFF_FORMAT_NO_OUTPUT;
@@ -3727,14 +3724,14 @@ static void merge_recursive_config(struct merge_options *opt)
 {
 	char *value = NULL;
 	git_config_get_int("merge.verbosity", &opt->verbosity);
-	git_config_get_int("diff.renamelimit", &opt->diff_rename_limit);
-	git_config_get_int("merge.renamelimit", &opt->merge_rename_limit);
+	git_config_get_int("diff.renamelimit", &opt->rename_limit);
+	git_config_get_int("merge.renamelimit", &opt->rename_limit);
 	if (!git_config_get_string("diff.renames", &value)) {
-		opt->diff_detect_rename = git_config_rename("diff.renames", value);
+		opt->detect_renames = git_config_rename("diff.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.renames", &value)) {
-		opt->merge_detect_rename = git_config_rename("merge.renames", value);
+		opt->detect_renames = git_config_rename("merge.renames", value);
 		free(value);
 	}
 	if (!git_config_get_string("merge.directoryrenames", &value)) {
@@ -3760,11 +3757,9 @@ void init_merge_options(struct merge_options *opt,
 	opt->repo = repo;
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->diff_rename_limit = -1;
-	opt->merge_rename_limit = -1;
+	opt->rename_limit = -1;
 	opt->renormalize = 0;
-	opt->diff_detect_rename = -1;
-	opt->merge_detect_rename = -1;
+	opt->detect_renames = -1;
 	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
@@ -3816,16 +3811,16 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	else if (!strcmp(s, "no-renormalize"))
 		opt->renormalize = 0;
 	else if (!strcmp(s, "no-renames"))
-		opt->merge_detect_rename = 0;
+		opt->detect_renames = 0;
 	else if (!strcmp(s, "find-renames")) {
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 		opt->rename_score = 0;
 	}
 	else if (skip_prefix(s, "find-renames=", &arg) ||
 		 skip_prefix(s, "rename-threshold=", &arg)) {
 		if ((opt->rename_score = parse_rename_score(&arg)) == -1 || *arg != 0)
 			return -1;
-		opt->merge_detect_rename = 1;
+		opt->detect_renames = 1;
 	}
 	/*
 	 * Please update $__git_merge_strategy_options in
diff --git a/merge-recursive.h b/merge-recursive.h
index 0fdae904dd..f4bdfbc897 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -27,10 +27,8 @@ struct merge_options {
 		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
 		MERGE_DIRECTORY_RENAMES_TRUE = 2
 	} detect_directory_renames;
-	int diff_detect_rename;
-	int merge_detect_rename;
-	int diff_rename_limit;
-	int merge_rename_limit;
+	int detect_renames;
+	int rename_limit;
 	int rename_score;
 	int needed_rename_limit;
 	int show_rename_progress;
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 19/24] merge-recursive: comment and reorder the merge_options fields

[Elijah Newren]

The merge_options struct had lots of fields, making it a little
imposing, but the options naturally fall into multiple different groups.
Grouping similar options and adding a comment or two makes it easier to
read, easier for new folks to figure out which options are related, and
thus easier for them to find the options they need.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 16 +++++++++++-----
 merge-recursive.h | 40 ++++++++++++++++++++++++++--------------
 2 files changed, 37 insertions(+), 19 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0f0b952c04..43dec3307e 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3754,21 +3754,27 @@ void init_merge_options(struct merge_options *opt,
 {
 	const char *merge_verbosity;
 	memset(opt, 0, sizeof(struct merge_options));
+
 	opt->repo = repo;
+
+	opt->detect_renames = -1;
+	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
+	opt->rename_limit = -1;
+
 	opt->verbosity = 2;
 	opt->buffer_output = 1;
-	opt->rename_limit = -1;
+	strbuf_init(&opt->obuf, 0);
+
 	opt->renormalize = 0;
-	opt->detect_renames = -1;
-	opt->detect_directory_renames = MERGE_DIRECTORY_RENAMES_CONFLICT;
+
+	string_list_init(&opt->df_conflict_file_set, 1);
+
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
 		opt->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (opt->verbosity >= 5)
 		opt->buffer_output = 0;
-	strbuf_init(&opt->obuf, 0);
-	string_list_init(&opt->df_conflict_file_set, 1);
 }
 
 int parse_merge_opt(struct merge_options *opt, const char *s)
diff --git a/merge-recursive.h b/merge-recursive.h
index f4bdfbc897..9e040608fe 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -9,36 +9,48 @@ struct commit;
 struct repository;
 
 struct merge_options {
+	struct repository *repo;
+
+	/* ref names used in console messages and conflict markers */
 	const char *ancestor;
 	const char *branch1;
 	const char *branch2;
-	enum {
-		MERGE_RECURSIVE_NORMAL = 0,
-		MERGE_RECURSIVE_OURS,
-		MERGE_RECURSIVE_THEIRS
-	} recursive_variant;
-	const char *subtree_shift;
-	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
-	unsigned renormalize : 1;
-	long xdl_opts;
-	int verbosity;
+
+	/* rename related options */
+	int detect_renames;
 	enum {
 		MERGE_DIRECTORY_RENAMES_NONE = 0,
 		MERGE_DIRECTORY_RENAMES_CONFLICT = 1,
 		MERGE_DIRECTORY_RENAMES_TRUE = 2
 	} detect_directory_renames;
-	int detect_renames;
 	int rename_limit;
 	int rename_score;
-	int needed_rename_limit;
 	int show_rename_progress;
+
+	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
+	long xdl_opts;
+	enum {
+		MERGE_RECURSIVE_NORMAL = 0,
+		MERGE_RECURSIVE_OURS,
+		MERGE_RECURSIVE_THEIRS
+	} recursive_variant;
+
+	/* console output related options */
+	int verbosity;
+	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
+	struct strbuf obuf;     /* output buffer */
+
+	/* miscellaneous control options */
+	const char *subtree_shift;
+	unsigned renormalize : 1;
+
+	/* internal fields used by the implementation (do NOT set these) */
 	int call_depth;
-	struct strbuf obuf;
+	int needed_rename_limit;
 	struct hashmap current_file_dir_set;
 	struct string_list df_conflict_file_set;
 	struct unpack_trees_options unpack_opts;
 	struct index_state orig_index;
-	struct repository *repo;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 20/24] merge-recursive: avoid losing output and leaking memory holding that output

[Elijah Newren]

If opt->buffer_output is less than 2, then merge_trees(),
merge_recursive(), and merge_recursive_generic() are all supposed to
flush the opt->obuf output buffer to stdout and release any memory it
holds.  merge_trees() did not do this.  Move the logic that handles this
for merge_recursive_internal() to merge_finalize() so that all three
methods handle this requirement.

Note that this bug didn't cause any problems currently, because there
are only two callers of merge_trees() right now (a git grep for
'merge_trees(' is misleading because builtin/merge-tree.c also defines a
'merge_tree' function that is unrelated), and only one of those is
called with buffer_output less than 2 (builtin/checkout.c), but it set
opt->verbosity to 0, for which there is only currently one non-error
message that would be shown: "Already up to date!".  However, that one
message can only occur when the merge is utterly trivial (the merge base
tree exactly matches the merge tree), and builtin/checkout.c already
attempts a trivial merge via unpack_trees() before falling back to
merge_trees().

Also, if opt->buffer_output is 2, then the caller is responsible to
handle showing any output in opt->obuf and for free'ing it.  This
requirement might be easy to overlook, so add a comment to
merge-recursive.h pointing it out.  (There are currently two callers
that set buffer_output to 2, both in sequencer.c, and both of which
handle this correctly.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 6 +++---
 merge-recursive.h | 3 ++-
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 43dec3307e..262db8bebb 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3598,9 +3598,6 @@ static int merge_recursive_internal(struct merge_options *opt,
 		commit_list_insert(h1, &(*result)->parents);
 		commit_list_insert(h2, &(*result)->parents->next);
 	}
-	flush_output(opt);
-	if (!opt->call_depth && opt->buffer_output < 2)
-		strbuf_release(&opt->obuf);
 	return clean;
 }
 
@@ -3620,6 +3617,9 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 
 static void merge_finalize(struct merge_options *opt)
 {
+	flush_output(opt);
+	if (!opt->call_depth && opt->buffer_output < 2)
+		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
 				       opt->needed_rename_limit, 0);
diff --git a/merge-recursive.h b/merge-recursive.h
index 9e040608fe..933d6e7642 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -38,7 +38,8 @@ struct merge_options {
 	/* console output related options */
 	int verbosity;
 	unsigned buffer_output; /* 1: output at end, 2: keep buffered */
-	struct strbuf obuf;     /* output buffer */
+	struct strbuf obuf;     /* output buffer; if buffer_output == 2, caller
+				 * must handle and call strbuf_release */
 
 	/* miscellaneous control options */
 	const char *subtree_shift;
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 21/24] merge-recursive: split internal fields into a separate struct

[Elijah Newren]

merge_options has several internal fields that should not be set or read
by external callers.  This just complicates the API.  Move them into an
opaque merge_options_internal struct that is defined only in
merge-recursive.c and keep these out of merge-recursive.h.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 185 ++++++++++++++++++++++++----------------------
 merge-recursive.h |  17 ++---
 2 files changed, 105 insertions(+), 97 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 262db8bebb..c92993e8f0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -29,6 +29,15 @@
 #include "revision.h"
 #include "commit-reach.h"
 
+struct merge_options_internal {
+	int call_depth;
+	int needed_rename_limit;
+	struct hashmap current_file_dir_set;
+	struct string_list df_conflict_file_set;
+	struct unpack_trees_options unpack_opts;
+	struct index_state orig_index;
+};
+
 struct path_hashmap_entry {
 	struct hashmap_entry e;
 	char path[FLEX_ARRAY];
@@ -309,7 +318,8 @@ static inline void setup_rename_conflict_info(enum rename_type rename_type,
 
 static int show(struct merge_options *opt, int v)
 {
-	return (!opt->call_depth && opt->verbosity >= v) || opt->verbosity >= 5;
+	return (!opt->priv->call_depth && opt->verbosity >= v) ||
+		opt->verbosity >= 5;
 }
 
 __attribute__((format (printf, 3, 4)))
@@ -320,7 +330,7 @@ static void output(struct merge_options *opt, int v, const char *fmt, ...)
 	if (!show(opt, v))
 		return;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 
 	va_start(ap, fmt);
 	strbuf_vaddf(&opt->obuf, fmt, ap);
@@ -335,7 +345,7 @@ static void output_commit_title(struct merge_options *opt, struct commit *commit
 {
 	struct merge_remote_desc *desc;
 
-	strbuf_addchars(&opt->obuf, ' ', opt->call_depth * 2);
+	strbuf_addchars(&opt->obuf, ' ', opt->priv->call_depth * 2);
 	desc = merge_remote_util(commit);
 	if (desc)
 		strbuf_addf(&opt->obuf, "virtual %s\n", desc->name);
@@ -403,43 +413,43 @@ static int unpack_trees_start(struct merge_options *opt,
 	struct tree_desc t[3];
 	struct index_state tmp_index = { NULL };
 
-	memset(&opt->unpack_opts, 0, sizeof(opt->unpack_opts));
-	if (opt->call_depth)
-		opt->unpack_opts.index_only = 1;
+	memset(&opt->priv->unpack_opts, 0, sizeof(opt->priv->unpack_opts));
+	if (opt->priv->call_depth)
+		opt->priv->unpack_opts.index_only = 1;
 	else
-		opt->unpack_opts.update = 1;
-	opt->unpack_opts.merge = 1;
-	opt->unpack_opts.head_idx = 2;
-	opt->unpack_opts.fn = threeway_merge;
-	opt->unpack_opts.src_index = opt->repo->index;
-	opt->unpack_opts.dst_index = &tmp_index;
-	opt->unpack_opts.aggressive = !merge_detect_rename(opt);
-	setup_unpack_trees_porcelain(&opt->unpack_opts, "merge");
+		opt->priv->unpack_opts.update = 1;
+	opt->priv->unpack_opts.merge = 1;
+	opt->priv->unpack_opts.head_idx = 2;
+	opt->priv->unpack_opts.fn = threeway_merge;
+	opt->priv->unpack_opts.src_index = opt->repo->index;
+	opt->priv->unpack_opts.dst_index = &tmp_index;
+	opt->priv->unpack_opts.aggressive = !merge_detect_rename(opt);
+	setup_unpack_trees_porcelain(&opt->priv->unpack_opts, "merge");
 
 	init_tree_desc_from_tree(t+0, common);
 	init_tree_desc_from_tree(t+1, head);
 	init_tree_desc_from_tree(t+2, merge);
 
-	rc = unpack_trees(3, t, &opt->unpack_opts);
+	rc = unpack_trees(3, t, &opt->priv->unpack_opts);
 	cache_tree_free(&opt->repo->index->cache_tree);
 
 	/*
-	 * Update opt->repo->index to match the new results, AFTER saving a copy
-	 * in opt->orig_index.  Update src_index to point to the saved copy.
-	 * (verify_uptodate() checks src_index, and the original index is
-	 * the one that had the necessary modification timestamps.)
+	 * Update opt->repo->index to match the new results, AFTER saving a
+	 * copy in opt->priv->orig_index.  Update src_index to point to the
+	 * saved copy.  (verify_uptodate() checks src_index, and the original
+	 * index is the one that had the necessary modification timestamps.)
 	 */
-	opt->orig_index = *opt->repo->index;
+	opt->priv->orig_index = *opt->repo->index;
 	*opt->repo->index = tmp_index;
-	opt->unpack_opts.src_index = &opt->orig_index;
+	opt->priv->unpack_opts.src_index = &opt->priv->orig_index;
 
 	return rc;
 }
 
 static void unpack_trees_finish(struct merge_options *opt)
 {
-	discard_index(&opt->orig_index);
-	clear_unpack_trees_porcelain(&opt->unpack_opts);
+	discard_index(&opt->priv->orig_index);
+	clear_unpack_trees_porcelain(&opt->priv->unpack_opts);
 }
 
 static int save_files_dirs(const struct object_id *oid,
@@ -454,7 +464,7 @@ static int save_files_dirs(const struct object_id *oid,
 
 	FLEX_ALLOC_MEM(entry, path, base->buf, base->len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 
 	strbuf_setlen(base, baselen);
 	return (S_ISDIR(mode) ? READ_TREE_RECURSIVE : 0);
@@ -585,7 +595,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	 * If we're merging merge-bases, we don't want to bother with
 	 * any working directory changes.
 	 */
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return;
 
 	/* Ensure D/F conflicts are adjacent in the entries list. */
@@ -597,7 +607,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 	df_sorted_entries.cmp = string_list_df_name_compare;
 	string_list_sort(&df_sorted_entries);
 
-	string_list_clear(&opt->df_conflict_file_set, 1);
+	string_list_clear(&opt->priv->df_conflict_file_set, 1);
 	for (i = 0; i < df_sorted_entries.nr; i++) {
 		const char *path = df_sorted_entries.items[i].string;
 		int len = strlen(path);
@@ -613,7 +623,7 @@ static void record_df_conflict_files(struct merge_options *opt,
 		    len > last_len &&
 		    memcmp(path, last_file, last_len) == 0 &&
 		    path[last_len] == '/') {
-			string_list_insert(&opt->df_conflict_file_set, last_file);
+			string_list_insert(&opt->priv->df_conflict_file_set, last_file);
 		}
 
 		/*
@@ -680,8 +690,8 @@ static void update_entry(struct stage_data *entry,
 static int remove_file(struct merge_options *opt, int clean,
 		       const char *path, int no_wd)
 {
-	int update_cache = opt->call_depth || clean;
-	int update_working_directory = !opt->call_depth && !no_wd;
+	int update_cache = opt->priv->call_depth || clean;
+	int update_working_directory = !opt->priv->call_depth && !no_wd;
 
 	if (update_cache) {
 		if (remove_file_from_index(opt->repo->index, path))
@@ -724,16 +734,16 @@ static char *unique_path(struct merge_options *opt,
 	add_flattened_path(&newpath, branch);
 
 	base_len = newpath.len;
-	while (hashmap_get_from_hash(&opt->current_file_dir_set,
+	while (hashmap_get_from_hash(&opt->priv->current_file_dir_set,
 				     path_hash(newpath.buf), newpath.buf) ||
-	       (!opt->call_depth && file_exists(newpath.buf))) {
+	       (!opt->priv->call_depth && file_exists(newpath.buf))) {
 		strbuf_setlen(&newpath, base_len);
 		strbuf_addf(&newpath, "_%d", suffix++);
 	}
 
 	FLEX_ALLOC_MEM(entry, path, newpath.buf, newpath.len);
 	hashmap_entry_init(entry, path_hash(entry->path));
-	hashmap_add(&opt->current_file_dir_set, entry);
+	hashmap_add(&opt->priv->current_file_dir_set, entry);
 	return strbuf_detach(&newpath, NULL);
 }
 
@@ -775,7 +785,7 @@ static int dir_in_way(struct index_state *istate, const char *path,
 static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 				   const struct diff_filespec *blob)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 	struct cache_entry *ce;
 
 	if (0 > pos)
@@ -783,7 +793,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
 		return 0;
 
 	/* See if the file we were tracking before matches */
-	ce = opt->orig_index.cache[pos];
+	ce = opt->priv->orig_index.cache[pos];
 	return (oid_eq(&ce->oid, &blob->oid) && ce->ce_mode == blob->mode);
 }
 
@@ -792,7 +802,7 @@ static int was_tracked_and_matches(struct merge_options *opt, const char *path,
  */
 static int was_tracked(struct merge_options *opt, const char *path)
 {
-	int pos = index_name_pos(&opt->orig_index, path, strlen(path));
+	int pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
 
 	if (0 <= pos)
 		/* we were tracking this path before the merge */
@@ -849,12 +859,12 @@ static int was_dirty(struct merge_options *opt, const char *path)
 	struct cache_entry *ce;
 	int dirty = 1;
 
-	if (opt->call_depth || !was_tracked(opt, path))
+	if (opt->priv->call_depth || !was_tracked(opt, path))
 		return !dirty;
 
-	ce = index_file_exists(opt->unpack_opts.src_index,
+	ce = index_file_exists(opt->priv->unpack_opts.src_index,
 			       path, strlen(path), ignore_case);
-	dirty = verify_uptodate(ce, &opt->unpack_opts) != 0;
+	dirty = verify_uptodate(ce, &opt->priv->unpack_opts) != 0;
 	return dirty;
 }
 
@@ -864,8 +874,8 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 	const char *msg = _("failed to create path '%s'%s");
 
 	/* Unlink any D/F conflict files that are in the way */
-	for (i = 0; i < opt->df_conflict_file_set.nr; i++) {
-		const char *df_path = opt->df_conflict_file_set.items[i].string;
+	for (i = 0; i < opt->priv->df_conflict_file_set.nr; i++) {
+		const char *df_path = opt->priv->df_conflict_file_set.items[i].string;
 		size_t pathlen = strlen(path);
 		size_t df_pathlen = strlen(df_path);
 		if (df_pathlen < pathlen &&
@@ -875,7 +885,7 @@ static int make_room_for_path(struct merge_options *opt, const char *path)
 			       _("Removing %s to make room for subdirectory\n"),
 			       df_path);
 			unlink(df_path);
-			unsorted_string_list_delete_item(&opt->df_conflict_file_set,
+			unsorted_string_list_delete_item(&opt->priv->df_conflict_file_set,
 							 i, 0);
 			break;
 		}
@@ -916,7 +926,7 @@ static int update_file_flags(struct merge_options *opt,
 {
 	int ret = 0;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		update_wd = 0;
 
 	if (update_wd) {
@@ -1001,7 +1011,7 @@ static int update_file(struct merge_options *opt,
 		       const char *path)
 {
 	return update_file_flags(opt, contents, path,
-				 opt->call_depth || clean, !opt->call_depth);
+				 opt->priv->call_depth || clean, !opt->priv->call_depth);
 }
 
 /* Low level file merging, update and removal */
@@ -1030,7 +1040,7 @@ static int merge_3way(struct merge_options *opt,
 	ll_opts.extra_marker_size = extra_marker_size;
 	ll_opts.xdl_opts = opt->xdl_opts;
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		ll_opts.virtual_ancestor = 1;
 		ll_opts.variant = 0;
 	} else {
@@ -1161,7 +1171,7 @@ static int merge_submodule(struct merge_options *opt,
 	struct object_array merges;
 
 	int i;
-	int search = !opt->call_depth;
+	int search = !opt->priv->call_depth;
 
 	/* store a in result in case we fail */
 	oidcpy(result, a);
@@ -1383,7 +1393,7 @@ static int handle_rename_via_dir(struct merge_options *opt,
 			       MERGE_DIRECTORY_RENAMES_CONFLICT);
 	assert(ren->dir_rename_original_dest);
 
-	if (!opt->call_depth && would_lose_untracked(opt, dest->path)) {
+	if (!opt->priv->call_depth && would_lose_untracked(opt, dest->path)) {
 		mark_conflicted = 1;
 		file_path = unique_path(opt, dest->path, ren->branch);
 		output(opt, 1, _("Error: Refusing to lose untracked file at %s; "
@@ -1426,12 +1436,12 @@ static int handle_change_delete(struct merge_options *opt,
 	const char *update_path = path;
 	int ret = 0;
 
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0) ||
-	    (!opt->call_depth && would_lose_untracked(opt, path))) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0) ||
+	    (!opt->priv->call_depth && would_lose_untracked(opt, path))) {
 		update_path = alt_path = unique_path(opt, path, change_branch);
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * We cannot arbitrarily accept either a_sha or b_sha as
 		 * correct; since there is no true "middle point" between
@@ -1506,14 +1516,14 @@ static int handle_rename_delete(struct merge_options *opt,
 				     opt->branch2 : opt->branch1);
 
 	if (handle_change_delete(opt,
-				 opt->call_depth ? orig->path : dest->path,
-				 opt->call_depth ? NULL : orig->path,
+				 opt->priv->call_depth ? orig->path : dest->path,
+				 opt->priv->call_depth ? NULL : orig->path,
 				 orig, dest,
 				 rename_branch, delete_branch,
 				 _("rename"), _("renamed")))
 		return -1;
 
-	if (opt->call_depth)
+	if (opt->priv->call_depth)
 		return remove_file_from_index(opt->repo->index, dest->path);
 	else
 		return update_stages(opt, dest->path, NULL,
@@ -1550,7 +1560,7 @@ static int handle_file_collision(struct merge_options *opt,
 	/*
 	 * In the recursive case, we just opt to undo renames
 	 */
-	if (opt->call_depth && (prev_path1 || prev_path2)) {
+	if (opt->priv->call_depth && (prev_path1 || prev_path2)) {
 		/* Put first file (a->oid, a->mode) in its original spot */
 		if (prev_path1) {
 			if (update_file(opt, 1, a, prev_path1))
@@ -1579,10 +1589,10 @@ static int handle_file_collision(struct merge_options *opt,
 	/* Remove rename sources if rename/add or rename/rename(2to1) */
 	if (prev_path1)
 		remove_file(opt, 1, prev_path1,
-			    opt->call_depth || would_lose_untracked(opt, prev_path1));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path1));
 	if (prev_path2)
 		remove_file(opt, 1, prev_path2,
-			    opt->call_depth || would_lose_untracked(opt, prev_path2));
+			    opt->priv->call_depth || would_lose_untracked(opt, prev_path2));
 
 	/*
 	 * Remove the collision path, if it wouldn't cause dirty contents
@@ -1624,12 +1634,12 @@ static int handle_file_collision(struct merge_options *opt,
 	null.mode = 0;
 
 	if (merge_mode_and_contents(opt, &null, a, b, collide_path,
-				    branch1, branch2, opt->call_depth * 2, &mfi))
+				    branch1, branch2, opt->priv->call_depth * 2, &mfi))
 		return -1;
 	mfi.clean &= !alt_path;
 	if (update_file(opt, mfi.clean, &mfi.blob, update_path))
 		return -1;
-	if (!mfi.clean && !opt->call_depth &&
+	if (!mfi.clean && !opt->priv->call_depth &&
 	    update_stages(opt, collide_path, NULL, a, b))
 		return -1;
 	free(alt_path);
@@ -1669,7 +1679,7 @@ static int handle_rename_add(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[other_stage],
 				    prev_path_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi))
+				    1 + opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(prev_path_desc);
 
@@ -1687,7 +1697,7 @@ static char *find_path_for_conflict(struct merge_options *opt,
 				    const char *branch2)
 {
 	char *new_path = NULL;
-	if (dir_in_way(opt->repo->index, path, !opt->call_depth, 0)) {
+	if (dir_in_way(opt->repo->index, path, !opt->priv->call_depth, 0)) {
 		new_path = unique_path(opt, path, branch1);
 		output(opt, 1, _("%s is a directory in %s adding "
 			       "as %s instead"),
@@ -1718,17 +1728,17 @@ static int handle_rename_rename_1to2(struct merge_options *opt,
 	       "rename \"%s\"->\"%s\" in \"%s\"%s"),
 	       o->path, a->path, ci->ren1->branch,
 	       o->path, b->path, ci->ren2->branch,
-	       opt->call_depth ? _(" (left unresolved)") : "");
+	       opt->priv->call_depth ? _(" (left unresolved)") : "");
 
 	path_desc = xstrfmt("%s and %s, both renamed from %s",
 			    a->path, b->path, o->path);
 	if (merge_mode_and_contents(opt, o, a, b, path_desc,
 				    ci->ren1->branch, ci->ren2->branch,
-				    opt->call_depth * 2, &mfi))
+				    opt->priv->call_depth * 2, &mfi))
 		return -1;
 	free(path_desc);
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		/*
 		 * FIXME: For rename/add-source conflicts (if we could detect
 		 * such), this is wrong.  We should instead find a unique
@@ -1843,12 +1853,12 @@ static int handle_rename_rename_2to1(struct merge_options *opt,
 				    &ci->ren1->src_entry->stages[ostage1],
 				    path_side_1_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c1) ||
+				    1 + opt->priv->call_depth * 2, &mfi_c1) ||
 	    merge_mode_and_contents(opt, b,
 				    &ci->ren2->src_entry->stages[ostage2],
 				    c2, path_side_2_desc,
 				    opt->branch1, opt->branch2,
-				    1 + opt->call_depth * 2, &mfi_c2))
+				    1 + opt->priv->call_depth * 2, &mfi_c2))
 		return -1;
 	free(path_side_1_desc);
 	free(path_side_2_desc);
@@ -1889,8 +1899,8 @@ static struct diff_queue_struct *get_diffpairs(struct merge_options *opt,
 	diff_setup_done(&opts);
 	diff_tree_oid(&o_tree->object.oid, &tree->object.oid, "", &opts);
 	diffcore_std(&opts);
-	if (opts.needed_rename_limit > opt->needed_rename_limit)
-		opt->needed_rename_limit = opts.needed_rename_limit;
+	if (opts.needed_rename_limit > opt->priv->needed_rename_limit)
+		opt->priv->needed_rename_limit = opts.needed_rename_limit;
 
 	ret = xmalloc(sizeof(*ret));
 	*ret = diff_queued_diff;
@@ -2865,7 +2875,7 @@ static int detect_and_process_renames(struct merge_options *opt,
 
 	if ((opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE) ||
 	    (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_CONFLICT &&
-	     !opt->call_depth)) {
+	     !opt->priv->call_depth)) {
 		dir_re_head = get_directory_renames(head_pairs);
 		dir_re_merge = get_directory_renames(merge_pairs);
 
@@ -3022,13 +3032,13 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		reason = _("add/add");
 
 	assert(o->path && a->path && b->path);
-	if (ci && dir_in_way(opt->repo->index, path, !opt->call_depth,
+	if (ci && dir_in_way(opt->repo->index, path, !opt->priv->call_depth,
 			     S_ISGITLINK(ci->ren1->pair->two->mode)))
 		df_conflict_remains = 1;
 
 	if (merge_mode_and_contents(opt, o, a, b, path,
 				    opt->branch1, opt->branch2,
-				    opt->call_depth * 2, mfi))
+				    opt->priv->call_depth * 2, mfi))
 		return -1;
 
 	/*
@@ -3044,7 +3054,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 		output(opt, 3, _("Skipped %s (merged same as existing)"), path);
 		if (add_cacheinfo(opt, &mfi->blob, path,
-				  0, (!opt->call_depth && !is_dirty), 0))
+				  0, (!opt->priv->call_depth && !is_dirty), 0))
 			return -1;
 		/*
 		 * However, add_cacheinfo() will delete the old cache entry
@@ -3052,8 +3062,8 @@ static int handle_content_merge(struct merge_file_info *mfi,
 		 * flag to avoid making the file appear as if it were
 		 * deleted by the user.
 		 */
-		pos = index_name_pos(&opt->orig_index, path, strlen(path));
-		ce = opt->orig_index.cache[pos];
+		pos = index_name_pos(&opt->priv->orig_index, path, strlen(path));
+		ce = opt->priv->orig_index.cache[pos];
 		if (ce_skip_worktree(ce)) {
 			pos = index_name_pos(opt->repo->index, path, strlen(path));
 			ce = opt->repo->index->cache[pos];
@@ -3074,7 +3084,7 @@ static int handle_content_merge(struct merge_file_info *mfi,
 
 	if (df_conflict_remains || is_dirty) {
 		char *new_path;
-		if (opt->call_depth) {
+		if (opt->priv->call_depth) {
 			remove_file_from_index(opt->repo->index, path);
 		} else {
 			if (!mfi->clean) {
@@ -3333,7 +3343,7 @@ static int process_entry(struct merge_options *opt,
 			conf = _("directory/file");
 		}
 		if (dir_in_way(opt->repo->index, path,
-			       !opt->call_depth && !S_ISGITLINK(a->mode),
+			       !opt->priv->call_depth && !S_ISGITLINK(a->mode),
 			       0)) {
 			char *new_path = unique_path(opt, path, add_branch);
 			clean_merge = 0;
@@ -3342,7 +3352,7 @@ static int process_entry(struct merge_options *opt,
 			       conf, path, other_branch, path, new_path);
 			if (update_file(opt, 0, contents, new_path))
 				clean_merge = -1;
-			else if (opt->call_depth)
+			else if (opt->priv->call_depth)
 				remove_file_from_index(opt->repo->index, path);
 			free(new_path);
 		} else {
@@ -3407,7 +3417,7 @@ static int merge_trees_internal(struct merge_options *opt,
 	code = unpack_trees_start(opt, merge_base, head, merge);
 
 	if (code != 0) {
-		if (show(opt, 4) || opt->call_depth)
+		if (show(opt, 4) || opt->priv->call_depth)
 			err(opt, _("merging of trees %s and %s failed"),
 			    oid_to_hex(&head->object.oid),
 			    oid_to_hex(&merge->object.oid));
@@ -3426,7 +3436,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		 * opposed to decaring a local hashmap is for convenience
 		 * so that we don't have to pass it to around.
 		 */
-		hashmap_init(&opt->current_file_dir_set, path_hashmap_cmp,
+		hashmap_init(&opt->priv->current_file_dir_set, path_hashmap_cmp,
 			     NULL, 512);
 		get_files_dirs(opt, head);
 		get_files_dirs(opt, merge);
@@ -3463,7 +3473,7 @@ static int merge_trees_internal(struct merge_options *opt,
 		string_list_clear(entries, 1);
 		free(entries);
 
-		hashmap_free(&opt->current_file_dir_set, 1);
+		hashmap_free(&opt->priv->current_file_dir_set, 1);
 
 		if (clean < 0) {
 			unpack_trees_finish(opt);
@@ -3475,7 +3485,7 @@ static int merge_trees_internal(struct merge_options *opt,
 
 	unpack_trees_finish(opt);
 
-	if (opt->call_depth &&
+	if (opt->priv->call_depth &&
 	    !(*result = write_in_core_index_as_tree(opt->repo)))
 		return -1;
 
@@ -3550,7 +3560,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 
 	for (iter = merge_bases; iter; iter = iter->next) {
 		const char *saved_b1, *saved_b2;
-		opt->call_depth++;
+		opt->priv->call_depth++;
 		/*
 		 * When the merge fails, the result contains files
 		 * with conflict markers. The cleanness flag is
@@ -3569,14 +3579,14 @@ static int merge_recursive_internal(struct merge_options *opt,
 			return -1;
 		opt->branch1 = saved_b1;
 		opt->branch2 = saved_b2;
-		opt->call_depth--;
+		opt->priv->call_depth--;
 
 		if (!merged_merge_bases)
 			return err(opt, _("merge returned no commit"));
 	}
 
 	discard_index(opt->repo->index);
-	if (!opt->call_depth)
+	if (!opt->priv->call_depth)
 		repo_read_index(opt->repo);
 
 	opt->ancestor = ancestor_name;
@@ -3592,7 +3602,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 		return clean;
 	}
 
-	if (opt->call_depth) {
+	if (opt->priv->call_depth) {
 		*result = make_virtual_commit(opt->repo, result_tree,
 					      "merged tree");
 		commit_list_insert(h1, &(*result)->parents);
@@ -3612,17 +3622,20 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 		return -1;
 	}
 
+	opt->priv = xcalloc(1, sizeof(*opt->priv));
+	string_list_init(&opt->priv->df_conflict_file_set, 1);
 	return 0;
 }
 
 static void merge_finalize(struct merge_options *opt)
 {
 	flush_output(opt);
-	if (!opt->call_depth && opt->buffer_output < 2)
+	if (!opt->priv->call_depth && opt->buffer_output < 2)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0);
+	FREE_AND_NULL(opt->priv);
 }
 
 int merge_trees(struct merge_options *opt,
@@ -3767,8 +3780,6 @@ void init_merge_options(struct merge_options *opt,
 
 	opt->renormalize = 0;
 
-	string_list_init(&opt->df_conflict_file_set, 1);
-
 	merge_recursive_config(opt);
 	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	if (merge_verbosity)
diff --git a/merge-recursive.h b/merge-recursive.h
index 933d6e7642..58a4c5238a 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -1,13 +1,15 @@
 #ifndef MERGE_RECURSIVE_H
 #define MERGE_RECURSIVE_H
 
-#include "string-list.h"
-#include "unpack-trees.h"
+#include "strbuf.h"
 
 struct commit;
-
+struct commit_list;
+struct object_id;
 struct repository;
+struct tree;
 
+struct merge_options_internal;
 struct merge_options {
 	struct repository *repo;
 
@@ -45,13 +47,8 @@ struct merge_options {
 	const char *subtree_shift;
 	unsigned renormalize : 1;
 
-	/* internal fields used by the implementation (do NOT set these) */
-	int call_depth;
-	int needed_rename_limit;
-	struct hashmap current_file_dir_set;
-	struct string_list df_conflict_file_set;
-	struct unpack_trees_options unpack_opts;
-	struct index_state orig_index;
+	/* internal fields used by the implementation */
+	struct merge_options_internal *priv;
 };
 
 void init_merge_options(struct merge_options *opt, struct repository *repo);
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 22/24] merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*

[Elijah Newren]

I want to implement the same outward facing API as found within
merge-recursive.h in a different merge strategy.  However, that makes
names like MERGE_RECURSIVE_{NORMAL,OURS,THEIRS} look a little funny;
rename to MERGE_VARIANT_{NORMAL,OURS,THEIRS}.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 14 +++++++-------
 merge-recursive.h |  6 +++---
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index c92993e8f0..fa3f8eb0b2 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -1045,10 +1045,10 @@ static int merge_3way(struct merge_options *opt,
 		ll_opts.variant = 0;
 	} else {
 		switch (opt->recursive_variant) {
-		case MERGE_RECURSIVE_OURS:
+		case MERGE_VARIANT_OURS:
 			ll_opts.variant = XDL_MERGE_FAVOR_OURS;
 			break;
-		case MERGE_RECURSIVE_THEIRS:
+		case MERGE_VARIANT_THEIRS:
 			ll_opts.variant = XDL_MERGE_FAVOR_THEIRS;
 			break;
 		default:
@@ -1355,15 +1355,15 @@ static int merge_mode_and_contents(struct merge_options *opt,
 							&b->oid);
 		} else if (S_ISLNK(a->mode)) {
 			switch (opt->recursive_variant) {
-			case MERGE_RECURSIVE_NORMAL:
+			case MERGE_VARIANT_NORMAL:
 				oidcpy(&result->blob.oid, &a->oid);
 				if (!oid_eq(&a->oid, &b->oid))
 					result->clean = 0;
 				break;
-			case MERGE_RECURSIVE_OURS:
+			case MERGE_VARIANT_OURS:
 				oidcpy(&result->blob.oid, &a->oid);
 				break;
-			case MERGE_RECURSIVE_THEIRS:
+			case MERGE_VARIANT_THEIRS:
 				oidcpy(&result->blob.oid, &b->oid);
 				break;
 			}
@@ -3795,9 +3795,9 @@ int parse_merge_opt(struct merge_options *opt, const char *s)
 	if (!s || !*s)
 		return -1;
 	if (!strcmp(s, "ours"))
-		opt->recursive_variant = MERGE_RECURSIVE_OURS;
+		opt->recursive_variant = MERGE_VARIANT_OURS;
 	else if (!strcmp(s, "theirs"))
-		opt->recursive_variant = MERGE_RECURSIVE_THEIRS;
+		opt->recursive_variant = MERGE_VARIANT_THEIRS;
 	else if (!strcmp(s, "subtree"))
 		opt->subtree_shift = "";
 	else if (skip_prefix(s, "subtree=", &arg))
diff --git a/merge-recursive.h b/merge-recursive.h
index 58a4c5238a..978847e672 100644
--- a/merge-recursive.h
+++ b/merge-recursive.h
@@ -32,9 +32,9 @@ struct merge_options {
 	/* xdiff-related options (patience, ignore whitespace, ours/theirs) */
 	long xdl_opts;
 	enum {
-		MERGE_RECURSIVE_NORMAL = 0,
-		MERGE_RECURSIVE_OURS,
-		MERGE_RECURSIVE_THEIRS
+		MERGE_VARIANT_NORMAL = 0,
+		MERGE_VARIANT_OURS,
+		MERGE_VARIANT_THEIRS
 	} recursive_variant;
 
 	/* console output related options */
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 23/24] merge-recursive: add sanity checks for relevant merge_options

[Elijah Newren]

There are lots of options that callers can set, yet most have a limited
range of valid values, some options are meant for output (e.g.
opt->obuf, which is expected to start empty), and callers are expected
to not set opt->priv.  Add several sanity checks to ensure callers
provide sane values.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/merge-recursive.c b/merge-recursive.c
index fa3f8eb0b2..0231d7b049 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3615,6 +3615,30 @@ static int merge_start(struct merge_options *opt, struct tree *head)
 {
 	struct strbuf sb = STRBUF_INIT;
 
+	/* Sanity checks on opt */
+	assert(opt->repo);
+
+	assert(opt->branch1 && opt->branch2);
+
+	assert(opt->detect_renames >= -1 &&
+	       opt->detect_renames <= DIFF_DETECT_COPY);
+	assert(opt->detect_directory_renames >= MERGE_DIRECTORY_RENAMES_NONE &&
+	       opt->detect_directory_renames <= MERGE_DIRECTORY_RENAMES_TRUE);
+	assert(opt->rename_limit >= -1);
+	assert(opt->rename_score >= 0 && opt->rename_score <= MAX_SCORE);
+	assert(opt->show_rename_progress >= 0 && opt->show_rename_progress <= 1);
+
+	assert(opt->xdl_opts >= 0);
+	assert(opt->recursive_variant >= MERGE_VARIANT_NORMAL &&
+	       opt->recursive_variant <= MERGE_VARIANT_THEIRS);
+
+	assert(opt->verbosity >= 0 && opt->verbosity <= 5);
+	assert(opt->buffer_output <= 2);
+	assert(opt->obuf.len == 0);
+
+	assert(opt->priv == NULL);
+
+	/* Sanity check on repo state; index must match head */
 	if (repo_index_has_changes(opt->repo, head, &sb)) {
 		err(opt, _("Your local changes to the following files would be overwritten by merge:\n  %s"),
 		    sb.buf);
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

[PATCH v4 24/24] merge-recursive: alphabetize include list

[Elijah Newren]

Other than cache.h which needs to appear first, and merge-recursive.h
which I want to be second so that we are more likely to notice if
merge-recursive.h has any missing includes, the rest of the list is
long and easier to look through if it's alphabetical.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 0231d7b049..b058741f00 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -4,30 +4,31 @@
  * The thieves were Alex Riesen and Johannes Schindelin, in June/July 2006
  */
 #include "cache.h"
-#include "config.h"
+#include "merge-recursive.h"
+
 #include "advice.h"
-#include "lockfile.h"
-#include "cache-tree.h"
-#include "object-store.h"
-#include "repository.h"
-#include "commit.h"
+#include "alloc.h"
+#include "attr.h"
 #include "blob.h"
 #include "builtin.h"
-#include "tree-walk.h"
+#include "cache-tree.h"
+#include "commit.h"
+#include "commit-reach.h"
+#include "config.h"
 #include "diff.h"
 #include "diffcore.h"
+#include "dir.h"
+#include "ll-merge.h"
+#include "lockfile.h"
+#include "object-store.h"
+#include "repository.h"
+#include "revision.h"
+#include "string-list.h"
+#include "submodule.h"
 #include "tag.h"
-#include "alloc.h"
+#include "tree-walk.h"
 #include "unpack-trees.h"
-#include "string-list.h"
 #include "xdiff-interface.h"
-#include "ll-merge.h"
-#include "attr.h"
-#include "merge-recursive.h"
-#include "dir.h"
-#include "submodule.h"
-#include "revision.h"
-#include "commit-reach.h"
 
 struct merge_options_internal {
 	int call_depth;
-- 
2.23.0.rc2.28.g5f89f15d7b.dirty

Re: [PATCH v4 21/24] merge-recursive: split internal fields into a separate struct

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> merge_options has several internal fields that should not be set or read
> by external callers.  This just complicates the API.  Move them into an
> opaque merge_options_internal struct that is defined only in
> merge-recursive.c and keep these out of merge-recursive.h.

Makes quite a lot of sense.  Thanks.

Re: [PATCH v4 08/24] Ensure index matches head before invoking merge machinery, round N

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2474 bytes --]

Hi Elijah,

On Sat, 17 Aug 2019, Elijah Newren wrote:

>   * t3030-merge-recursive.h: this test has always been broken in that it
>     didn't make sure to make index match head before running.  But, it
>     didn't care about the index or even the merge result, just the
>     verbose output while running.  While commit eddd1a411d93
>     ("merge-recursive: enforce rule that index matches head before
>     merging", 2018-06-30) should have uncovered this broken test, it
>     used a test_must_fail wrapper around the merge-recursive call
>     because it was known that the merge resulted in a rename/rename
>     conflict.  Thus, that fix only made this test fail for a different
>     reason, and since the index == head check didn't happen until after
>     coming all the way back out of the recursion, the testcase had
>     enough information to pass the one check that it did perform.

I fear that this test is still broken, it is a Schrödinger bug. Where
`qsort()` is the cat, and the property "is it stable?" instead of death.

In particular, at some stage in the recursive merge, a diff is generated
with rename detection for the target file `a` that contains two lines `hello`
but has two equally valid source files: `e` and `a~Temporary merge
branch 2_0`, both containing just the line `hello`. And since their file
contents are identical, the solution to the problem "which is the
correct source file?" is ambiguous.

If the `qsort()` in use is stable, the file `e` comes first, and wins.
If the `qsort()` in use is not stable, all bets are off, and the file
`a~Temporary merge branch 2_0` might be sorted first (which is the case,
for example, when using the `qsort()` implementation of MS Visual C's
runtime).

Now, the _real_ problem is that t3030.35 expects the recursive merge to
fail, which it does when `qsort()` is stable. However, when the order of
`e` and `a~Temporary merge branch 2_0` is reversed, then that particular
merge does _not_ result in a `rename/rename` conflict. And the exit code
of the recursive merge is 0 for some reason!

I don't quite understand why: clearly, there are conflicts (otherwise we
would not have that funny suffix `~Temporary merge branch 2_0`.

The real problem, though, is that even if it would fail, the outcome of
that recursive merge is ambiguous, and that test case should not try to
verify the precise order of the generated intermediate trees.

Ciao,
Dscho

Re: [PATCH v4 08/24] Ensure index matches head before invoking merge machinery, round N

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3145 bytes --]

Hi Elijah,

On Tue, 3 Sep 2019, Johannes Schindelin wrote:

> On Sat, 17 Aug 2019, Elijah Newren wrote:
>
> >   * t3030-merge-recursive.h: this test has always been broken in that it
> >     didn't make sure to make index match head before running.  But, it
> >     didn't care about the index or even the merge result, just the
> >     verbose output while running.  While commit eddd1a411d93
> >     ("merge-recursive: enforce rule that index matches head before
> >     merging", 2018-06-30) should have uncovered this broken test, it
> >     used a test_must_fail wrapper around the merge-recursive call
> >     because it was known that the merge resulted in a rename/rename
> >     conflict.  Thus, that fix only made this test fail for a different
> >     reason, and since the index == head check didn't happen until after
> >     coming all the way back out of the recursion, the testcase had
> >     enough information to pass the one check that it did perform.
>
> I fear that this test is still broken, it is a Schrödinger bug. Where
> `qsort()` is the cat, and the property "is it stable?" instead of death.
>
> In particular, at some stage in the recursive merge, a diff is generated
> with rename detection for the target file `a` that contains two lines `hello`
> but has two equally valid source files: `e` and `a~Temporary merge
> branch 2_0`, both containing just the line `hello`. And since their file
> contents are identical, the solution to the problem "which is the
> correct source file?" is ambiguous.
>
> If the `qsort()` in use is stable, the file `e` comes first, and wins.
> If the `qsort()` in use is not stable, all bets are off, and the file
> `a~Temporary merge branch 2_0` might be sorted first (which is the case,
> for example, when using the `qsort()` implementation of MS Visual C's
> runtime).
>
> Now, the _real_ problem is that t3030.35 expects the recursive merge to
> fail, which it does when `qsort()` is stable. However, when the order of
> `e` and `a~Temporary merge branch 2_0` is reversed, then that particular
> merge does _not_ result in a `rename/rename` conflict. And the exit code
> of the recursive merge is 0 for some reason!
>
> I don't quite understand why: clearly, there are conflicts (otherwise we
> would not have that funny suffix `~Temporary merge branch 2_0`.
>
> The real problem, though, is that even if it would fail, the outcome of
> that recursive merge is ambiguous, and that test case should not try to
> verify the precise order of the generated intermediate trees.

It might not be obvious from my mail, but it took me about 7 hours to
figure all of this out, hence I was a bit grumpy when I wrote that. My
apologies.

After having slept (and written a long review about the
`--update-branches` patch), it occurred to me that we might be better
off enforcing the use of `git_qsort()` in `diffcore-rename.c`, so that
we can at least guarantee stable rename detection in Git (which would
incidentally fix the test suite for the MSVC build that I maintain in
Git for Windows).

What do you think?

Ciao,
Dscho

Re: [PATCH v4 08/24] Ensure index matches head before invoking merge machinery, round N

[Elijah Newren]

Hi Dscho,

On Tue, Sep 3, 2019 at 6:34 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Tue, 3 Sep 2019, Johannes Schindelin wrote:
>
> > On Sat, 17 Aug 2019, Elijah Newren wrote:
> >
> > >   * t3030-merge-recursive.h: this test has always been broken in that it
> > >     didn't make sure to make index match head before running.  But, it
> > >     didn't care about the index or even the merge result, just the
> > >     verbose output while running.  While commit eddd1a411d93
> > >     ("merge-recursive: enforce rule that index matches head before
> > >     merging", 2018-06-30) should have uncovered this broken test, it
> > >     used a test_must_fail wrapper around the merge-recursive call
> > >     because it was known that the merge resulted in a rename/rename
> > >     conflict.  Thus, that fix only made this test fail for a different
> > >     reason, and since the index == head check didn't happen until after
> > >     coming all the way back out of the recursion, the testcase had
> > >     enough information to pass the one check that it did perform.
> >
> > I fear that this test is still broken, it is a Schrödinger bug. Where
> > `qsort()` is the cat, and the property "is it stable?" instead of death.
> >
> > In particular, at some stage in the recursive merge, a diff is generated
> > with rename detection for the target file `a` that contains two lines `hello`
> > but has two equally valid source files: `e` and `a~Temporary merge
> > branch 2_0`, both containing just the line `hello`. And since their file
> > contents are identical, the solution to the problem "which is the
> > correct source file?" is ambiguous.
> >
> > If the `qsort()` in use is stable, the file `e` comes first, and wins.
> > If the `qsort()` in use is not stable, all bets are off, and the file
> > `a~Temporary merge branch 2_0` might be sorted first (which is the case,
> > for example, when using the `qsort()` implementation of MS Visual C's
> > runtime).
> >
> > Now, the _real_ problem is that t3030.35 expects the recursive merge to
> > fail, which it does when `qsort()` is stable. However, when the order of
> > `e` and `a~Temporary merge branch 2_0` is reversed, then that particular
> > merge does _not_ result in a `rename/rename` conflict. And the exit code
> > of the recursive merge is 0 for some reason!
> >
> > I don't quite understand why: clearly, there are conflicts (otherwise we
> > would not have that funny suffix `~Temporary merge branch 2_0`.

So, there are conflicts in the inner merge, but depending on the
tie-breaker for rename handling when two equal matches exist (the
tie-breaker being the order of the filenames after qsort()), there may
or may not be conflicts in the outer merge.  Ouch.

I suspect such cases are pretty rare in "real world repositories"
because (1) exactly equal filename similarities are rare, (2) "slowly
changing trees of content" implies that most files will only be
modified on (at most) one side of history, (3) when files are changed
on both sides of history odds of conflicting changes rapidly go up
making conflicts likely.  You essentially have to thread a needle to
have the end result ambiguously conflict like this.

> > The real problem, though, is that even if it would fail, the outcome of
> > that recursive merge is ambiguous, and that test case should not try to
> > verify the precise order of the generated intermediate trees.

Yes, it is ambiguous -- and the problem is a little deeper too. It's
not just "does-this-merge-conflict?" depending upon the qsort order,
it is also about whether file contents after the merge depend upon the
qsort order.  Whenever there are two filenames that are equally
similar to a rename source, picking one of the two equally similar
filenames for rename pairing means we are basically choosing at random
where to merge the changes from the other side of history to.

Unfortunately, changing this might be difficult to enforce with the
current merge-recursive structure.  For example, what if there are two
equally similar filenames for us to choose from, but the other side of
history didn't modify the rename source file at all?  (e.g. on one
side of history, user leaves A alone, on other side of history, A is
copied to B and C and then A is deleted.  B and C are identical.)  In
such cases, the choice of which of B and C we pair A up with happens
to be irrelevant because we'll get the same result either way and
there should be no merge conflict.  But if we error out early or throw
warnings and conflict notices because the intermediate internal choice
was ambiguous, then we've created useless conflicts for the user.  I'm
worried we have more cases of this kind of thing happening than we do
with ambiguous pairings that change the end result.

I think I might be able to do something here with my alternative merge
strategy, but I haven't gotten back to that for quite a while, so...

> It might not be obvious from my mail, but it took me about 7 hours to
> figure all of this out, hence I was a bit grumpy when I wrote that. My
> apologies.
>
> After having slept (and written a long review about the
> `--update-branches` patch), it occurred to me that we might be better
> off enforcing the use of `git_qsort()` in `diffcore-rename.c`, so that
> we can at least guarantee stable rename detection in Git (which would
> incidentally fix the test suite for the MSVC build that I maintain in
> Git for Windows).
>
> What do you think?

Ooh, absolutely, we should do that in the short term.  Not just
because it fixes the testsuite, but because it increases the
likelihood that folks can reproduce each others' merge problems.  I
want to be able to help users on Windows who report problems and
provide testcases, and making this fix reduces one hurdle toward doing
so.

Re: [PATCH v4 04/24] merge-recursive: provide a better label for diff3 common ancestor

[Jeff King]

On Sat, Aug 17, 2019 at 11:41:24AM -0700, Elijah Newren wrote:

> In commit 7ca56aa07619 ("merge-recursive: add a label for ancestor",
> 2010-03-20), a label was added for the '||||||' line to make it have
> the more informative heading '|||||| merged common ancestors', with
> the statement:
> 
>     It would be nicer to use a more informative label.  Perhaps someone
>     will provide one some day.
> 
> This chosen label was perfectly reasonable when recursiveness kicks in,
> i.e. when there are multiple merge bases.  (I can't think of a better
> label in such cases.)  But it is actually somewhat misleading when there
> is a unique merge base or no merge base.  Change this based on the
> number of merge bases:
>     >=2: "merged common ancestors"
>     1:   <abbreviated commit hash>
>     0:   "<empty tree>"

I got a funny result from this today while rebasing some patches in
git.git, where the base is reported as "00000000". I didn't make a
minimal case, but you can easily reproduce it with:

  cd /your/git/clone
  git fetch https://github.com/peff/git odd-diff3-base
  git checkout -b odd-diff3-base FETCH_HEAD
  git -c merge.conflictstyle=diff3 rebase --onto origin/master HEAD~2

Maybe this has to do with "git apply --build-fake-ancestor" being used
under the hood?

-Peff

Re: [PATCH v4 04/24] merge-recursive: provide a better label for diff3 common ancestor

[Jeff King]

On Mon, Sep 30, 2019 at 05:14:37PM -0400, Jeff King wrote:

> > This chosen label was perfectly reasonable when recursiveness kicks in,
> > i.e. when there are multiple merge bases.  (I can't think of a better
> > label in such cases.)  But it is actually somewhat misleading when there
> > is a unique merge base or no merge base.  Change this based on the
> > number of merge bases:
> >     >=2: "merged common ancestors"
> >     1:   <abbreviated commit hash>
> >     0:   "<empty tree>"
> 
> I got a funny result from this today while rebasing some patches in
> git.git, where the base is reported as "00000000". I didn't make a
> minimal case, but you can easily reproduce it with:
> 
>   cd /your/git/clone
>   git fetch https://github.com/peff/git odd-diff3-base
>   git checkout -b odd-diff3-base FETCH_HEAD
>   git -c merge.conflictstyle=diff3 rebase --onto origin/master HEAD~2
> 
> Maybe this has to do with "git apply --build-fake-ancestor" being used
> under the hood?

Oh, indeed, this seems to be the case for all rebases. Doing:

  git init repo && cd repo
  
  echo base >file && git add file && git commit -m base
  echo master >file && git commit -am master
  git checkout -b side HEAD^
  echo side >file && git commit -am side
  git config merge.conflictstyle diff3
  
  git rebase master

yields:

  <<<<<<< HEAD
  master
  ||||||| 0000000
  base
  =======
  side
  >>>>>>> side

-Peff

[PATCH] merge-recursive: fix the diff3 common ancestor label for virtual commits

[Elijah Newren]

In commit 743474cbfa8b ("merge-recursive: provide a better label for
diff3 common ancestor", 2019-08-17), the label for the common ancestor
was changed from always being

         "merged common ancestors"

to instead be based on the number of merge bases:

    >=2: "merged common ancestors"
      1: <abbreviated commit hash>
      0: "<empty tree>"

Unfortunately, this did not take into account that when we have a single
merge base, that merge base could be fake or constructed.  In such
cases, this resulted in a label of "00000000".  Of course, the previous
label of "merged common ancestors" was also misleading for these cases.
Since we have an API that is explicitly about creating fake commits in
merge_recursive_generic(), we should provide a better label when using
that API.  So, when merge_recursive_generic() is called, set the label
to:

         "constructed merge base"

Note that users of merge_recursive_generic include the builtin commands
git-am (in combination with git apply --build-fake-ancestor),
git-merge-recursive, and git-stash.

Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
Applies to the top of en/merge-recursive-cleanup, which is in next.

 merge-recursive.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index b058741f00..2b7722e350 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3550,6 +3550,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 		merged_merge_bases = make_virtual_commit(opt->repo, tree,
 							 "ancestor");
 		ancestor_name = "empty tree";
+	} else if (opt->ancestor) {
+		ancestor_name = opt->ancestor;
 	} else if (merge_bases) {
 		ancestor_name = "merged common ancestors";
 	} else {
@@ -3689,7 +3691,8 @@ int merge_recursive(struct merge_options *opt,
 {
 	int clean;
 
-	assert(opt->ancestor == NULL);
+	assert(opt->ancestor == NULL ||
+	       !strcmp(opt->ancestor, "constructed merge base"));
 
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
@@ -3741,6 +3744,7 @@ int merge_recursive_generic(struct merge_options *opt,
 					   oid_to_hex(merge_bases[i]));
 			commit_list_insert(base, &ca);
 		}
+		opt->ancestor = "constructed merge base";
 	}
 
 	repo_hold_locked_index(opt->repo, &lock, LOCK_DIE_ON_ERROR);
-- 
2.23.0.25.ge3b5015985

Re: [PATCH] merge-recursive: fix the diff3 common ancestor label for virtual commits

[Elijah Newren]

On Mon, Sep 30, 2019 at 3:55 PM Elijah Newren <newren@gmail.com> wrote:
>
> In commit 743474cbfa8b ("merge-recursive: provide a better label for
> diff3 common ancestor", 2019-08-17), the label for the common ancestor
> was changed from always being
>
>          "merged common ancestors"
>
> to instead be based on the number of merge bases:
>
>     >=2: "merged common ancestors"
>       1: <abbreviated commit hash>
>       0: "<empty tree>"
>
> Unfortunately, this did not take into account that when we have a single
> merge base, that merge base could be fake or constructed.  In such
> cases, this resulted in a label of "00000000".  Of course, the previous
> label of "merged common ancestors" was also misleading for these cases.
> Since we have an API that is explicitly about creating fake commits in
> merge_recursive_generic(), we should provide a better label when using
> that API.  So, when merge_recursive_generic() is called, set the label
> to:
>
>          "constructed merge base"
>
> Note that users of merge_recursive_generic include the builtin commands
> git-am (in combination with git apply --build-fake-ancestor),
> git-merge-recursive, and git-stash.
>
> Reported-by: Jeff King <peff@peff.net>
...
> @@ -3741,6 +3744,7 @@ int merge_recursive_generic(struct merge_options *opt,
>                                            oid_to_hex(merge_bases[i]));
>                         commit_list_insert(base, &ca);
>                 }
> +               opt->ancestor = "constructed merge base";

This should have a 'if (num_merge_bases == 1)' check before it; I'll
be sending a v2 shortly and update the commit message slightly.

[PATCH v2] merge-recursive: fix the diff3 common ancestor label for virtual commits

[Elijah Newren]

In commit 743474cbfa8b ("merge-recursive: provide a better label for
diff3 common ancestor", 2019-08-17), the label for the common ancestor
was changed from always being

         "merged common ancestors"

to instead be based on the number of merge bases:

    >=2: "merged common ancestors"
      1: <abbreviated commit hash>
      0: "<empty tree>"

Unfortunately, this did not take into account that when we have a single
merge base, that merge base could be fake or constructed.  In such
cases, this resulted in a label of "00000000".  Of course, the previous
label of "merged common ancestors" was also misleading for this case.
Since we have an API that is explicitly about creating fake merge base
commits in merge_recursive_generic(), we should provide a better label
when using that API with one merge base.  So, when
merge_recursive_generic() is called with one merge base, set the label
to:

         "constructed merge base"

Note that callers of merge_recursive_generic() include the builtin
commands git-am (in combination with git apply --build-fake-ancestor),
git-merge-recursive, and git-stash.

Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
Applies to the top of en/merge-recursive-cleanup, which is in next.

Changes since v1:
  - We only had a problem if the number of fake merge bases was exactly
    one; update the patch to check for that and update the commit message
    accordingly.

Range-diff:
1:  e3b5015985 ! 1:  3f4444bfd7 merge-recursive: fix the diff3 common ancestor label for virtual commits
    @@ Commit message
         Unfortunately, this did not take into account that when we have a single
         merge base, that merge base could be fake or constructed.  In such
         cases, this resulted in a label of "00000000".  Of course, the previous
    -    label of "merged common ancestors" was also misleading for these cases.
    -    Since we have an API that is explicitly about creating fake commits in
    -    merge_recursive_generic(), we should provide a better label when using
    -    that API.  So, when merge_recursive_generic() is called, set the label
    +    label of "merged common ancestors" was also misleading for this case.
    +    Since we have an API that is explicitly about creating fake merge base
    +    commits in merge_recursive_generic(), we should provide a better label
    +    when using that API with one merge base.  So, when
    +    merge_recursive_generic() is called with one merge base, set the label
         to:
     
                  "constructed merge base"
     
    -    Note that users of merge_recursive_generic include the builtin commands
    -    git-am (in combination with git apply --build-fake-ancestor),
    +    Note that callers of merge_recursive_generic() include the builtin
    +    commands git-am (in combination with git apply --build-fake-ancestor),
         git-merge-recursive, and git-stash.
     
         Reported-by: Jeff King <peff@peff.net>
    @@ merge-recursive.c: int merge_recursive_generic(struct merge_options *opt,
      					   oid_to_hex(merge_bases[i]));
      			commit_list_insert(base, &ca);
      		}
    -+		opt->ancestor = "constructed merge base";
    ++		if (num_merge_bases == 1)
    ++			opt->ancestor = "constructed merge base";
      	}
      
      	repo_hold_locked_index(opt->repo, &lock, LOCK_DIE_ON_ERROR);

 merge-recursive.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index b058741f00..e12d91f48a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3550,6 +3550,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 		merged_merge_bases = make_virtual_commit(opt->repo, tree,
 							 "ancestor");
 		ancestor_name = "empty tree";
+	} else if (opt->ancestor) {
+		ancestor_name = opt->ancestor;
 	} else if (merge_bases) {
 		ancestor_name = "merged common ancestors";
 	} else {
@@ -3689,7 +3691,8 @@ int merge_recursive(struct merge_options *opt,
 {
 	int clean;
 
-	assert(opt->ancestor == NULL);
+	assert(opt->ancestor == NULL ||
+	       !strcmp(opt->ancestor, "constructed merge base"));
 
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
@@ -3741,6 +3744,8 @@ int merge_recursive_generic(struct merge_options *opt,
 					   oid_to_hex(merge_bases[i]));
 			commit_list_insert(base, &ca);
 		}
+		if (num_merge_bases == 1)
+			opt->ancestor = "constructed merge base";
 	}
 
 	repo_hold_locked_index(opt->repo, &lock, LOCK_DIE_ON_ERROR);
-- 
2.23.0.25.ge3b5015985.dirty

Re: [PATCH v2] merge-recursive: fix the diff3 common ancestor label for virtual commits

[Jeff King]

On Mon, Sep 30, 2019 at 11:58:49PM -0700, Elijah Newren wrote:

> In commit 743474cbfa8b ("merge-recursive: provide a better label for
> diff3 common ancestor", 2019-08-17), the label for the common ancestor
> was changed from always being
> 
>          "merged common ancestors"
> 
> to instead be based on the number of merge bases:
> 
>     >=2: "merged common ancestors"
>       1: <abbreviated commit hash>
>       0: "<empty tree>"
> 
> Unfortunately, this did not take into account that when we have a single
> merge base, that merge base could be fake or constructed.  In such
> cases, this resulted in a label of "00000000".  Of course, the previous
> label of "merged common ancestors" was also misleading for this case.

Yeah, I agree the original was not great, either, but the "0000000"
looked like a bug to me. Hey, at least we didn't segfault! :)

> Since we have an API that is explicitly about creating fake merge base
> commits in merge_recursive_generic(), we should provide a better label
> when using that API with one merge base.  So, when
> merge_recursive_generic() is called with one merge base, set the label
> to:
> 
>          "constructed merge base"

That makes perfect sense to me. Thanks for the quick turnaround.

> Changes since v1:
>   - We only had a problem if the number of fake merge bases was exactly
>     one; update the patch to check for that and update the commit message
>     accordingly.

That makes sense. We'd still want to say "merged common ancestors" even
if one of those ancestors was fake.

>  merge-recursive.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)

The patch itself looks good to me (though admittedly I'm not too
familiar with this area).

Maybe worth squashing in this test?

diff --git a/t/t6047-diff3-conflict-markers.sh b/t/t6047-diff3-conflict-markers.sh
index 3fb68e0aae..860542aad0 100755
--- a/t/t6047-diff3-conflict-markers.sh
+++ b/t/t6047-diff3-conflict-markers.sh
@@ -186,4 +186,17 @@ test_expect_success 'check multiple merge bases' '
 	)
 '
 
+test_expect_success 'rebase describes fake ancestor base' '
+	test_create_repo rebase &&
+	(
+		cd rebase &&
+		test_commit base file &&
+		test_commit master file &&
+		git checkout -b side HEAD^ &&
+		test_commit side file &&
+		test_must_fail git -c merge.conflictstyle=diff3 rebase master &&
+		grep "||||||| constructed merge base" file
+	)
+'
+
 test_done

-Peff

[PATCH v3] merge-recursive: fix the diff3 common ancestor label for virtual commits

[Elijah Newren]

In commit 743474cbfa8b ("merge-recursive: provide a better label for
diff3 common ancestor", 2019-08-17), the label for the common ancestor
was changed from always being

         "merged common ancestors"

to instead be based on the number of merge bases:

    >=2: "merged common ancestors"
      1: <abbreviated commit hash>
      0: "<empty tree>"

Unfortunately, this did not take into account that when we have a single
merge base, that merge base could be fake or constructed.  In such
cases, this resulted in a label of "00000000".  Of course, the previous
label of "merged common ancestors" was also misleading for this case.
Since we have an API that is explicitly about creating fake merge base
commits in merge_recursive_generic(), we should provide a better label
when using that API with one merge base.  So, when
merge_recursive_generic() is called with one merge base, set the label
to:

         "constructed merge base"

Note that callers of merge_recursive_generic() include the builtin
commands git-am (in combination with git apply --build-fake-ancestor),
git-merge-recursive, and git-stash.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
Note: Applies to the top of en/merge-recursive-cleanup, which is in next.

Changes since v2:
  - Squashed in the testcase Peff provided and changed his attribution
    from Reported-by to Helped-by due to the testcase

Range-diff:
1:  3f4444bfd7 ! 1:  208e69a4eb merge-recursive: fix the diff3 common ancestor label for virtual commits
    @@ Commit message
         commands git-am (in combination with git apply --build-fake-ancestor),
         git-merge-recursive, and git-stash.
     
    -    Reported-by: Jeff King <peff@peff.net>
    +    Helped-by: Jeff King <peff@peff.net>
         Signed-off-by: Elijah Newren <newren@gmail.com>
     
      ## merge-recursive.c ##
    @@ merge-recursive.c: int merge_recursive_generic(struct merge_options *opt,
      	}
      
      	repo_hold_locked_index(opt->repo, &lock, LOCK_DIE_ON_ERROR);
    +
    + ## t/t6047-diff3-conflict-markers.sh ##
    +@@ t/t6047-diff3-conflict-markers.sh: test_expect_success 'check multiple merge bases' '
    + 	)
    + '
    + 
    ++test_expect_success 'rebase describes fake ancestor base' '
    ++	test_create_repo rebase &&
    ++	(
    ++		cd rebase &&
    ++		test_commit base file &&
    ++		test_commit master file &&
    ++		git checkout -b side HEAD^ &&
    ++		test_commit side file &&
    ++		test_must_fail git -c merge.conflictstyle=diff3 rebase master &&
    ++		grep "||||||| constructed merge base" file
    ++	)
    ++'
    ++
    + test_done

 merge-recursive.c                 |  7 ++++++-
 t/t6047-diff3-conflict-markers.sh | 13 +++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index b058741f00..e12d91f48a 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3550,6 +3550,8 @@ static int merge_recursive_internal(struct merge_options *opt,
 		merged_merge_bases = make_virtual_commit(opt->repo, tree,
 							 "ancestor");
 		ancestor_name = "empty tree";
+	} else if (opt->ancestor) {
+		ancestor_name = opt->ancestor;
 	} else if (merge_bases) {
 		ancestor_name = "merged common ancestors";
 	} else {
@@ -3689,7 +3691,8 @@ int merge_recursive(struct merge_options *opt,
 {
 	int clean;
 
-	assert(opt->ancestor == NULL);
+	assert(opt->ancestor == NULL ||
+	       !strcmp(opt->ancestor, "constructed merge base"));
 
 	if (merge_start(opt, repo_get_commit_tree(opt->repo, h1)))
 		return -1;
@@ -3741,6 +3744,8 @@ int merge_recursive_generic(struct merge_options *opt,
 					   oid_to_hex(merge_bases[i]));
 			commit_list_insert(base, &ca);
 		}
+		if (num_merge_bases == 1)
+			opt->ancestor = "constructed merge base";
 	}
 
 	repo_hold_locked_index(opt->repo, &lock, LOCK_DIE_ON_ERROR);
diff --git a/t/t6047-diff3-conflict-markers.sh b/t/t6047-diff3-conflict-markers.sh
index 3fb68e0aae..860542aad0 100755
--- a/t/t6047-diff3-conflict-markers.sh
+++ b/t/t6047-diff3-conflict-markers.sh
@@ -186,4 +186,17 @@ test_expect_success 'check multiple merge bases' '
 	)
 '
 
+test_expect_success 'rebase describes fake ancestor base' '
+	test_create_repo rebase &&
+	(
+		cd rebase &&
+		test_commit base file &&
+		test_commit master file &&
+		git checkout -b side HEAD^ &&
+		test_commit side file &&
+		test_must_fail git -c merge.conflictstyle=diff3 rebase master &&
+		grep "||||||| constructed merge base" file
+	)
+'
+
 test_done
-- 
2.23.0.25.g3f4444bfd7.dirty

Re: [PATCH v3] merge-recursive: fix the diff3 common ancestor label for virtual commits

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> In commit 743474cbfa8b ("merge-recursive: provide a better label for
> diff3 common ancestor", 2019-08-17), the label for the common ancestor
> was changed from always being
>
>          "merged common ancestors"
>
> to instead be based on the number of merge bases:
>
>     >=2: "merged common ancestors"
>       1: <abbreviated commit hash>
>       0: "<empty tree>"

I just saw, while running "git checkout next && git merge master",
this:

    diff --git a/packfile.c b/packfile.c
    index 3220761fb4..6413693a17 100644
    --- a/packfile.c
    +++ b/packfile.c
    @@ -16,7 +16,22 @@
     #include "object-store.h"
     #include "midx.h"
     #include "commit-graph.h"
    +<<<<<<< HEAD
     #include "promisor-remote.h"
    +||||||| merged common ancestors
    +<<<<<<<<< Temporary merge branch 1
    +||||||||| 瀉3.
    +=========
    +#include "promisor-remote.h"
    +>>>>>>>>> Temporary merge branch 2
    +||||||||| 瀉3.
    +>>>>>>>>>>> Temporary merge branch 2
    +=========
    +#include "promisor-remote.h"
    +>>>>>>>>> Temporary merge branch 2
    +=======
    +#include "promisor-remote.h"
    +>>>>>>> master

     char *odb_pack_name(struct strbuf *buf,
                        const unsigned char *hash,

where the common ancestor blocks are marked with random garbage.

I do not have time or patience to bisect and do not know if this is
a regression around this commit, but it smells suspiciously close.

[PATCH] merge-recursive: fix the fix to the diff3 common ancestor label

[Elijah Newren]

In commit 208e69a4ebce ("merge-recursive: fix the diff3 common ancestor
label for virtual commits", 2019-09-30) which was a fix to commit
743474cbfa8b ("merge-recursive: provide a better label for diff3 common
ancestor", 2019-08-17), the label for the common ancestor was changed
from always being

         "merged common ancestors"

to instead be based on the number of merge bases and whether the merge
base was a real commit or a virtual one:

    >=2: "merged common ancestors"
      1, via merge_recursive_generic: "constructed merge base"
      1, otherwise: <abbreviated commit hash>
      0: "<empty tree>"

The handling for "constructed merge base" worked by allowing
opt->ancestor to be set in merge_recursive_generic(), so we payed
attention to the setting of that variable in merge_recursive_internal().
Now, for the outer merge, the code flow was simply the following:

	ancestor_name = "merged merge bases"
	loop over merge_bases: merge_recursive_internal()

The first merge base not needing recursion would determine its own
ancestor_name however necessary and thus run

	ancestor_name = $SOMETHING
	empty loop over merge_bases...
	opt->ancestor = ancestor_name
        merge_trees_internal()

Now, the next set of merge_bases that would need to be merged after this
particular merge had completed would note that opt->ancestor has been
set to something (to a local ancestor_name variable that has since been
popped off the stack), and thus it would run:

	... else if (opt->ancestor) {
		ancestor_name = opt->ancestor;  /* OOPS! */
        loop over merge_bases: merge_recursive_internal()
        opt->ancestor = ancestor_name
        merge_trees_internal()

This resulted in garbage strings being printed for the virtual merge
bases, which was visible in git.git by just merging commit b744c3af07
into commit 6d8cb22a4f.  There are two ways to fix this: set
opt->ancestor to NULL after using it to avoid re-use, or add a
!opt->priv->call_depth check to the if block for using a pre-defined
opt->ancestor.  Apply both fixes.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-recursive.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index e12d91f48a..2653ba9a50 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3550,7 +3550,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 		merged_merge_bases = make_virtual_commit(opt->repo, tree,
 							 "ancestor");
 		ancestor_name = "empty tree";
-	} else if (opt->ancestor) {
+	} else if (opt->ancestor && !opt->priv->call_depth) {
 		ancestor_name = opt->ancestor;
 	} else if (merge_bases) {
 		ancestor_name = "merged common ancestors";
@@ -3600,6 +3600,7 @@ static int merge_recursive_internal(struct merge_options *opt,
 							  merged_merge_bases),
 				     &result_tree);
 	strbuf_release(&merge_base_abbrev);
+	opt->ancestor = NULL;  /* avoid accidental re-use of opt->ancestor */
 	if (clean < 0) {
 		flush_output(opt);
 		return clean;
-- 
2.23.0.26.gfc82117b87

Re: [PATCH] merge-recursive: fix the fix to the diff3 common ancestor label

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> This resulted in garbage strings being printed for the virtual merge
> bases, which was visible in git.git by just merging commit b744c3af07
> into commit 6d8cb22a4f.  There are two ways to fix this: set
> opt->ancestor to NULL after using it to avoid re-use, or add a
> !opt->priv->call_depth check to the if block for using a pre-defined
> opt->ancestor.  Apply both fixes.

Thanks for quickly fixing this.  Will apply.

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-recursive.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index e12d91f48a..2653ba9a50 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3550,7 +3550,7 @@ static int merge_recursive_internal(struct merge_options *opt,
>  		merged_merge_bases = make_virtual_commit(opt->repo, tree,
>  							 "ancestor");
>  		ancestor_name = "empty tree";
> -	} else if (opt->ancestor) {
> +	} else if (opt->ancestor && !opt->priv->call_depth) {
>  		ancestor_name = opt->ancestor;
>  	} else if (merge_bases) {
>  		ancestor_name = "merged common ancestors";
> @@ -3600,6 +3600,7 @@ static int merge_recursive_internal(struct merge_options *opt,
>  							  merged_merge_bases),
>  				     &result_tree);
>  	strbuf_release(&merge_base_abbrev);
> +	opt->ancestor = NULL;  /* avoid accidental re-use of opt->ancestor */
>  	if (clean < 0) {
>  		flush_output(opt);
>  		return clean;

Re: [PATCH] merge-recursive: fix the fix to the diff3 common ancestor label

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> In commit 208e69a4ebce ("merge-recursive: fix the diff3 common ancestor

I think the above was an earlier incarntion of what is now known as
8e4ec337 ("merge-recursive: fix the diff3 common ancestor label for
virtual commits", 2019-10-01).

> label for virtual commits", 2019-09-30) which was a fix to commit
> 743474cbfa8b ("merge-recursive: provide a better label for diff3 common
> ...
> The handling for "constructed merge base" worked by allowing
> opt->ancestor to be set in merge_recursive_generic(), so we payed

s/payed/paid/

> attention to the setting of that variable in merge_recursive_internal().

Thanks, again.

Re: [PATCH] merge-recursive: fix the fix to the diff3 common ancestor label

[Elijah Newren]

On Mon, Oct 7, 2019 at 7:36 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > In commit 208e69a4ebce ("merge-recursive: fix the diff3 common ancestor
>
> I think the above was an earlier incarntion of what is now known as
> 8e4ec337 ("merge-recursive: fix the diff3 common ancestor label for
> virtual commits", 2019-10-01).

Oops, yes.

> > label for virtual commits", 2019-09-30) which was a fix to commit
> > 743474cbfa8b ("merge-recursive: provide a better label for diff3 common
> > ...
> > The handling for "constructed merge base" worked by allowing
> > opt->ancestor to be set in merge_recursive_generic(), so we payed
>
> s/payed/paid/

Ugh, two simple mistakes in the commit message.  I see you've not only
proofread the commit but fixed up the commit message for me in pu;
thanks.

Elijah