[PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Elijah Newren via GitGitGadget]

Note1: Depends on en/remerge-diff (but no need to pick it up; it's still
RFC).

(Note2: This is a continuation of my series at [0], but I can't change the
base repository for a pull request in GitHub so I had to open a new PR and
that makes it look like a new series. [0]
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/)

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs.

== Updates Log ==

Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
suggestions and comments on v2):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since v1 (thanks to Johannes Altmanninger and Fabian for suggestions
on v1):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

== Other notes ==

Stuff intentionally NOT included, but which others seemed to feel strongly
about; they'd need to convince me more on these:

 * Any form of diff output[1]
 * A way to omit printing the generated tree hash[2][3] In regards to these,
   also see also some of the new info in Documentation/git-merge-tree.txt,
   namely the expanded paragraph on "the second form is deprecated" in the
   description as regards diff output usability and performance, and also
   the final paragraph of the new "Mistakes to avoid" section in regards to
   tree hash.

[1]
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2201101427440.339@tvgsbejvaqbjf.bet/
(section starting with "Providing a tree") [2]
https://lore.kernel.org/git/CABPp-BHvXrP0sTTmuTYfACoJTCcm9+wk_f441nj4TstrmQdqMQ@mail.gmail.com/
(sections starting with "avoid printing" and "Where did I suggest") [3]
https://lore.kernel.org/git/CABPp-BGdbh=HM7jA+_RTwSWVcMr_qvEY7RoNXooeBT2NB4Ubzw@mail.gmail.com/
(section starting with "Providing a tree object")

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-ort: allow update messages to be written to different file
    stream
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

 Documentation/git-merge-tree.txt | 182 +++++++++++++++++++++++++++++--
 builtin/merge-tree.c             | 178 ++++++++++++++++++++++++++++--
 git.c                            |   2 +-
 merge-ort.c                      | 109 ++++++++++++------
 merge-ort.h                      |  30 +++++
 t/t4301-merge-tree-real.sh       | 163 +++++++++++++++++++++++++++
 6 files changed, 603 insertions(+), 61 deletions(-)
 create mode 100755 t/t4301-merge-tree-real.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1122
-- 
gitgitgadget

[PATCH 01/12] merge-tree: rename merge_trees() to trivial_merge_trees()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget

[PATCH 02/12] merge-tree: move logic for existing merge into new function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget

[PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 67 ++++++++++++++++++++++++++++++++++++++------
 git.c                |  2 +-
 2 files changed, 59 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..33e47cc1534 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,57 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+struct merge_tree_options {
+	int real;
+	int trivial;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_BOOL(0, "write-tree", &o.real,
+			 N_("do a real merge instead of a trivial merge")),
+		OPT_BOOL(0, "trivial-merge", &o.trivial,
+			 N_("do a trivial merge only")),
+		OPT_END()
+	};
+
+	/* Check for a request for basic help */
+	if (argc == 2 && !strcmp(argv[1], "-h"))
+		usage_with_options(merge_tree_usage, mt_options);
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, 0);
+	if (o.real && o.trivial)
+		die(_("--write-tree and --trivial-merge are incompatible"));
+	if (o.real || o.trivial) {
+		expected_remaining_argc = (o.real ? 2 : 3);
+		if (argc != expected_remaining_argc)
+			usage_with_options(merge_tree_usage, mt_options);
+	} else {
+		if (argc < 2 || argc > 3)
+			usage_with_options(merge_tree_usage, mt_options);
+		o.real = (argc == 2);
+	}
+
+	/* Do the relevant type of merge */
+	if (o.real)
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget

[PATCH 04/12] merge-tree: implement real merges

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean) or 1 (conflicts present)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 74 ++++++++++++++++++++++-----
 builtin/merge-tree.c             | 55 +++++++++++++++++++-
 t/t4301-merge-tree-real.sh       | 87 ++++++++++++++++++++++++++++++++
 3 files changed, 203 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-real.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..b900bc1362c 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,76 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The first form will merge the two branches, doing a full recursive
+merge with rename detection.  The rest of this manual (other than the
+next paragraph) describes the first form in more detail -- including
+options, output format, exit status, and usage notes.
+
+The second form is deprecated; it is kept for backward compatibility
+reasons but may be deleted in the future.  It will only do a trivial
+merge.  It reads three tree-ish, and outputs trivial merge results and
+conflicting stages to the standard output in a semi-diff format.
+Since this was designed for higher level scripts to consume and merge
+the results back into the index, it omits entries that match
+<branch1>.  The result of this second form is is similar to what
+three-way 'git read-tree -m' does, but instead of storing the results
+in the index, the command outputs the entries to the standard output.
+This form not only has limited applicability, the output format is
+also difficult to work with, and it will generally be less performant
+than the first form even on successful merges (especially if working
+in large repositories).  The remainder of this manual will only
+discuss the first form.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1.
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
+be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+However, it does not quite fit into the same category of low-level
+plumbing commands since the possibility of merge conflicts give it a
+much higher chance of the command not succeeding.
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 33e47cc1534..0c19639594d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -393,7 +396,57 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+	/*
+	 * TODO: Support subtree and other -X options?
+	if (use_strategies_nr == 1 &&
+	    !strcmp(use_strategies[0]->name, "subtree"))
+		opt.subtree_shift = "";
+	for (x = 0; x < xopts_nr; x++)
+		if (parse_merge_opt(&opt, xopts[x]))
+			die(_("Unknown strategy option: -X%s"), xopts[x]);
+	*/
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
+	opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	else if (!result.clean)
+		printf(_("Conflicts!\n"));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
new file mode 100755
index 00000000000..e03688515c5
--- /dev/null
+++ b/t/t4301-merge-tree-real.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+}
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(cat .git/AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	cat tmp | sed -e s/HEAD/side1/ >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget

[PATCH 05/12] merge-ort: split out a separate display_update_messages() function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

No functional changes included in this patch; it's just a preparatory
step to allow the printed messages to be handled differently by other
callers, such as in `git merge-tree --write-tree`.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 77 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 36 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..f9e35b0f96b 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4273,42 +4312,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
 
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH 06/12] merge-ort: allow update messages to be written to different file stream

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This modifies the new display_update_messages() function to allow
printing to somewhere other than stdout.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 7 ++++---
 merge-ort.h | 3 ++-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index f9e35b0f96b..b78dde55ad9 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 }
 
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result)
+				   struct merge_result *result,
+				   FILE *stream)
 {
 	struct merge_options_internal *opti = result->priv;
 	struct hashmap_iter iter;
@@ -4263,7 +4264,7 @@ void merge_display_update_messages(struct merge_options *opt,
 	for (i = 0; i < olist.nr; ++i) {
 		struct strbuf *sb = olist.items[i].util;
 
-		printf("%s", sb->buf);
+		fprintf(stream, "%s", sb->buf);
 	}
 	string_list_clear(&olist, 0);
 
@@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
 	}
 
 	if (display_update_msgs)
-		merge_display_update_messages(opt, result);
+		merge_display_update_messages(opt, result, stdout);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..d643b47cb7c 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
  * so only call this when bypassing merge_switch_to_result().
  */
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result);
+				   struct merge_result *result,
+				   FILE *stream);
 
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
-- 
gitgitgadget

[PATCH 07/12] merge-tree: support including merge messages in output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
conflicts occur (such as file/directory, modify/delete, add/add with
differing modes, rename/rename (1to2), etc.), these informational
messages are often the only notification since these conflicts are not
representable in the contents of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 24 +++++++++++++----
 t/t4301-merge-tree-real.sh       | 21 +++++++++++++++
 3 files changed, 78 insertions(+), 12 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index b900bc1362c..fd7a867de60 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
 
 DESCRIPTION
@@ -38,17 +38,47 @@ than the first form even on successful merges (especially if working
 in large repositories).  The remainder of this manual will only
 discuss the first form.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
+
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -72,7 +102,8 @@ be used as a part of a series of steps such as
 
 However, it does not quite fit into the same category of low-level
 plumbing commands since the possibility of merge conflicts give it a
-much higher chance of the command not succeeding.
+much higher chance of the command not succeeding (and NEWTREE containing
+a bunch of stuff other than just a toplevel tree).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 0c19639594d..560640ad911 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -391,6 +391,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int real;
 	int trivial;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
 		commit_list_insert(j->item, &merge_bases);
 
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
-	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+
 	if (result.clean < 0)
 		die(_("failure to merge"));
-	else if (!result.clean)
-		printf(_("Conflicts!\n"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result, stdout);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -464,6 +473,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			 N_("do a real merge instead of a trivial merge")),
 		OPT_BOOL(0, "trivial-merge", &o.trivial,
 			 N_("do a trivial merge only")),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
@@ -472,10 +483,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 		usage_with_options(merge_tree_usage, mt_options);
 
 	/* Parse arguments */
+	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, 0);
 	if (o.real && o.trivial)
 		die(_("--write-tree and --trivial-merge are incompatible"));
+	if (!o.real && original_argc < argc)
+		die(_("--write-tree must be specified if any other options are"));
 	if (o.real || o.trivial) {
 		expected_remaining_argc = (o.real ? 2 : 3);
 		if (argc != expected_remaining_argc)
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index e03688515c5..c34f8e6c1ed 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -84,4 +84,25 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u` -- conflicted
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index b78dde55ad9..5e7cea6cc8f 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index d643b47cb7c..e635a294ea8 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result,
 				   FILE *stream);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH 09/12] merge-tree: provide a list of which files have conflicts

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-real.sh       | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index fd7a867de60..041a4ac2785 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -58,6 +58,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -69,6 +70,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 560640ad911..d8eeeb3f306 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -395,7 +398,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result, stdout);
@@ -502,7 +522,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.real)
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index c34f8e6c1ed..43c9950dedb 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -94,6 +94,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -105,4 +107,13 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide an --exclude-oids-and-modes option for users to exclude the
mode, oid, and stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 30 ++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-real.sh       | 26 ++++++++++++++++++++++++--
 3 files changed, 58 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 041a4ac2785..beb08269a70 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -41,6 +41,11 @@ discuss the first form.
 OPTIONS
 -------
 
+--exclude-oids-and-modes::
+	Instead of writing a list of (mode, oid, stage, path) tuples
+	to output for conflicted files, just provide a list of
+	filenames with conflicts.
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -58,7 +63,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -70,18 +75,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--exclude-oids-and-modes` option is passed, the mode, object, and
+stage will be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -113,6 +123,14 @@ plumbing commands since the possibility of merge conflicts give it a
 much higher chance of the command not succeeding (and NEWTREE containing
 a bunch of stuff other than just a toplevel tree).
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d8eeeb3f306..7aa7f9fd54a 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -395,6 +395,7 @@ struct merge_tree_options {
 	int real;
 	int trivial;
 	int show_messages;
+	int exclude_oids_and_modes;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -461,7 +462,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->exclude_oids_and_modes)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -495,6 +500,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			 N_("do a trivial merge only")),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F(0, "exclude-oids-and-modes",
+			   &o.exclude_oids_and_modes,
+			   N_("list conflicted files without oids and modes"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index 43c9950dedb..e921115cd2a 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -46,6 +46,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(cat .git/AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -85,7 +86,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --exclude-oids-and-modes side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -108,7 +109,7 @@ test_expect_success 'test conflict notices and such' '
 '
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-oids-and-modes side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -116,4 +117,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH 11/12] merge-tree: add a --allow-unrelated-histories flag

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-real.sh       | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index beb08269a70..df10a5963c7 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -52,6 +52,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 7aa7f9fd54a..98441d5e05b 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -394,6 +394,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int real;
 	int trivial;
+	int allow_unrelated_histories;
 	int show_messages;
 	int exclude_oids_and_modes;
 };
@@ -440,7 +441,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -504,6 +505,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.exclude_oids_and_modes,
 			   N_("list conflicted files without oids and modes"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index e921115cd2a..a0447410655 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -37,7 +37,13 @@ test_expect_success setup '
 	>whatever/empty &&
 	git add numbers greeting whatever/empty &&
 	test_tick &&
-	git commit -m other-modifications
+	git commit -m other-modifications &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Content merge and a few conflicts' '
@@ -138,4 +144,20 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH 12/12] git-merge-tree.txt: add a section on potentional usage mistakes

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index df10a5963c7..3c1aa0ffbae 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -136,6 +136,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function

[René Scharfe]

Am 22.01.22 um 22:55 schrieb Elijah Newren via GitGitGadget:
> From: Elijah Newren <newren@gmail.com>
>
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.
>
> Note that real merges differ from trivial merges in that they handle:
>   - three way content merges
>   - recursive ancestor consolidation
>   - renames
>   - proper directory/file conflict handling
>   - etc.
> Basically all the stuff you'd expect from `git merge`, just without
> updating the index and working tree.  The initial shell added here does
> nothing more than die with "real merges are not yet implemented", but
> that will be fixed in subsequent commits.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/merge-tree.c | 67 ++++++++++++++++++++++++++++++++++++++------
>  git.c                |  2 +-
>  2 files changed, 59 insertions(+), 10 deletions(-)
>
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 914ec960b7e..33e47cc1534 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -3,13 +3,12 @@
>  #include "tree-walk.h"
>  #include "xdiff-interface.h"
>  #include "object-store.h"
> +#include "parse-options.h"
>  #include "repository.h"
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>
> -static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
> -
>  struct merge_list {
>  	struct merge_list *next;
>  	struct merge_list *link;	/* other stages for this object */
> @@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
>  	return buf;
>  }
>
> -static int trivial_merge(int argc, const char **argv)
> +static int trivial_merge(const char *base,
> +			 const char *branch1,
> +			 const char *branch2)
>  {
>  	struct repository *r = the_repository;
>  	struct tree_desc t[3];
>  	void *buf1, *buf2, *buf3;
>
> -	buf1 = get_tree_descriptor(r, t+0, argv[1]);
> -	buf2 = get_tree_descriptor(r, t+1, argv[2]);
> -	buf3 = get_tree_descriptor(r, t+2, argv[3]);
> +	buf1 = get_tree_descriptor(r, t+0, base);
> +	buf2 = get_tree_descriptor(r, t+1, branch1);
> +	buf3 = get_tree_descriptor(r, t+2, branch2);
>  	trivial_merge_trees(t, "");
>  	free(buf1);
>  	free(buf2);
> @@ -384,9 +385,57 @@ static int trivial_merge(int argc, const char **argv)
>  	return 0;
>  }
>
> +struct merge_tree_options {
> +	int real;
> +	int trivial;
> +};
> +
> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_BOOL(0, "write-tree", &o.real,
> +			 N_("do a real merge instead of a trivial merge")),
> +		OPT_BOOL(0, "trivial-merge", &o.trivial,
> +			 N_("do a trivial merge only")),
> +		OPT_END()
> +	};
> +
> +	/* Check for a request for basic help */
> +	if (argc == 2 && !strcmp(argv[1], "-h"))
> +		usage_with_options(merge_tree_usage, mt_options);

This is unnecessary; parse_options() handles -h already.

> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, 0);
> +	if (o.real && o.trivial)
> +		die(_("--write-tree and --trivial-merge are incompatible"));

12909b6b8a (i18n: turn "options are incompatible" into "cannot be used
together", 2022-01-05) standardized messages of that kind; let's stick
to that to simplify translation:

		die(_("options '%s' and '%s' cannot be used together"),
		    "--write-tree", "--trivial-merge");

> +	if (o.real || o.trivial) {
> +		expected_remaining_argc = (o.real ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.real = (argc == 2);
> +	}
> +
> +	/* Do the relevant type of merge */
> +	if (o.real)
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/git.c b/git.c
> index 5ff21be21f3..6090a1289db 100644
> --- a/git.c
> +++ b/git.c
> @@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
>  	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
> -	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
> +	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
>  	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
>  	{ "mktree", cmd_mktree, RUN_SETUP },
>  	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },

Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.
>
> Note that real merges differ from trivial merges in that they handle:
>   - three way content merges
>   - recursive ancestor consolidation
>   - renames
>   - proper directory/file conflict handling
>   - etc.
> Basically all the stuff you'd expect from `git merge`, just without
> updating the index and working tree.  The initial shell added here does
> nothing more than die with "real merges are not yet implemented", but
> that will be fixed in subsequent commits.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/merge-tree.c | 67 ++++++++++++++++++++++++++++++++++++++------
>  git.c                |  2 +-
>  2 files changed, 59 insertions(+), 10 deletions(-)
>
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 914ec960b7e..33e47cc1534 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -3,13 +3,12 @@
>  #include "tree-walk.h"
>  #include "xdiff-interface.h"
>  #include "object-store.h"
> +#include "parse-options.h"
>  #include "repository.h"
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>  
> -static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
> -
>  struct merge_list {
>  	struct merge_list *next;
>  	struct merge_list *link;	/* other stages for this object */
> @@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
>  	return buf;
>  }
>  
> -static int trivial_merge(int argc, const char **argv)
> +static int trivial_merge(const char *base,
> +			 const char *branch1,
> +			 const char *branch2)
>  {
>  	struct repository *r = the_repository;
>  	struct tree_desc t[3];
>  	void *buf1, *buf2, *buf3;
>  
> -	buf1 = get_tree_descriptor(r, t+0, argv[1]);
> -	buf2 = get_tree_descriptor(r, t+1, argv[2]);
> -	buf3 = get_tree_descriptor(r, t+2, argv[3]);
> +	buf1 = get_tree_descriptor(r, t+0, base);
> +	buf2 = get_tree_descriptor(r, t+1, branch1);
> +	buf3 = get_tree_descriptor(r, t+2, branch2);
>  	trivial_merge_trees(t, "");
>  	free(buf1);
>  	free(buf2);
> @@ -384,9 +385,57 @@ static int trivial_merge(int argc, const char **argv)
>  	return 0;
>  }
>  
> +struct merge_tree_options {
> +	int real;
> +	int trivial;
> +};
> +
> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_BOOL(0, "write-tree", &o.real,
> +			 N_("do a real merge instead of a trivial merge")),
> +		OPT_BOOL(0, "trivial-merge", &o.trivial,
> +			 N_("do a trivial merge only")),
> +		OPT_END()
> +	};
> +
> +	/* Check for a request for basic help */
> +	if (argc == 2 && !strcmp(argv[1], "-h"))
> +		usage_with_options(merge_tree_usage, mt_options);

Is this bit cargo-culted from something else, perhaps
non-parse-options.c usage? I don't think this is needed, the
parse_options() below intercepts "-h" by default.

> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, 0);
> +	if (o.real && o.trivial)
> +		die(_("--write-tree and --trivial-merge are incompatible"));

Shouldn't those two just be OPT_CMDMODE()? Then you get this
incompatibility checking for free. See 485fd2c3dae (cat-file: make
--batch-all-objects a CMDMODE, 2021-12-28).

> +	if (o.real || o.trivial) {
> +		expected_remaining_argc = (o.real ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.real = (argc == 2);
> +	}

And this can also be done like this, but I wonder if using
PARSE_OPT_STOP_AT_NON_OPTION and then routing to a sub-function wouldn't
be better, i.e. to treat these like sub-commands if they've got
different arity etc.

> +	/* Do the relevant type of merge */
> +	if (o.real)
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/git.c b/git.c
> index 5ff21be21f3..6090a1289db 100644
> --- a/git.c
> +++ b/git.c
> @@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
>  	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
> -	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
> +	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
>  	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
>  	{ "mktree", cmd_mktree, RUN_SETUP },
>  	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },

Re: [PATCH 04/12] merge-tree: implement real merges

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> +	/*
> +	 * TODO: Support subtree and other -X options?
> +	if (use_strategies_nr == 1 &&
> +	    !strcmp(use_strategies[0]->name, "subtree"))
> +		opt.subtree_shift = "";
> +	for (x = 0; x < xopts_nr; x++)
> +		if (parse_merge_opt(&opt, xopts[x]))

Better omitted WIP code in a non-RFC series?

> +			die(_("Unknown strategy option: -X%s"), xopts[x]);

As a general issue with this series, die(), BUG() etc. messages should
start with a non-capital letter.

> +	printf("%s\n", oid_to_hex(&result.tree->object.oid));

And for both this...

> +		printf(_("Conflicts!\n"));

... and this we can just use puts(). For the former it's just less code,
but for the latter translators also don't need to see the always-there
\n in the translated message.

> +# This test is ort-specific
> +test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {

Is this ${} trickery really needed? We're not testing with "set -u". So just:
	
	if test "$GIT_..." != "ort"
	then
		...
	fi

> +test_expect_success 'Barf on too many arguments' '
> +	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
> +
> +	grep "^usage: git merge-tree" expect
> +'

Nit: In most other tests these usage assertions are at the top of the
test, and for those we also make do with just testing the 129 exit code,
which is probably enough here too...

Re: [PATCH 05/12] merge-ort: split out a separate display_update_messages() function

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
> [...]
> +	/* Hack to pre-allocate olist to the desired size */
> +	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> +		   olist.alloc);

Perhaps just add a string_list_grow()? But I wonder if this is really
needed v.s. just using the default growing pattern here.

> +
> +	/* Put every entry from output into olist, then sort */
> +	strmap_for_each_entry(&opti->output, &iter, e) {
> +		string_list_append(&olist, e->key)->util = e->value;
> +	}
> +	string_list_sort(&olist);
> +
> +	/* Iterate over the items, printing them */
> +	for (i = 0; i < olist.nr; ++i) {
> +		struct strbuf *sb = olist.items[i].util;
> +
> +		printf("%s", sb->buf);
> +	}

Shorter/nicer:
	
	for_each_string_list_item(item, &olist) {
		struct strbuf *sb = item->util;
	        puts(sb->buf);
	}
	
> -	if (display_update_msgs) {
> -		struct merge_options_internal *opti = result->priv;
> -		struct hashmap_iter iter;
> -		struct strmap_entry *e;
> -		struct string_list olist = STRING_LIST_INIT_NODUP;
> -		int i;
> -
> -		if (opt->record_conflict_msgs_as_headers)
> -			BUG("Either display conflict messages or record them as headers, not both");
> -
> -		trace2_region_enter("merge", "display messages", opt->repo);
> -
> -		/* Hack to pre-allocate olist to the desired size */
> -		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> -			   olist.alloc);
> -
> -		/* Put every entry from output into olist, then sort */
> -		strmap_for_each_entry(&opti->output, &iter, e) {
> -			string_list_append(&olist, e->key)->util = e->value;
> -		}
> -		string_list_sort(&olist);
> -
> -		/* Iterate over the items, printing them */
> -		for (i = 0; i < olist.nr; ++i) {
> -			struct strbuf *sb = olist.items[i].util;
> -
> -			printf("%s", sb->buf);
> -		}
> -		string_list_clear(&olist, 0);

Ah, at this point I see you're just moving code around :) Sending this
anyway in case it's useful :)

Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Callers of `git merge-tree --write-tree` will often want to know which
> files had conflicts.  While they could potentially attempt to parse the
> CONFLICT notices printed, those messages are not meant to be machine
> readable.  Provide a simpler mechanism of just printing the files (in
> the same format as `git ls-files` with quoting, but restricted to
> [...]
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 560640ad911..d8eeeb3f306 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -11,6 +11,9 @@
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
> +#include "quote.h"
> +
> +static int line_termination = '\n';

But unlike ls-files we don't do anything with line_termination as a !=
'\n', maybe in a later commit?

>  struct merge_list {
>  	struct merge_list *next;
> @@ -395,7 +398,8 @@ struct merge_tree_options {
>  };
>  
>  static int real_merge(struct merge_tree_options *o,
> -		      const char *branch1, const char *branch2)
> +		      const char *branch1, const char *branch2,
> +		      const char *prefix)
>  {
>  	struct commit *parent1, *parent2;
>  	struct commit_list *common;
> @@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
>  		o->show_messages = !result.clean;
>  
>  	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +	if (!result.clean) {
> +		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> +		const char *last = NULL;
> +		int i;
> +
> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);

But here it's never \0 or whatever.

Re: [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info

[Ævar Arnfjörð Bjarmason]

On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Much like `git merge` updates the index with information of the form
>     (mode, oid, stage, name)
> provide this output for conflicted files for merge-tree as well.
> Provide an --exclude-oids-and-modes option for users to exclude the
> mode, oid, and stage and only get the list of conflicted filenames.
> [...]
> +--exclude-oids-and-modes::
> +	Instead of writing a list of (mode, oid, stage, path) tuples
> +	to output for conflicted files, just provide a list of
> +	filenames with conflicts.
> +
> [...]
> -This is a sequence of lines containing a filename on each line, quoted
> -as explained for the configuration variable `core.quotePath` (see
> -linkgit:git-config[1]).
> +This is a sequence of lines with the format
> +
> +	<mode> <object> <stage> <filename>
> +
> +The filename will be quoted as explained for the configuration
> +variable `core.quotePath` (see linkgit:git-config[1]).  However, if
> +the `--exclude-oids-and-modes` option is passed, the mode, object, and
> +stage will be omitted.
>  
>  Informational messages
>  ~~~~~~~~~~~~~~~~~~~~~~
>  
>  This always starts with a blank line to separate it from the previous
> -section, and then has free-form messages about the merge, such as:
> +sections, and then has free-form messages about the merge, such as:
>  
>    * "Auto-merging <file>"
>    * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
> @@ -113,6 +123,14 @@ plumbing commands since the possibility of merge conflicts give it a
>  much higher chance of the command not succeeding (and NEWTREE containing
>  a bunch of stuff other than just a toplevel tree).
>  
> +git-merge-tree was written to provide users with the same information
> +that they'd have access to if using `git merge`:
> +  * what would be written to the working tree (the <OID of toplevel tree>)
> +  * the higher order stages that would be written to the index (the
> +    <Conflicted file info>)
> +  * any messages that would have been printed to stdout (the <Informational
> +    messages>)
> +
>  GIT
>  ---
>  Part of the linkgit:git[1] suite
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index d8eeeb3f306..7aa7f9fd54a 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -395,6 +395,7 @@ struct merge_tree_options {
>  	int real;
>  	int trivial;
>  	int show_messages;
> +	int exclude_oids_and_modes;
>  };
>  
>  static int real_merge(struct merge_tree_options *o,
> @@ -461,7 +462,11 @@ static int real_merge(struct merge_tree_options *o,
>  		merge_get_conflicted_files(&result, &conflicted_files);
>  		for (i = 0; i < conflicted_files.nr; i++) {
>  			const char *name = conflicted_files.items[i].string;
> -			if (last && !strcmp(last, name))
> +			struct stage_info *c = conflicted_files.items[i].util;
> +			if (!o->exclude_oids_and_modes)
> +				printf("%06o %s %d\t",
> +				       c->mode, oid_to_hex(&c->oid), c->stage);
> +			else if (last && !strcmp(last, name))
>  				continue;
>  			write_name_quoted_relative(
>  				name, prefix, stdout, line_termination);
> @@ -495,6 +500,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			 N_("do a trivial merge only")),
>  		OPT_BOOL(0, "messages", &o.show_messages,
>  			 N_("also show informational/conflict messages")),
> +		OPT_BOOL_F(0, "exclude-oids-and-modes",
> +			   &o.exclude_oids_and_modes,
> +			   N_("list conflicted files without oids and modes"),
> +			   PARSE_OPT_NONEG),
>  		OPT_END()
>  	};

Perhaps this really is the last formatting information anyone will want,
but with a default of "<mode> <object> <stage> <filename>" being made
"<stage> <filename>" with --exclude-oids-and-modes perhaps we'll want
--exclude-all-except-filename etc. later.

It seems a lot simpler for new code to just support a --conflict-format
option, lifting some code from the in-flight
https://lore.kernel.org/git/db058bf670c5668fc5b95baf83667cc282cb739b.1641978175.git.dyroneteng@gmail.com/

I.e. that inner loop becomes a slightly more verbose strbuf_expand(),
but it's all easy boilerplate code.

Then we just feed "%(objectmode) %(objectname) %(objectstage)
%(filename)" into it by default, and allow the user to customize it.

Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren]

On Sun, Jan 23, 2022 at 12:05 AM René Scharfe <l.s.r@web.de> wrote:
>
> Am 22.01.22 um 22:55 schrieb Elijah Newren via GitGitGadget:
...
> > +     /* Check for a request for basic help */
> > +     if (argc == 2 && !strcmp(argv[1], "-h"))
> > +             usage_with_options(merge_tree_usage, mt_options);
>
> This is unnecessary; parse_options() handles -h already.
>
> > +
> > +     /* Parse arguments */
> > +     argc = parse_options(argc, argv, prefix, mt_options,
> > +                          merge_tree_usage, 0);
> > +     if (o.real && o.trivial)
> > +             die(_("--write-tree and --trivial-merge are incompatible"));
>
> 12909b6b8a (i18n: turn "options are incompatible" into "cannot be used
> together", 2022-01-05) standardized messages of that kind; let's stick
> to that to simplify translation:
>
>                 die(_("options '%s' and '%s' cannot be used together"),
>                     "--write-tree", "--trivial-merge");

Ah, thanks for both the pointers; will fix.

Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren]

On Mon, Jan 24, 2022 at 1:50 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
...
> > +     /* Check for a request for basic help */
> > +     if (argc == 2 && !strcmp(argv[1], "-h"))
> > +             usage_with_options(merge_tree_usage, mt_options);
>
> Is this bit cargo-culted from something else, perhaps
> non-parse-options.c usage? I don't think this is needed, the
> parse_options() below intercepts "-h" by default.

Yep, sure was cargo-culted from somewhere else (my parse-options usage
always is), but I'm pretty sure it was from another place also using
parse-options.  Probably one of these 15 places:

 $ comm -12 <(git grep -l parse-options builtin/ | sort) <(git grep -l
strcmp.*-h\\b builtin/ | sort)
builtin/am.c
builtin/branch.c
builtin/checkout-index.c
builtin/checkout--worker.c
builtin/commit.c
builtin/commit-tree.c
builtin/gc.c
builtin/ls-files.c
builtin/merge.c
builtin/merge-tree.c
builtin/rebase.c
builtin/rev-parse.c
builtin/sparse-checkout.c
builtin/submodule--helper.c
builtin/update-index.c

> > +     /* Parse arguments */
> > +     argc = parse_options(argc, argv, prefix, mt_options,
> > +                          merge_tree_usage, 0);
> > +     if (o.real && o.trivial)
> > +             die(_("--write-tree and --trivial-merge are incompatible"));
>
> Shouldn't those two just be OPT_CMDMODE()? Then you get this
> incompatibility checking for free. See 485fd2c3dae (cat-file: make
> --batch-all-objects a CMDMODE, 2021-12-28).

TIL.  Thanks.

> > +     if (o.real || o.trivial) {
> > +             expected_remaining_argc = (o.real ? 2 : 3);
> > +             if (argc != expected_remaining_argc)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +     } else {
> > +             if (argc < 2 || argc > 3)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +             o.real = (argc == 2);
> > +     }
>
> And this can also be done like this, but I wonder if using
> PARSE_OPT_STOP_AT_NON_OPTION and then routing to a sub-function wouldn't
> be better, i.e. to treat these like sub-commands if they've got
> different arity etc.

Not sure what you mean; I already route to sub-functions.  But I
should definitely add PARSE_OPT_STOP_AT_NON_OPTION; it's unfortunate
that it's not the default.

Re: [PATCH 04/12] merge-tree: implement real merges

[Elijah Newren]

On Mon, Jan 24, 2022 at 1:55 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > +     /*
> > +      * TODO: Support subtree and other -X options?
> > +     if (use_strategies_nr == 1 &&
> > +         !strcmp(use_strategies[0]->name, "subtree"))
> > +             opt.subtree_shift = "";
> > +     for (x = 0; x < xopts_nr; x++)
> > +             if (parse_merge_opt(&opt, xopts[x]))
>
> Better omitted WIP code in a non-RFC series?

It's RFC: https://lore.kernel.org/git/pull.1122.git.1642888562.gitgitgadget@gmail.com/

But yeah, I should drop it.  Previous rounds of this RFC submission
got me feedback that the other commented-out code bit I used to have
was something folks wanted in the initial version of the series so I
uncommented and cleaned it up.  The fact that no one has commented on
this part suggests these options don't need to be supported from the
start.

> > +                     die(_("Unknown strategy option: -X%s"), xopts[x]);
>
> As a general issue with this series, die(), BUG() etc. messages should
> start with a non-capital letter.

Right, thanks for the reminder.  I'll go through and try to fix up.

> > +     printf("%s\n", oid_to_hex(&result.tree->object.oid));
>
> And for both this...
>
> > +             printf(_("Conflicts!\n"));
>
> ... and this we can just use puts(). For the former it's just less code,
> but for the latter translators also don't need to see the always-there
> \n in the translated message.

Makes sense.

> > +# This test is ort-specific
> > +test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {
>
> Is this ${} trickery really needed? We're not testing with "set -u". So just:
>
>         if test "$GIT_..." != "ort"
>         then
>                 ...
>         fi

Ah, that would be simpler; thanks.

> > +test_expect_success 'Barf on too many arguments' '
> > +     test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
> > +
> > +     grep "^usage: git merge-tree" expect
> > +'
>
> Nit: In most other tests these usage assertions are at the top of the
> test, and for those we also make do with just testing the 129 exit code,
> which is probably enough here too...

I see a fair number of counterexamples searching for 129 in the test
suite, and I've been bitten enough times seeing tests expect an error
but get a different kind of error than the commit message stated they
were expecting that I prefer the extra check beyond the error code.
Anyway, I'll leave this piece as-is.

Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts

[Elijah Newren]

On Mon, Jan 24, 2022 at 2:02 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Callers of `git merge-tree --write-tree` will often want to know which
> > files had conflicts.  While they could potentially attempt to parse the
> > CONFLICT notices printed, those messages are not meant to be machine
> > readable.  Provide a simpler mechanism of just printing the files (in
> > the same format as `git ls-files` with quoting, but restricted to
> > [...]
> > diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> > index 560640ad911..d8eeeb3f306 100644
> > --- a/builtin/merge-tree.c
> > +++ b/builtin/merge-tree.c
> > @@ -11,6 +11,9 @@
> >  #include "blob.h"
> >  #include "exec-cmd.h"
> >  #include "merge-blobs.h"
> > +#include "quote.h"
> > +
> > +static int line_termination = '\n';
>
> But unlike ls-files we don't do anything with line_termination as a !=
> '\n', maybe in a later commit?
>
> >  struct merge_list {
> >       struct merge_list *next;
> > @@ -395,7 +398,8 @@ struct merge_tree_options {
> >  };
> >
> >  static int real_merge(struct merge_tree_options *o,
> > -                   const char *branch1, const char *branch2)
> > +                   const char *branch1, const char *branch2,
> > +                   const char *prefix)
> >  {
> >       struct commit *parent1, *parent2;
> >       struct commit_list *common;
> > @@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
> >               o->show_messages = !result.clean;
> >
> >       printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +     if (!result.clean) {
> > +             struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> > +             const char *last = NULL;
> > +             int i;
> > +
> > +             merge_get_conflicted_files(&result, &conflicted_files);
> > +             for (i = 0; i < conflicted_files.nr; i++) {
> > +                     const char *name = conflicted_files.items[i].string;
> > +                     if (last && !strcmp(last, name))
> > +                             continue;
> > +                     write_name_quoted_relative(
> > +                             name, prefix, stdout, line_termination);
>
> But here it's never \0 or whatever.

Correct, I didn't add any option for changing it.  But why hardcode it
to "\n"?  Leaving it this way makes it easier to change later if folks
say they want NUL-terminated output.  Since the series is RFC and the
output already has changed drastically and appears to be the primary
discussion and disagreement point, I wanted to provide what seemed
like a reasonable suggestion and maintain flexiibility to address
feedback (though who knows -- I might need to just completely redo the
output again in ways much bigger than adding a -z option).

Re: [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren]

On Mon, Jan 24, 2022 at 2:11 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Much like `git merge` updates the index with information of the form
> >     (mode, oid, stage, name)
> > provide this output for conflicted files for merge-tree as well.
> > Provide an --exclude-oids-and-modes option for users to exclude the
> > mode, oid, and stage and only get the list of conflicted filenames.
> > [...]
> > +--exclude-oids-and-modes::
> > +     Instead of writing a list of (mode, oid, stage, path) tuples
> > +     to output for conflicted files, just provide a list of
> > +     filenames with conflicts.
> > +
> > [...]
> > -This is a sequence of lines containing a filename on each line, quoted
> > -as explained for the configuration variable `core.quotePath` (see
> > -linkgit:git-config[1]).
> > +This is a sequence of lines with the format
> > +
> > +     <mode> <object> <stage> <filename>
> > +
> > +The filename will be quoted as explained for the configuration
> > +variable `core.quotePath` (see linkgit:git-config[1]).  However, if
> > +the `--exclude-oids-and-modes` option is passed, the mode, object, and
> > +stage will be omitted.
> >
> >  Informational messages
> >  ~~~~~~~~~~~~~~~~~~~~~~
> >
> >  This always starts with a blank line to separate it from the previous
> > -section, and then has free-form messages about the merge, such as:
> > +sections, and then has free-form messages about the merge, such as:
> >
> >    * "Auto-merging <file>"
> >    * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
> > @@ -113,6 +123,14 @@ plumbing commands since the possibility of merge conflicts give it a
> >  much higher chance of the command not succeeding (and NEWTREE containing
> >  a bunch of stuff other than just a toplevel tree).
> >
> > +git-merge-tree was written to provide users with the same information
> > +that they'd have access to if using `git merge`:
> > +  * what would be written to the working tree (the <OID of toplevel tree>)
> > +  * the higher order stages that would be written to the index (the
> > +    <Conflicted file info>)
> > +  * any messages that would have been printed to stdout (the <Informational
> > +    messages>)
> > +
> >  GIT
> >  ---
> >  Part of the linkgit:git[1] suite
> > diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> > index d8eeeb3f306..7aa7f9fd54a 100644
> > --- a/builtin/merge-tree.c
> > +++ b/builtin/merge-tree.c
> > @@ -395,6 +395,7 @@ struct merge_tree_options {
> >       int real;
> >       int trivial;
> >       int show_messages;
> > +     int exclude_oids_and_modes;
> >  };
> >
> >  static int real_merge(struct merge_tree_options *o,
> > @@ -461,7 +462,11 @@ static int real_merge(struct merge_tree_options *o,
> >               merge_get_conflicted_files(&result, &conflicted_files);
> >               for (i = 0; i < conflicted_files.nr; i++) {
> >                       const char *name = conflicted_files.items[i].string;
> > -                     if (last && !strcmp(last, name))
> > +                     struct stage_info *c = conflicted_files.items[i].util;
> > +                     if (!o->exclude_oids_and_modes)
> > +                             printf("%06o %s %d\t",
> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
> > +                     else if (last && !strcmp(last, name))
> >                               continue;
> >                       write_name_quoted_relative(
> >                               name, prefix, stdout, line_termination);
> > @@ -495,6 +500,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >                        N_("do a trivial merge only")),
> >               OPT_BOOL(0, "messages", &o.show_messages,
> >                        N_("also show informational/conflict messages")),
> > +             OPT_BOOL_F(0, "exclude-oids-and-modes",
> > +                        &o.exclude_oids_and_modes,
> > +                        N_("list conflicted files without oids and modes"),
> > +                        PARSE_OPT_NONEG),
> >               OPT_END()
> >       };
>
> Perhaps this really is the last formatting information anyone will want,
> but with a default of "<mode> <object> <stage> <filename>" being made
> "<stage> <filename>" with --exclude-oids-and-modes perhaps we'll want
> --exclude-all-except-filename etc. later.

Um, that's actually what this option does.  Maybe my chosen name was bad.

--name-only like ls-files uses would have been nice, but it's
misleading since it only affects the <conflict info> section of the
output, not the printed tree or the informational messages.

> It seems a lot simpler for new code to just support a --conflict-format
> option, lifting some code from the in-flight
> https://lore.kernel.org/git/db058bf670c5668fc5b95baf83667cc282cb739b.1641978175.git.dyroneteng@gmail.com/
>
> I.e. that inner loop becomes a slightly more verbose strbuf_expand(),
> but it's all easy boilerplate code.
>
> Then we just feed "%(objectmode) %(objectname) %(objectstage)
> %(filename)" into it by default, and allow the user to customize it.

"simpler"?  More flexible certainly.

I'm not sure that the flexibility is warranted, in this case, though.
In ls-trees, where users don't need to process the output and can feed
it directly to something else, that flexibility makes sense.  But here
the output *needs* special post-processing anyway since it's mixing
three different types of output by default: top-level tree, conflicted
file info, and informational conflict messages.  (In my previous round
I tried to split these kinds of output to separate locations so they
could be parsed separately, but both Dscho and Christian didn't like
that).

So, the default I figured should be to just provide all the
information and just let others process it as wanted.  But Taylor had
said earlier that if there were conflicts the only information he
wanted was a list of conflicted files (no stages, oids, or modes), so
I figured an option making that a bit easier to extract was
worthwhile.  And that's what this patch is about.

Re: [PATCH 05/12] merge-ort: split out a separate display_update_messages() function

[Elijah Newren]

On Mon, Jan 24, 2022 at 2:00 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> > [...]
> > +     /* Hack to pre-allocate olist to the desired size */
> > +     ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> > +                olist.alloc);
>
> Perhaps just add a string_list_grow()? But I wonder if this is really
> needed v.s. just using the default growing pattern here.

A string_list_grow() would probably be helpful to add at some point;
then it could also be used in process_entries().

> > +
> > +     /* Put every entry from output into olist, then sort */
> > +     strmap_for_each_entry(&opti->output, &iter, e) {
> > +             string_list_append(&olist, e->key)->util = e->value;
> > +     }
> > +     string_list_sort(&olist);
> > +
> > +     /* Iterate over the items, printing them */
> > +     for (i = 0; i < olist.nr; ++i) {
> > +             struct strbuf *sb = olist.items[i].util;
> > +
> > +             printf("%s", sb->buf);
> > +     }
>
> Shorter/nicer:
>
>         for_each_string_list_item(item, &olist) {
>                 struct strbuf *sb = item->util;
>                 puts(sb->buf);
>         }

How did I not know about and not find for_each_string_list_item() when
I was writing this code a couple years ago?  (and still didn't learn
of it until now?)

Thanks for the pointer.  Won't change anything right now, though, since...

> > -     if (display_update_msgs) {
> > -             struct merge_options_internal *opti = result->priv;
> > -             struct hashmap_iter iter;
> > -             struct strmap_entry *e;
> > -             struct string_list olist = STRING_LIST_INIT_NODUP;
> > -             int i;
> > -
> > -             if (opt->record_conflict_msgs_as_headers)
> > -                     BUG("Either display conflict messages or record them as headers, not both");
> > -
> > -             trace2_region_enter("merge", "display messages", opt->repo);
> > -
> > -             /* Hack to pre-allocate olist to the desired size */
> > -             ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> > -                        olist.alloc);
> > -
> > -             /* Put every entry from output into olist, then sort */
> > -             strmap_for_each_entry(&opti->output, &iter, e) {
> > -                     string_list_append(&olist, e->key)->util = e->value;
> > -             }
> > -             string_list_sort(&olist);
> > -
> > -             /* Iterate over the items, printing them */
> > -             for (i = 0; i < olist.nr; ++i) {
> > -                     struct strbuf *sb = olist.items[i].util;
> > -
> > -                     printf("%s", sb->buf);
> > -             }
> > -             string_list_clear(&olist, 0);
>
> Ah, at this point I see you're just moving code around :) Sending this
> anyway in case it's useful :)

yep, so I won't change anything now, but yes th
for_each_string_list_item() tip is still useful as a heads up.
Thanks.

Re: [PATCH 04/12] merge-tree: implement real merges

[Johannes Schindelin]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> +The second form is deprecated; it is kept for backward compatibility
> +reasons but may be deleted in the future.  It will only do a trivial
> +merge.  It reads three tree-ish, and outputs trivial merge results and
> +conflicting stages to the standard output in a semi-diff format.
> +Since this was designed for higher level scripts to consume and merge
> +the results back into the index, it omits entries that match
> +<branch1>.  The result of this second form is is similar to what

There is a double "is" in this line. Taking a step back, I would suggest
to not only remove this paragraph, but to mark the `[--trivial-merge]`
option clearly as `(DEPRECATED)`.

> +three-way 'git read-tree -m' does, but instead of storing the results
> +in the index, the command outputs the entries to the standard output.
> +This form not only has limited applicability, the output format is
> +also difficult to work with, and it will generally be less performant
> +than the first form even on successful merges (especially if working
> +in large repositories).  The remainder of this manual will only
> +discuss the first form.

Thank you,
Dscho

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Christian Couder]

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
> suggestions and comments on v2):
>
>  * Significant changes to output format:
>    * Flags no longer take a filename for additional output; they write to
>      stdout instead.
>    * More information included by default when there are conflicts (no need
>      to request it with additional flags, instead flags can be used to
>      suppress it).
>    * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
>      information -- when there are conflicts. Add a flag to only list
>      conflicted files if that's preferred.

The above changes seem good to me.

>  * Much more thorough manual for git-merge-tree.txt
>  * Renamed option from --real to --write-tree
>  * Accept an optional --trivial-merge option to get old style merge-tree
>    behavior
>  * Allow both --write-tree and --trivial-merge to be omitted since we can
>    deduce which from number of arguments

I still think that it might be simpler and cleaner to leave 'git
merge-tree' alone for now, and just add a new command named for
example 'git write-merge-tree'. Later we can always add flags to 'git
merge-tree' or add 'git trivial-merge-tree' as an alias for 'git
merge-tree', and eventually slowly switch 'git merge-tree' to mean
only 'git write-merge-tree' if that's where we want to go.

>  * Document exit code when the merge cannot be run (so we can distinguish
>    other error cases from conflicts)
>  * testcase cleanups: test_tick, early skip of test when using recursive
>    backend, variable renames, etc.
>  * various minor code cleanups
>  * Add a new --allow-unrelated-histories option (with same meaning as the
>    one used in git merge)

The above changes seem good to me too.

> Stuff intentionally NOT included, but which others seemed to feel strongly
> about; they'd need to convince me more on these:
>
>  * Any form of diff output[1]

It's not a big issue for me to not include them right now as long as
it's possible to add cli options later that add them. The reason is
that I think in many cases when there are conflicts, the conflicts
will be small and the user will want to see them. So it would be
simpler to just have an option to show any conflict right away, rather
than have the user launch another command (a diff-tree against which
tree and with which options?).

Thanks for working on this anyway!

Re: [PATCH 04/12] merge-tree: implement real merges

[Christian Couder]

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Elijah Newren <newren@gmail.com>
>
> This adds the ability to perform real merges rather than just trivial
> merges (meaning handling three way content merges, recursive ancestor
> consolidation, renames, proper directory/file conflict handling, and so
> forth).  However, unlike `git merge`, the working tree and index are
> left alone and no branch is updated.
>
> The only output is:
>   - the toplevel resulting tree printed on stdout
>   - exit status of 0 (clean) or 1 (conflicts present)

The exit status can now actually be something other than 0 and 1
according to the doc and code below.

> +Performs a merge, but does not make any new commits and does not read
> +from or write to either the working tree or index.
> +
> +The first form will merge the two branches, doing a full recursive
> +merge with rename detection.

Maybe this could already tell that the first form will also write a
tree with the result of the merge (even in case of conflict) as this
could help understand the reason why the associated option is called
'--write-tree'. It could also help to say that we call such a merge a
'real' merge.

> The rest of this manual (other than the
> +next paragraph) describes the first form in more detail -- including
> +options, output format, exit status, and usage notes.

>  static int real_merge(struct merge_tree_options *o,
>                       const char *branch1, const char *branch2)
>  {
> -       die(_("real merges are not yet implemented"));
> +       struct commit *parent1, *parent2;
> +       struct commit_list *common;
> +       struct commit_list *merge_bases = NULL;
> +       struct commit_list *j;
> +       struct merge_options opt;
> +       struct merge_result result = { 0 };
> +
> +       parent1 = get_merge_parent(branch1);
> +       if (!parent1)
> +               help_unknown_ref(branch1, "merge",
> +                                _("not something we can merge"));

The second argument is supposed to be the command (it's called "cmd"),
so maybe "merge-tree" instead of "merge".

> +       parent2 = get_merge_parent(branch2);
> +       if (!parent2)
> +               help_unknown_ref(branch2, "merge",
> +                                _("not something we can merge"));

idem

> +       opt.show_rename_progress = 0;
> +
> +       opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
> +       opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */

I think just:

       opt.branch1 = branch1
       opt.branch2 = branch2

might be better for users as it should show the name as it was passed
to the command.

> +       merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> +       printf("%s\n", oid_to_hex(&result.tree->object.oid));

I wonder if we can actually always output a valid tree when
result.clean < 0. In case we might not, the printing should go a few
lines below.

> +       if (result.clean < 0)
> +               die(_("failure to merge"));
> +       else if (!result.clean)

The "else" is not necessary above.

> +               printf(_("Conflicts!\n"));
> +       merge_finalize(&opt, &result);
> +       return !result.clean; /* result.clean < 0 handled above */
>  }

> diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> new file mode 100755
> index 00000000000..e03688515c5
> --- /dev/null
> +++ b/t/t4301-merge-tree-real.sh

I wonder if it would be better named 't4301-merge-tree-write-tree.sh'...

> @@ -0,0 +1,87 @@
> +#!/bin/sh
> +
> +test_description='git merge-tree --write-tree'

... especially given this description.

Re: [PATCH 07/12] merge-tree: support including merge messages in output

[Christian Couder]

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

>  EXIT STATUS
>  -----------
> @@ -72,7 +102,8 @@ be used as a part of a series of steps such as
>
>  However, it does not quite fit into the same category of low-level
>  plumbing commands since the possibility of merge conflicts give it a
> -much higher chance of the command not succeeding.
> +much higher chance of the command not succeeding (and NEWTREE containing
> +a bunch of stuff other than just a toplevel tree).

Is this hunk really related to this commit or should it go into a
previous commit?

> @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
>                 commit_list_insert(j->item, &merge_bases);
>
>         merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> -       printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +
>         if (result.clean < 0)
>                 die(_("failure to merge"));

So this addresses the comment I made in a previous commit related to
the fact that if result.clean < 0 we might not have a valid tree that
we can print. I think though that it would be better if that was
addressed in a previous commit.

> -       else if (!result.clean)
> -               printf(_("Conflicts!\n"));

Ok, so we don't print "Conflicts!\n" now, which makes me wonder if we
should have printed it in the first place in previous commits.

>         if (o.real && o.trivial)
>                 die(_("--write-tree and --trivial-merge are incompatible"));
> +       if (!o.real && original_argc < argc)
> +               die(_("--write-tree must be specified if any other options are"));

Is this necessary? It looks to me like another thing that would be
simplified if we were just adding a new command...

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Christian Couder]

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> After a merge, this function allows the user to extract the same
> information that would be printed by `ls-files -u` -- conflicted

This made me wonder if "-- conflicted" should be part of the `ls-files
-u` command. Maybe "by `ls-files -u`, which means" would make things a
bit clearer.

> files with their mode, oid, and stage.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Christian Couder]

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> +void merge_get_conflicted_files(struct merge_result *result,
> +                               struct string_list *conflicted_files)
> +{
> +       struct hashmap_iter iter;
> +       struct strmap_entry *e;
> +       struct merge_options_internal *opti = result->priv;
> +
> +       strmap_for_each_entry(&opti->conflicted, &iter, e) {
> +               const char *path = e->key;
> +               struct conflict_info *ci = e->value;
> +               int i;
> +
> +               VERIFY_CI(ci);
> +
> +               for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> +                       struct stage_info *si;
> +
> +                       if (!(ci->filemask & (1ul << i)))
> +                               continue;
> +
> +                       si = xmalloc(sizeof(*si));

It's probably a premature optimization, so feel free to ignore, but as
MERGE_BASE and MERGE_SIDE2 are constants, and ci->filemask is constant
inside the 'for' loop, we could compute before the 'for' loop how many
'struct stage_info' we will need and allocate them all at once before
the 'for' loop.

> +                       si->stage = i+1;
> +                       si->mode = ci->stages[i].mode;
> +                       oidcpy(&si->oid, &ci->stages[i].oid);
> +                       string_list_append(conflicted_files, path)->util = si;
> +               }
> +       }
> +       /* string_list_sort() uses a stable sort, so we're good */
> +       string_list_sort(conflicted_files);
> +}

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Johannes Schindelin]

Hi Christian,

On Wed, 26 Jan 2022, Christian Couder wrote:

> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> >  * Accept an optional --trivial-merge option to get old style merge-tree
> >    behavior
> >  * Allow both --write-tree and --trivial-merge to be omitted since we can
> >    deduce which from number of arguments
>
> I still think that it might be simpler and cleaner to leave 'git
> merge-tree' alone for now, and just add a new command named for
> example 'git write-merge-tree'.

That would assume that the original `git merge-tree` implementation was
useful. That notion has been thoroughly refuted in the meantime, though.

I am really opposed to introducing a new command here. Elijah took the
best approach we can take here: save the `merge-tree` command by teaching
it to do something useful.

> Later we can always add flags to 'git merge-tree' or add 'git
> trivial-merge-tree' as an alias for 'git merge-tree', and eventually
> slowly switch 'git merge-tree' to mean only 'git write-merge-tree' if
> that's where we want to go.

I suggested before, and seem to need to repeat again, that we need to let
ourselves be guided less by hypothetical scenarios, and more by actual,
concrete use cases where the revamped `merge-tree` command is useful.

And since I already provided some feedback based on my work from working
on a server-side backend, I am fairly certain that we already have a
pretty good idea where we want to go.

> > Stuff intentionally NOT included, but which others seemed to feel strongly
> > about; they'd need to convince me more on these:
> >
> >  * Any form of diff output[1]
>
> It's not a big issue for me to not include them right now as long as
> it's possible to add cli options later that add them.

But why? That _so_ smells like a hypothetical scenario.

We do not need the diffs. It is highly unlikely that the server-side wants
to have diffs, and if a user does want the diffs, it is very, very easy to
generate them by chaining low-level commands.

So there is absolutely no need for `git merge-tree` to produce diffs.

> The reason is that I think in many cases when there are conflicts, the
> conflicts will be small and the user will want to see them. So it would
> be simpler to just have an option to show any conflict right away,
> rather than have the user launch another command (a diff-tree against
> which tree and with which options?).

That assumes that server-side merge UIs will present merge conflicts in
the form of diffs containing merge conflict markers. Which I don't think
will happen, like, ever.

In short: I completely disagree that we should introduce a new command,
and I also completely disagree that the `merge-tree` command should output
any diffs.

I do agree that we need to be mindful of what we actually need, and in
that regard, I reiterate that we need to let concrete use cases guide us.
As part of GitLab, you might be in an excellent position to look at
GitLab's concrete server-side needs when it comes to use `git merge-tree`
to perform merges.

Ciao,
Dscho

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Christian Couder]

Hi Dscho,

On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Christian,
>
> On Wed, 26 Jan 2022, Christian Couder wrote:
>
> > On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >
> > >  * Accept an optional --trivial-merge option to get old style merge-tree
> > >    behavior
> > >  * Allow both --write-tree and --trivial-merge to be omitted since we can
> > >    deduce which from number of arguments
> >
> > I still think that it might be simpler and cleaner to leave 'git
> > merge-tree' alone for now, and just add a new command named for
> > example 'git write-merge-tree'.
>
> That would assume that the original `git merge-tree` implementation was
> useful. That notion has been thoroughly refuted in the meantime, though.
>
> I am really opposed to introducing a new command here. Elijah took the
> best approach we can take here: save the `merge-tree` command by teaching
> it to do something useful.

I think it's a question of point of view. If a command is completely
useless, then most of the time it needs to die, not be "saved". We
would need good statistics, but I doubt we have "saved" many useless
commands before, compared to commands we have just killed.

> > Later we can always add flags to 'git merge-tree' or add 'git
> > trivial-merge-tree' as an alias for 'git merge-tree', and eventually
> > slowly switch 'git merge-tree' to mean only 'git write-merge-tree' if
> > that's where we want to go.
>
> I suggested before, and seem to need to repeat again, that we need to let
> ourselves be guided less by hypothetical scenarios, and more by actual,
> concrete use cases where the revamped `merge-tree` command is useful.

Ok, see below.

> And since I already provided some feedback based on my work from working
> on a server-side backend, I am fairly certain that we already have a
> pretty good idea where we want to go.
>
> > > Stuff intentionally NOT included, but which others seemed to feel strongly
> > > about; they'd need to convince me more on these:
> > >
> > >  * Any form of diff output[1]
> >
> > It's not a big issue for me to not include them right now as long as
> > it's possible to add cli options later that add them.
>
> But why? That _so_ smells like a hypothetical scenario.
>
> We do not need the diffs. It is highly unlikely that the server-side wants
> to have diffs, and if a user does want the diffs, it is very, very easy to
> generate them by chaining low-level commands.
>
> So there is absolutely no need for `git merge-tree` to produce diffs.
>
> > The reason is that I think in many cases when there are conflicts, the
> > conflicts will be small and the user will want to see them. So it would
> > be simpler to just have an option to show any conflict right away,
> > rather than have the user launch another command (a diff-tree against
> > which tree and with which options?).
>
> That assumes that server-side merge UIs will present merge conflicts in
> the form of diffs containing merge conflict markers. Which I don't think
> will happen, like, ever.

Please take a look at:

https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor

As you can see in the image there are conflict markers in the file
displayed by the server UI.

> In short: I completely disagree that we should introduce a new command,
> and I also completely disagree that the `merge-tree` command should output
> any diffs.
>
> I do agree that we need to be mindful of what we actually need, and in
> that regard, I reiterate that we need to let concrete use cases guide us.
> As part of GitLab, you might be in an excellent position to look at
> GitLab's concrete server-side needs when it comes to use `git merge-tree`
> to perform merges.

I hope I provided a concrete use case with the link above.

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Johannes Schindelin]

Hi Christian,

On Wed, 26 Jan 2022, Christian Couder wrote:

> On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Wed, 26 Jan 2022, Christian Couder wrote:
> >
> > > The reason is that I think in many cases when there are conflicts,
> > > the conflicts will be small and the user will want to see them. So
> > > it would be simpler to just have an option to show any conflict
> > > right away, rather than have the user launch another command (a
> > > diff-tree against which tree and with which options?).
> >
> > That assumes that server-side merge UIs will present merge conflicts in
> > the form of diffs containing merge conflict markers. Which I don't think
> > will happen, like, ever.
>
> Please take a look at:
>
> https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor
>
> As you can see in the image there are conflict markers in the file
> displayed by the server UI.

Please note the difference between what I wrote above (present merge
conflicts in the form of diffs containing merge conflict markers) and what
is shown in the document you linked to (present a file annotated with
merge conflict markers).

There is no diff in that page.

What's more: there are not only conflict markers in the editor, there is
clearly a visual marker next to the line number that indicates that the
editor has a fundamental understanding where the conflict markers are.
Which means that the conflict markers must have been generated
independently of Git rather than parsed in some random diff that was
produced by Git.

In other words: you are making my case for me that `git merge-tree` should
not generate diff output because it would not even be used.

> > In short: I completely disagree that we should introduce a new command,
> > and I also completely disagree that the `merge-tree` command should output
> > any diffs.
> >
> > I do agree that we need to be mindful of what we actually need, and in
> > that regard, I reiterate that we need to let concrete use cases guide us.
> > As part of GitLab, you might be in an excellent position to look at
> > GitLab's concrete server-side needs when it comes to use `git merge-tree`
> > to perform merges.
>
> I hope I provided a concrete use case with the link above.

Sorry, I apparently was a bit unclear.

In the context of discussing `git merge-tree`, a low-level Git command,
when I talk about a user, I do not mean a human being, but a program that
calls that command and parses its output.

Corollary: by "use case" I refer to a concrete implementation of a
server-side merge operation, I refer to backend code that currently calls
into libgit2 to perform the merge, and would benefit from calling `git
merge-tree` instead. Such a use case informs us about the type and amount
of information that is required of the code that is currently being
discussed in this mail thread. And I highly doubt that you will find such
a use case that wants libgit2 (and later, `git merge-tree`) to generate
diffs. Because _diffs_ are certainly _not_ what is consumed by the inline
editor you referenced.

Of course, I am still left guessing what the server-side needs concretely,
because that is not at all obvious from the user-facing web site to which
you sent me. What is needed is a good, hard look at the actual _code_, the
code that calls into libgit2 to perform a merge, and that could instead
spawn a `git merge-tree` process to accomplish the same thing.

We need to get away from hypothetical scenarios. They're not helping.

Ciao,
Johannes

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Christian Couder]

Hi Dscho,

On Fri, Jan 28, 2022 at 1:58 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Christian,
>
> On Wed, 26 Jan 2022, Christian Couder wrote:
>
> > On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > On Wed, 26 Jan 2022, Christian Couder wrote:
> > >
> > > > The reason is that I think in many cases when there are conflicts,
> > > > the conflicts will be small and the user will want to see them. So
> > > > it would be simpler to just have an option to show any conflict
> > > > right away, rather than have the user launch another command (a
> > > > diff-tree against which tree and with which options?).
> > >
> > > That assumes that server-side merge UIs will present merge conflicts in
> > > the form of diffs containing merge conflict markers. Which I don't think
> > > will happen, like, ever.
> >
> > Please take a look at:
> >
> > https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor
> >
> > As you can see in the image there are conflict markers in the file
> > displayed by the server UI.
>
> Please note the difference between what I wrote above (present merge
> conflicts in the form of diffs containing merge conflict markers) and what
> is shown in the document you linked to (present a file annotated with
> merge conflict markers).
>
> There is no diff in that page.

The server UI could just get a diff with the conflicts inside instead
of the full files with conflict inside, as the diff would be smaller
to parse than the full files. So even if it's not shown, the diff
could still be useful.

Also just above the section of the link I sent, there is this section

https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-interactive-mode

where one can see diff markers in the image. There are no conflict
markers in those images, but it's possible that a future UI could
combine both a diff and conflict markers.

Also please note that I don't absolutely require diffs. At the
beginning of the paragraph from my original email that you quoted
above, there was:

"It's not a big issue for me to not include them right now as long as
it's possible to add cli options later that add them."

So I was just saying that the format and code should be flexible
enough to be able to easily accommodate sending further data like
diffs with additional options. I think it's a very reasonable request.
So please don't make it a huge issue. You can always NACK a patch
adding such an option later.

> What's more: there are not only conflict markers in the editor,

You don't see the ">>>>>>>"?

> there is
> clearly a visual marker next to the line number that indicates that the
> editor has a fundamental understanding where the conflict markers are.

Yeah, so this shows that those markers can be important for the editor.

> Which means that the conflict markers must have been generated
> independently of Git rather than parsed in some random diff that was
> produced by Git.

Why couldn't they be generated by Git and then just parsed from a diff
in the future, even if that was not the case here?

> In other words: you are making my case for me that `git merge-tree` should
> not generate diff output because it would not even be used.

The other link above in this email actually shows that diffs are used
right now to resolve conflicts.

> > > In short: I completely disagree that we should introduce a new command,
> > > and I also completely disagree that the `merge-tree` command should output
> > > any diffs.
> > >
> > > I do agree that we need to be mindful of what we actually need, and in
> > > that regard, I reiterate that we need to let concrete use cases guide us.
> > > As part of GitLab, you might be in an excellent position to look at
> > > GitLab's concrete server-side needs when it comes to use `git merge-tree`
> > > to perform merges.
> >
> > I hope I provided a concrete use case with the link above.
>
> Sorry, I apparently was a bit unclear.
>
> In the context of discussing `git merge-tree`, a low-level Git command,
> when I talk about a user, I do not mean a human being, but a program that
> calls that command and parses its output.

So it could very well parse diffs containing conflict markers and show
those conflict markers.

[...]

> Of course, I am still left guessing what the server-side needs concretely,
> because that is not at all obvious from the user-facing web site to which
> you sent me. What is needed is a good, hard look at the actual _code_, the
> code that calls into libgit2 to perform a merge, and that could instead
> spawn a `git merge-tree` process to accomplish the same thing.
>
> We need to get away from hypothetical scenarios. They're not helping.

I am not even asking for a feature, just to make it possible to extend
the output of a brand new command in an RFC with some possibly useful
things, and you are making such requests...

Please relax a bit on this.

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Johannes Schindelin]

Hi Christian,

On Fri, 28 Jan 2022, Christian Couder wrote:

> On Fri, Jan 28, 2022 at 1:58 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Wed, 26 Jan 2022, Christian Couder wrote:
> >
> > > On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
> > > <Johannes.Schindelin@gmx.de> wrote:
> > > >
> > > > On Wed, 26 Jan 2022, Christian Couder wrote:
> > > >
> > > > > The reason is that I think in many cases when there are conflicts,
> > > > > the conflicts will be small and the user will want to see them. So
> > > > > it would be simpler to just have an option to show any conflict
> > > > > right away, rather than have the user launch another command (a
> > > > > diff-tree against which tree and with which options?).
> > > >
> > > > That assumes that server-side merge UIs will present merge conflicts in
> > > > the form of diffs containing merge conflict markers. Which I don't think
> > > > will happen, like, ever.
> > >
> > > Please take a look at:
> > >
> > > https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor
> > >
> > > As you can see in the image there are conflict markers in the file
> > > displayed by the server UI.
> >
> > Please note the difference between what I wrote above (present merge
> > conflicts in the form of diffs containing merge conflict markers) and what
> > is shown in the document you linked to (present a file annotated with
> > merge conflict markers).
> >
> > There is no diff in that page.
>
> The server UI could just get a diff with the conflicts inside instead
> of the full files with conflict inside, as the diff would be smaller
> to parse than the full files. So even if it's not shown, the diff
> could still be useful.

You really need to get away from talking about this in hypothetical terms.

> Also just above the section of the link I sent, there is this section
>
> https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-interactive-mode
>
> where one can see diff markers in the image.

That's a side-by-side diff. Git cannot even produce those.

> > What's more: there are not only conflict markers in the editor,
>
> You don't see the ">>>>>>>"?

Yes, I do. And not only that. I also see that the editor knows very
specifically where the conflict happens.

And since any file can contain `>>>>>>>` _without_ it being a conflict
marker, the editor most likely does not parse the output of Git that
contains a conflict marker. At least I hope it does not because it would
then very easily be confused by strings that look like conflict markers,
but aren't.

Think about our very own test suite, and why we specifically set
`conflict-marker-size=32` for those files. Same reason why the server
backend cannot simply ingest files with conflict markers and then hope to
figure out which `>>>>>>>` are real conflict markers and which are not.

> > there is clearly a visual marker next to the line number that
> > indicates that the editor has a fundamental understanding where the
> > conflict markers are.
>
> Yeah, so this shows that those markers can be important for the editor.

Of course they are important! That's my point!

> > In other words: you are making my case for me that `git merge-tree` should
> > not generate diff output because it would not even be used.
>
> The other link above in this email actually shows that diffs are used
> right now to resolve conflicts.

It shows that Git was not used to generate the diff, is what it shows.

I see that you are still trying to guess what the server-side needs
actually are. It really is time to stop guessing. So I will keep
challenging you to actually look at the GitLab code, to take a stab at
teaching it to use `git merge-tree` to perform merges. And then to come
back with what you learned. I guarantee you that that will be multiple
times more useful than talking about it in hypotheticals.

And you are in such an almost unique position to contribute to this patch
series, to provide that very valuable feedback how `git merge-tree` could
be improved to support actual, real-life server-side code that is
currently in use! So why not make the most out of it?

Ciao,
Johannes

Re: [PATCH 05/12] merge-ort: split out a separate display_update_messages() function

[Johannes Schindelin]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> No functional changes included in this patch; it's just a preparatory
> step to allow the printed messages to be handled differently by other
> callers, such as in `git merge-tree --write-tree`.

Looks good. FWIW I cheated and looked at the output of

	git show pr-1122/newren/in-core-merge-tree-v1~7 \
		--color-moved \
		--color-moved-ws=allow-indentation-change

(after fetching the tag from https://github.com/gitgitgadget/git) instead
of this patch (which is a lot easier to digest for me, because of the
color-coding, and since GitGitGadget sent the patch, I know that it is
identical to the commit).

Ciao,
Dscho

Re: [PATCH 06/12] merge-ort: allow update messages to be written to different file stream

[Johannes Schindelin]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> diff --git a/merge-ort.c b/merge-ort.c
> index f9e35b0f96b..b78dde55ad9 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
>  }
>
>  void merge_display_update_messages(struct merge_options *opt,
> -				   struct merge_result *result)
> +				   struct merge_result *result,
> +				   FILE *stream)
>  {
>  	struct merge_options_internal *opti = result->priv;
>  	struct hashmap_iter iter;
> @@ -4263,7 +4264,7 @@ void merge_display_update_messages(struct merge_options *opt,
>  	for (i = 0; i < olist.nr; ++i) {
>  		struct strbuf *sb = olist.items[i].util;
>
> -		printf("%s", sb->buf);
> +		fprintf(stream, "%s", sb->buf);

Maybe `strbuf_write(sb, stream);` instead? Whenever I see a `"%s"`, I feel
like it's unnecessary churn...

>  	}
>  	string_list_clear(&olist, 0);
>

Missing from this hunk:

        /* Also include needed rename limit adjustment now */
        diff_warn_rename_limit("merge.renamelimit",
                               opti->renames.needed_limit, 0);

This function explicitly writes to `stdout`, and will need to be adjusted,
I think, before we can include an adjustment to this call in this patch.

Unless we override `warn_routine()` (which is used inside that function),
that is. Which is hacky, and we would not have addressed the
`fflush(stdout)` in `diff_warn_rename_limit()`. So I would much prefer
something like this:

-- snip --
diff --git a/diff.c b/diff.c
index 861282db1c3..87966602cbd 100644
--- a/diff.c
+++ b/diff.c
@@ -6312,17 +6312,25 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");

-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out)
 {
-	fflush(stdout);
+	const char *fmt = NULL;
+
 	if (degraded_cc)
-		warning(_(degrade_cc_to_c_warning));
+		fmt = _(degrade_cc_to_c_warning);
 	else if (needed)
-		warning(_(rename_limit_warning));
+		fmt = _(rename_limit_warning);
 	else
 		return;
 	if (0 < needed)
-		warning(_(rename_limit_advice), varname, needed);
+		fmt = _(rename_limit_advice);
+
+	fflush(out);
+	if (out == stdout)
+		warning(fmt, varname, needed);
+	else
+		fprintf(out, fmt, varname, needed);
 }

 static void diff_flush_patch_all_file_pairs(struct diff_options *o)
@@ -6754,7 +6762,7 @@ int diff_result_code(struct diff_options *opt, int status)

 	diff_warn_rename_limit("diff.renameLimit",
 			       opt->needed_rename_limit,
-			       opt->degraded_cc_to_c);
+			       opt->degraded_cc_to_c, stdout);
 	if (!opt->flags.exit_with_status &&
 	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
 		return status;
diff --git a/diff.h b/diff.h
index 8ba85c5e605..be4ee68c0a2 100644
--- a/diff.h
+++ b/diff.h
@@ -596,7 +596,8 @@ void diffcore_fix_diff_index(void);
 int diff_queue_is_empty(void);
 void diff_flush(struct diff_options*);
 void diff_free(struct diff_options*);
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out);

 /* diff-raw status letters */
 #define DIFF_STATUS_ADDED		'A'
diff --git a/merge-ort.c b/merge-ort.c
index 0342f104836..e6b5a0e7c64 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4264,7 +4264,7 @@ void merge_switch_to_result(struct merge_options *opt,

 		/* Also include needed rename limit adjustment now */
 		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
+				       opti->renames.needed_limit, 0, stdout);

 		trace2_region_leave("merge", "display messages", opt->repo);
 	}
diff --git a/merge-recursive.c b/merge-recursive.c
index d9457797dbb..10b2948678c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3731,7 +3731,8 @@ static void merge_finalize(struct merge_options *opt)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->priv->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0,
+				       stdout);
 	FREE_AND_NULL(opt->priv);
 }

-- snap --

The rest of the patch looks good to me.

Thanks,
Dscho

> @@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
>  	}
>
>  	if (display_update_msgs)
> -		merge_display_update_messages(opt, result);
> +		merge_display_update_messages(opt, result, stdout);
>
>  	merge_finalize(opt, result);
>  }
> diff --git a/merge-ort.h b/merge-ort.h
> index e5aec45b18f..d643b47cb7c 100644
> --- a/merge-ort.h
> +++ b/merge-ort.h
> @@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
>   * so only call this when bypassing merge_switch_to_result().
>   */
>  void merge_display_update_messages(struct merge_options *opt,
> -				   struct merge_result *result);
> +				   struct merge_result *result,
> +				   FILE *stream);
>
>  /* Do needed cleanup when not calling merge_switch_to_result() */
>  void merge_finalize(struct merge_options *opt,
> --
> gitgitgadget
>
>

Re: [PATCH 07/12] merge-tree: support including merge messages in output

[Johannes Schindelin]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> When running `git merge-tree --write-tree`, we previously would only
> return an exit status reflecting the cleanness of a merge, and print out
> the toplevel tree of the resulting merge.  Merges also have
> informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
> ...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
> conflicts occur (such as file/directory, modify/delete, add/add with
> differing modes, rename/rename (1to2), etc.), these informational
> messages are often the only notification since these conflicts are not
> representable in the contents of the file.
>
> Add a --[no-]messages option so that callers can request these messages
> be included at the end of the output.  Include such messages by default
> when there are conflicts, and omit them by default when the merge is
> clean.

Makes sense.

> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 0c19639594d..560640ad911 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
>  		commit_list_insert(j->item, &merge_bases);
>
>  	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> -	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +
>  	if (result.clean < 0)
>  		die(_("failure to merge"));
> -	else if (!result.clean)
> -		printf(_("Conflicts!\n"));
> +
> +	if (o->show_messages == -1)
> +		o->show_messages = !result.clean;
> +
> +	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +	if (o->show_messages) {
> +		printf("\n");
> +		merge_display_update_messages(&opt, &result, stdout);
> +	}

Excellent.

>  	merge_finalize(&opt, &result);
>  	return !result.clean; /* result.clean < 0 handled above */
>  }
>
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	struct merge_tree_options o = { 0 };
> +	struct merge_tree_options o = { .show_messages = -1 };
>  	int expected_remaining_argc;
> +	int original_argc;
>
>  	const char * const merge_tree_usage[] = {
> -		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
>  		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>  		NULL
>  	};
> @@ -464,6 +473,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			 N_("do a real merge instead of a trivial merge")),
>  		OPT_BOOL(0, "trivial-merge", &o.trivial,
>  			 N_("do a trivial merge only")),
> +		OPT_BOOL(0, "messages", &o.show_messages,
> +			 N_("also show informational/conflict messages")),
>  		OPT_END()
>  	};
>
> @@ -472,10 +483,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  		usage_with_options(merge_tree_usage, mt_options);
>
>  	/* Parse arguments */
> +	original_argc = argc;
>  	argc = parse_options(argc, argv, prefix, mt_options,
>  			     merge_tree_usage, 0);
>  	if (o.real && o.trivial)
>  		die(_("--write-tree and --trivial-merge are incompatible"));
> +	if (!o.real && original_argc < argc)
> +		die(_("--write-tree must be specified if any other options are"));

Hmm. Well. Hmm.


I'd rather keep `--write-tree` neat and optional. What's wrong with
allowing

	git merge-tree --no-messages HEAD MERGE_HEAD

?

To be clear, I think we need this instead:

	if (o.trivial && o.show_messages >= 0)
		die(_("--trivial-merge is incompatible with additional options"));

I like the rest of the patch very much!

Thank you,
Dscho

>  	if (o.real || o.trivial) {
>  		expected_remaining_argc = (o.real ? 2 : 3);
>  		if (argc != expected_remaining_argc)
> diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> index e03688515c5..c34f8e6c1ed 100755
> --- a/t/t4301-merge-tree-real.sh
> +++ b/t/t4301-merge-tree-real.sh
> @@ -84,4 +84,25 @@ test_expect_success 'Barf on too many arguments' '
>  	grep "^usage: git merge-tree" expect
>  '
>
> +test_expect_success 'test conflict notices and such' '
> +	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
> +	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
> +
> +	# Expected results:
> +	#   "greeting" should merge with conflicts
> +	#   "numbers" should merge cleanly
> +	#   "whatever" has *both* a modify/delete and a file/directory conflict
> +	cat <<-EOF >expect &&
> +	HASH
> +
> +	Auto-merging greeting
> +	CONFLICT (content): Merge conflict in greeting
> +	Auto-merging numbers
> +	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
> +	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
> +	EOF
> +
> +	test_cmp expect actual
> +'
> +
>  test_done
> --
> gitgitgadget
>
>

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> After a merge, this function allows the user to extract the same
> information that would be printed by `ls-files -u` -- conflicted
> files with their mode, oid, and stage.

Hmm. Okay.

I am not really a fan of that output where we use a variable
number of lines per file. As in: in the regular case, where a file was
modified in divergent ways, we get three lines. But if it was deleted in
one branch, or if it was created in both branches, we have only two lines.

Frankly, I'd much rather have 3 lines for each and every conflict.

Granted, we currently only have three stages, and we can pretty much
guarantee that at least two of these stages are non-empty (otherwise where
would be the conflict?).

Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
missing from the second conflict, in the output we would see stages 1, 2,
2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
different conflicts.

But still. It makes me uneasy to have that much variability in
machine-consumable output.

And who knows, maybe if we're ever implementing more sophisticated merge
strategies for octopus merges, we might end up with more stages...

In other words...

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-ort.c | 31 +++++++++++++++++++++++++++++++
>  merge-ort.h | 21 +++++++++++++++++++++
>  2 files changed, 52 insertions(+)
>
> diff --git a/merge-ort.c b/merge-ort.c
> index b78dde55ad9..5e7cea6cc8f 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
>  	trace2_region_leave("merge", "display messages", opt->repo);
>  }
>
> +void merge_get_conflicted_files(struct merge_result *result,
> +				struct string_list *conflicted_files)
> +{
> +	struct hashmap_iter iter;
> +	struct strmap_entry *e;
> +	struct merge_options_internal *opti = result->priv;
> +
> +	strmap_for_each_entry(&opti->conflicted, &iter, e) {
> +		const char *path = e->key;
> +		struct conflict_info *ci = e->value;
> +		int i;
> +
> +		VERIFY_CI(ci);
> +
> +		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> +			struct stage_info *si;
> +
> +			if (!(ci->filemask & (1ul << i)))
> +				continue;
> +
> +			si = xmalloc(sizeof(*si));
> +			si->stage = i+1;
> +			si->mode = ci->stages[i].mode;
> +			oidcpy(&si->oid, &ci->stages[i].oid);
> +			string_list_append(conflicted_files, path)->util = si;
> +		}
> +	}
> +	/* string_list_sort() uses a stable sort, so we're good */
> +	string_list_sort(conflicted_files);
> +}
> +
>  void merge_switch_to_result(struct merge_options *opt,
>  			    struct tree *head,
>  			    struct merge_result *result,
> diff --git a/merge-ort.h b/merge-ort.h
> index d643b47cb7c..e635a294ea8 100644
> --- a/merge-ort.h
> +++ b/merge-ort.h
> @@ -2,6 +2,7 @@
>  #define MERGE_ORT_H
>
>  #include "merge-recursive.h"
> +#include "hash.h"
>
>  struct commit;
>  struct tree;
> @@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
>  				   struct merge_result *result,
>  				   FILE *stream);
>
> +struct stage_info {
> +	struct object_id oid;
> +	int mode;
> +	int stage;
> +};

... I'd rather not tack this onto a `string_list` but instead have
something like:

	struct conflict_info {
		struct {
			struct object_id oid;
			int mode;
			const char *path;
		} stages[3]
	};

where `path` can be `NULL` to indicate that this stage is missing.

Apart from this concern about the overall design, the patch looks good, of
course.

Ciao,
Dscho

> +
> +/*
> + * Provide a list of path -> {struct stage_info*} mappings for
> + * all conflicted files.  Note that each path could appear up to three
> + * times in the list, corresponding to 3 different stage entries.  In short,
> + * this basically provides the info that would be printed by `ls-files -u`.
> + *
> + * result should have been populated by a call to
> + * one of the merge_incore_[non]recursive() functions.
> + *
> + * conflicted_files should be empty before calling this function.
> + */
> +void merge_get_conflicted_files(struct merge_result *result,
> +				struct string_list *conflicted_files);
> +
>  /* Do needed cleanup when not calling merge_switch_to_result() */
>  void merge_finalize(struct merge_options *opt,
>  		    struct merge_result *result);
> --
> gitgitgadget
>
>

Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts

[Johannes Schindelin]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Callers of `git merge-tree --write-tree` will often want to know which
> files had conflicts.  While they could potentially attempt to parse the
> CONFLICT notices printed, those messages are not meant to be machine
> readable.  Provide a simpler mechanism of just printing the files (in
> the same format as `git ls-files` with quoting, but restricted to
> unmerged files) in the output before the free-form messages.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  Documentation/git-merge-tree.txt |  8 ++++++++
>  builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
>  t/t4301-merge-tree-real.sh       | 11 +++++++++++
>  3 files changed, 41 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index fd7a867de60..041a4ac2785 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -58,6 +58,7 @@ simply one line:
>  Whereas for a conflicted merge, the output is by default of the form:
>
>  	<OID of toplevel tree>
> +	<Conflicted file list>
>  	<Informational messages>

To distinguish between the list of conflicted files and the informational
messages, I think it would be good to insert an empty line, as a
separator, like. And...

>
>  These are discussed individually below.
> @@ -69,6 +70,13 @@ This is a tree object that represents what would be checked out in the
>  working tree at the end of `git merge`.  If there were conflicts, then
>  files within this tree may have embedded conflict markers.
>
> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).
> +
>  Informational messages
>  ~~~~~~~~~~~~~~~~~~~~~~
>
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 560640ad911..d8eeeb3f306 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -11,6 +11,9 @@
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
> +#include "quote.h"
> +
> +static int line_termination = '\n';
>
>  struct merge_list {
>  	struct merge_list *next;
> @@ -395,7 +398,8 @@ struct merge_tree_options {
>  };
>
>  static int real_merge(struct merge_tree_options *o,
> -		      const char *branch1, const char *branch2)
> +		      const char *branch1, const char *branch2,
> +		      const char *prefix)
>  {
>  	struct commit *parent1, *parent2;
>  	struct commit_list *common;
> @@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
>  		o->show_messages = !result.clean;
>
>  	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +	if (!result.clean) {
> +		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> +		const char *last = NULL;
> +		int i;
> +
> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;
> +		}
> +		string_list_clear(&conflicted_files, 1);
> +	}
>  	if (o->show_messages) {
>  		printf("\n");

... it seems that we do this already...

>  		merge_display_update_messages(&opt, &result, stdout);
> @@ -502,7 +522,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>
>  	/* Do the relevant type of merge */
>  	if (o.real)
> -		return real_merge(&o, argv[0], argv[1]);
> +		return real_merge(&o, argv[0], argv[1], prefix);
>  	else
>  		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> index c34f8e6c1ed..43c9950dedb 100755
> --- a/t/t4301-merge-tree-real.sh
> +++ b/t/t4301-merge-tree-real.sh
> @@ -94,6 +94,8 @@ test_expect_success 'test conflict notices and such' '
>  	#   "whatever" has *both* a modify/delete and a file/directory conflict
>  	cat <<-EOF >expect &&
>  	HASH
> +	greeting
> +	whatever~side1
>
>  	Auto-merging greeting
>  	CONFLICT (content): Merge conflict in greeting

... as illustrated by the test, too. I guess the documentation should show
the empty line, too?

Ciao,
Dscho

> @@ -105,4 +107,13 @@ test_expect_success 'test conflict notices and such' '
>  	test_cmp expect actual
>  '
>
> +test_expect_success 'Just the conflicted files without the messages' '
> +	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
> +	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
> +
> +	test_write_lines HASH greeting whatever~side1 >expect &&
> +
> +	test_cmp expect actual
> +'
> +
>  test_done
> --
> gitgitgadget
>
>

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1897 bytes --]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> Note1: Depends on en/remerge-diff (but no need to pick it up; it's still
> RFC).

For reasons, I will have to backport this on top of v2.33.1 ;-)

> == Updates Log ==
>
> Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
> suggestions and comments on v2):
>
>  * Significant changes to output format:
>    * Flags no longer take a filename for additional output; they write to
>      stdout instead.
>    * More information included by default when there are conflicts (no need
>      to request it with additional flags, instead flags can be used to
>      suppress it).
>    * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
>      information -- when there are conflicts. Add a flag to only list
>      conflicted files if that's preferred.
>  * Much more thorough manual for git-merge-tree.txt
>  * Renamed option from --real to --write-tree
>  * Accept an optional --trivial-merge option to get old style merge-tree
>    behavior
>  * Allow both --write-tree and --trivial-merge to be omitted since we can
>    deduce which from number of arguments
>  * Document exit code when the merge cannot be run (so we can distinguish
>    other error cases from conflicts)
>  * testcase cleanups: test_tick, early skip of test when using recursive
>    backend, variable renames, etc.
>  * various minor code cleanups
>  * Add a new --allow-unrelated-histories option (with same meaning as the
>    one used in git merge)
>  * Rebased on top of en/remerge-diff to avoid a small conflict

I am really happy with the way this patch series is going, and hope to be
a lot more active on it in the near future.

I've read through the current iteration and left a few suggestions,
nothing major.

Thank you so much for your outstanding work!
Dscho

Re: [PATCH 04/12] merge-tree: implement real merges

[Elijah Newren]

On Wed, Jan 26, 2022 at 1:45 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> >
> > From: Elijah Newren <newren@gmail.com>
> >
> > This adds the ability to perform real merges rather than just trivial
> > merges (meaning handling three way content merges, recursive ancestor
> > consolidation, renames, proper directory/file conflict handling, and so
> > forth).  However, unlike `git merge`, the working tree and index are
> > left alone and no branch is updated.
> >
> > The only output is:
> >   - the toplevel resulting tree printed on stdout
> >   - exit status of 0 (clean) or 1 (conflicts present)
>
> The exit status can now actually be something other than 0 and 1
> according to the doc and code below.

Thanks for catching; will fix.

> > +Performs a merge, but does not make any new commits and does not read
> > +from or write to either the working tree or index.
> > +
> > +The first form will merge the two branches, doing a full recursive
> > +merge with rename detection.
>
> Maybe this could already tell that the first form will also write a
> tree with the result of the merge (even in case of conflict) as this
> could help understand the reason why the associated option is called
> '--write-tree'. It could also help to say that we call such a merge a
> 'real' merge.

Makes sense.

> > The rest of this manual (other than the
> > +next paragraph) describes the first form in more detail -- including
> > +options, output format, exit status, and usage notes.
>
> >  static int real_merge(struct merge_tree_options *o,
> >                       const char *branch1, const char *branch2)
> >  {
> > -       die(_("real merges are not yet implemented"));
> > +       struct commit *parent1, *parent2;
> > +       struct commit_list *common;
> > +       struct commit_list *merge_bases = NULL;
> > +       struct commit_list *j;
> > +       struct merge_options opt;
> > +       struct merge_result result = { 0 };
> > +
> > +       parent1 = get_merge_parent(branch1);
> > +       if (!parent1)
> > +               help_unknown_ref(branch1, "merge",
> > +                                _("not something we can merge"));
>
> The second argument is supposed to be the command (it's called "cmd"),
> so maybe "merge-tree" instead of "merge".

Oh, good catch; thanks for pointing this out.

>
> > +       parent2 = get_merge_parent(branch2);
> > +       if (!parent2)
> > +               help_unknown_ref(branch2, "merge",
> > +                                _("not something we can merge"));
>
> idem
>
> > +       opt.show_rename_progress = 0;
> > +
> > +       opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
> > +       opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */
>
> I think just:
>
>        opt.branch1 = branch1
>        opt.branch2 = branch2
>
> might be better for users as it should show the name as it was passed
> to the command.

After digging for a bit, I think in this case there actually isn't a
difference to users because both will give the same result.  But, if
that's the case, the simpler code is warranted.

> > +       merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > +       printf("%s\n", oid_to_hex(&result.tree->object.oid));
>
> I wonder if we can actually always output a valid tree when
> result.clean < 0. In case we might not, the printing should go a few
> lines below.

Yeah, I caught that and fixed it, but got it squashed into a later
commit.  I'll fix it up.

> > +       if (result.clean < 0)
> > +               die(_("failure to merge"));
> > +       else if (!result.clean)
>
> The "else" is not necessary above.
>
> > +               printf(_("Conflicts!\n"));

Yes, and the else clause should just be ripped out.

> > +       merge_finalize(&opt, &result);
> > +       return !result.clean; /* result.clean < 0 handled above */
> >  }
>
> > diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> > new file mode 100755
> > index 00000000000..e03688515c5
> > --- /dev/null
> > +++ b/t/t4301-merge-tree-real.sh
>
> I wonder if it would be better named 't4301-merge-tree-write-tree.sh'...
>
> > @@ -0,0 +1,87 @@
> > +#!/bin/sh
> > +
> > +test_description='git merge-tree --write-tree'
>
> ... especially given this description.

Makes sense; will rename.

Re: [PATCH 06/12] merge-ort: allow update messages to be written to different file stream

[Elijah Newren]

On Fri, Jan 28, 2022 at 8:31 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > diff --git a/merge-ort.c b/merge-ort.c
> > index f9e35b0f96b..b78dde55ad9 100644
> > --- a/merge-ort.c
> > +++ b/merge-ort.c
> > @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
> >  }
> >
> >  void merge_display_update_messages(struct merge_options *opt,
> > -                                struct merge_result *result)
> > +                                struct merge_result *result,
> > +                                FILE *stream)
> >  {
> >       struct merge_options_internal *opti = result->priv;
> >       struct hashmap_iter iter;
> > @@ -4263,7 +4264,7 @@ void merge_display_update_messages(struct merge_options *opt,
> >       for (i = 0; i < olist.nr; ++i) {
> >               struct strbuf *sb = olist.items[i].util;
> >
> > -             printf("%s", sb->buf);
> > +             fprintf(stream, "%s", sb->buf);
>
> Maybe `strbuf_write(sb, stream);` instead? Whenever I see a `"%s"`, I feel
> like it's unnecessary churn...

Makes sense.

> >       }
> >       string_list_clear(&olist, 0);
> >
>
> Missing from this hunk:
>
>         /* Also include needed rename limit adjustment now */
>         diff_warn_rename_limit("merge.renamelimit",
>                                opti->renames.needed_limit, 0);
>
> This function explicitly writes to `stdout`, and will need to be adjusted,
> I think, before we can include an adjustment to this call in this patch.

Oh, good point.

> Unless we override `warn_routine()` (which is used inside that function),
> that is. Which is hacky, and we would not have addressed the
> `fflush(stdout)` in `diff_warn_rename_limit()`. So I would much prefer
> something like this:
>
> -- snip --
> diff --git a/diff.c b/diff.c
> index 861282db1c3..87966602cbd 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -6312,17 +6312,25 @@ static const char rename_limit_advice[] =
>  N_("you may want to set your %s variable to at least "
>     "%d and retry the command.");
>
> -void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
> +void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> +                           FILE *out)
>  {
> -       fflush(stdout);
> +       const char *fmt = NULL;
> +
>         if (degraded_cc)
> -               warning(_(degrade_cc_to_c_warning));
> +               fmt = _(degrade_cc_to_c_warning);
>         else if (needed)
> -               warning(_(rename_limit_warning));
> +               fmt = _(rename_limit_warning);
>         else
>                 return;
>         if (0 < needed)
> -               warning(_(rename_limit_advice), varname, needed);
> +               fmt = _(rename_limit_advice);
> +
> +       fflush(out);
> +       if (out == stdout)
> +               warning(fmt, varname, needed);
> +       else
> +               fprintf(out, fmt, varname, needed);
>  }
>
>  static void diff_flush_patch_all_file_pairs(struct diff_options *o)
> @@ -6754,7 +6762,7 @@ int diff_result_code(struct diff_options *opt, int status)
>
>         diff_warn_rename_limit("diff.renameLimit",
>                                opt->needed_rename_limit,
> -                              opt->degraded_cc_to_c);
> +                              opt->degraded_cc_to_c, stdout);
>         if (!opt->flags.exit_with_status &&
>             !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
>                 return status;
> diff --git a/diff.h b/diff.h
> index 8ba85c5e605..be4ee68c0a2 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -596,7 +596,8 @@ void diffcore_fix_diff_index(void);
>  int diff_queue_is_empty(void);
>  void diff_flush(struct diff_options*);
>  void diff_free(struct diff_options*);
> -void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
> +void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> +                           FILE *out);
>
>  /* diff-raw status letters */
>  #define DIFF_STATUS_ADDED              'A'
> diff --git a/merge-ort.c b/merge-ort.c
> index 0342f104836..e6b5a0e7c64 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4264,7 +4264,7 @@ void merge_switch_to_result(struct merge_options *opt,
>
>                 /* Also include needed rename limit adjustment now */
>                 diff_warn_rename_limit("merge.renamelimit",
> -                                      opti->renames.needed_limit, 0);
> +                                      opti->renames.needed_limit, 0, stdout);
>
>                 trace2_region_leave("merge", "display messages", opt->repo);
>         }
> diff --git a/merge-recursive.c b/merge-recursive.c
> index d9457797dbb..10b2948678c 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3731,7 +3731,8 @@ static void merge_finalize(struct merge_options *opt)
>                 strbuf_release(&opt->obuf);
>         if (show(opt, 2))
>                 diff_warn_rename_limit("merge.renamelimit",
> -                                      opt->priv->needed_rename_limit, 0);
> +                                      opt->priv->needed_rename_limit, 0,
> +                                      stdout);
>         FREE_AND_NULL(opt->priv);
>  }
>
> -- snap --

I like this, but believe it makes more sense as a preparatory patch.
So I created one and made you the author, and included your
signed-off-by.  That okay with you?

>
> The rest of the patch looks good to me.

Thanks!

Re: [PATCH 07/12] merge-tree: support including merge messages in output

[Elijah Newren]

On Fri, Jan 28, 2022 at 8:37 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > When running `git merge-tree --write-tree`, we previously would only
> > return an exit status reflecting the cleanness of a merge, and print out
> > the toplevel tree of the resulting merge.  Merges also have
> > informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
> > ...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
> > conflicts occur (such as file/directory, modify/delete, add/add with
> > differing modes, rename/rename (1to2), etc.), these informational
> > messages are often the only notification since these conflicts are not
> > representable in the contents of the file.
> >
> > Add a --[no-]messages option so that callers can request these messages
> > be included at the end of the output.  Include such messages by default
> > when there are conflicts, and omit them by default when the merge is
> > clean.
>
> Makes sense.
>
> > diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> > index 0c19639594d..560640ad911 100644
> > --- a/builtin/merge-tree.c
> > +++ b/builtin/merge-tree.c
> > @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
> >               commit_list_insert(j->item, &merge_bases);
> >
> >       merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > -     printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +
> >       if (result.clean < 0)
> >               die(_("failure to merge"));
> > -     else if (!result.clean)
> > -             printf(_("Conflicts!\n"));
> > +
> > +     if (o->show_messages == -1)
> > +             o->show_messages = !result.clean;
> > +
> > +     printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +     if (o->show_messages) {
> > +             printf("\n");
> > +             merge_display_update_messages(&opt, &result, stdout);
> > +     }
>
> Excellent.
>
> >       merge_finalize(&opt, &result);
> >       return !result.clean; /* result.clean < 0 handled above */
> >  }
> >
> >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >  {
> > -     struct merge_tree_options o = { 0 };
> > +     struct merge_tree_options o = { .show_messages = -1 };
> >       int expected_remaining_argc;
> > +     int original_argc;
> >
> >       const char * const merge_tree_usage[] = {
> > -             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> > +             N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
> >               N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> >               NULL
> >       };
> > @@ -464,6 +473,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >                        N_("do a real merge instead of a trivial merge")),
> >               OPT_BOOL(0, "trivial-merge", &o.trivial,
> >                        N_("do a trivial merge only")),
> > +             OPT_BOOL(0, "messages", &o.show_messages,
> > +                      N_("also show informational/conflict messages")),
> >               OPT_END()
> >       };
> >
> > @@ -472,10 +483,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >               usage_with_options(merge_tree_usage, mt_options);
> >
> >       /* Parse arguments */
> > +     original_argc = argc;
> >       argc = parse_options(argc, argv, prefix, mt_options,
> >                            merge_tree_usage, 0);
> >       if (o.real && o.trivial)
> >               die(_("--write-tree and --trivial-merge are incompatible"));
> > +     if (!o.real && original_argc < argc)
> > +             die(_("--write-tree must be specified if any other options are"));
>
> Hmm. Well. Hmm.
>
>
> I'd rather keep `--write-tree` neat and optional. What's wrong with
> allowing
>
>         git merge-tree --no-messages HEAD MERGE_HEAD
>
> ?

Yeah, oops.  Nothing is wrong with that...but do note that there is
something wrong with this:

    git merge-tree --no-messages MERGE_BASE HEAD MERGE_HEAD

because the three argument form is signalling the deprecated trivial
merge, and the trivial merge code doesn't handle any options.  You
were right to call out my code since I placed it too early and made it
a bit obtuse.  I've changed it to:

    if (o.mode == 't' && original_argc < argc)
        die(_("--trivial-merge is incompatible with all other options"));

where o.mode is guaranteed to be set before that check.

> To be clear, I think we need this instead:
>
>         if (o.trivial && o.show_messages >= 0)
>                 die(_("--trivial-merge is incompatible with additional options"));

Most of that is better, assuming the o.trivial check handles the
implicit case too.  Thanks.  I'm not so sure about the check on
o.show_messages, though, because then I'll have to keep adding
additional checks for each new additional option (and future patches
will add more).  So, I think I'll keep the original_argc < argc part
of the check instead of that.  :-)

> I like the rest of the patch very much!

Thanks!

Re: [PATCH 07/12] merge-tree: support including merge messages in output

[Elijah Newren]

On Wed, Jan 26, 2022 at 2:42 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> >  EXIT STATUS
> >  -----------
> > @@ -72,7 +102,8 @@ be used as a part of a series of steps such as
> >
> >  However, it does not quite fit into the same category of low-level
> >  plumbing commands since the possibility of merge conflicts give it a
> > -much higher chance of the command not succeeding.
> > +much higher chance of the command not succeeding (and NEWTREE containing
> > +a bunch of stuff other than just a toplevel tree).
>
> Is this hunk really related to this commit or should it go into a
> previous commit?

It's meant to first be related to this commit, though as you pointed
out below, I had some accidental stuff not cleaned out of the earlier
commit.

> > @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
> >                 commit_list_insert(j->item, &merge_bases);
> >
> >         merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > -       printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +
> >         if (result.clean < 0)
> >                 die(_("failure to merge"));
>
> So this addresses the comment I made in a previous commit related to
> the fact that if result.clean < 0 we might not have a valid tree that
> we can print. I think though that it would be better if that was
> addressed in a previous commit.
>
> > -       else if (!result.clean)
> > -               printf(_("Conflicts!\n"));
>
> Ok, so we don't print "Conflicts!\n" now, which makes me wonder if we
> should have printed it in the first place in previous commits.

Yep, good flag on both of these last two comments.  When I was fixing
this up I didn't squash it back in early enough.  Thanks for reading
carefully.

> >         if (o.real && o.trivial)
> >                 die(_("--write-tree and --trivial-merge are incompatible"));
> > +       if (!o.real && original_argc < argc)
> > +               die(_("--write-tree must be specified if any other options are"));
>
> Is this necessary? It looks to me like another thing that would be
> simplified if we were just adding a new command...

I think the code is harder to read than it should be.  I changed it to:

    if (o.mode == 't' && original_argc < argc)
        die(_("--trivial-merge is incompatible with all other options"));

which I think is clearer; it points out that it's only about the
deprecated --trivial-merge option and how it's incompatible with all
other options.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Wed, Jan 26, 2022 at 2:55 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> > After a merge, this function allows the user to extract the same
> > information that would be printed by `ls-files -u` -- conflicted
>
> This made me wonder if "-- conflicted" should be part of the `ls-files
> -u` command. Maybe "by `ls-files -u`, which means" would make things a
> bit clearer.
>
> > files with their mode, oid, and stage.

Yes, I like that wording better.  Thanks!

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Wed, Jan 26, 2022 at 3:07 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> > +void merge_get_conflicted_files(struct merge_result *result,
> > +                               struct string_list *conflicted_files)
> > +{
> > +       struct hashmap_iter iter;
> > +       struct strmap_entry *e;
> > +       struct merge_options_internal *opti = result->priv;
> > +
> > +       strmap_for_each_entry(&opti->conflicted, &iter, e) {
> > +               const char *path = e->key;
> > +               struct conflict_info *ci = e->value;
> > +               int i;
> > +
> > +               VERIFY_CI(ci);
> > +
> > +               for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> > +                       struct stage_info *si;
> > +
> > +                       if (!(ci->filemask & (1ul << i)))
> > +                               continue;
> > +
> > +                       si = xmalloc(sizeof(*si));
>
> It's probably a premature optimization, so feel free to ignore, but as
> MERGE_BASE and MERGE_SIDE2 are constants, and ci->filemask is constant
> inside the 'for' loop, we could compute before the 'for' loop how many
> 'struct stage_info' we will need and allocate them all at once before
> the 'for' loop.

That's an interesting idea, but if we allocate all at once, then we'd
need to be able to deallocate all at once.  I'm not sure how to do
that, but basically the straightforward
    string_list_free(conflicted_files, 1);
that callers can use with the current function would no longer be
valid, and it might be somewhat difficult for callers to figure out
how to free the memory that was allocated.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > After a merge, this function allows the user to extract the same
> > information that would be printed by `ls-files -u` -- conflicted
> > files with their mode, oid, and stage.
>
> Hmm. Okay.
>
> I am not really a fan of that output where we use a variable
> number of lines per file. As in: in the regular case, where a file was
> modified in divergent ways, we get three lines. But if it was deleted in
> one branch, or if it was created in both branches, we have only two lines.
>
> Frankly, I'd much rather have 3 lines for each and every conflict.

Out of curiosity, does this also mean you feel like `git ls-files -u`
is mis-designed (even if we're stuck with it for backward
compatibility reasons)?

Anyway, if we make this change guaranteeing there are 3 lines for each
and every conflict, that probably implies we can remove the stage
field (just letting it be implicit), right?  And it also implies that
the "path" field of the lines is now duplicate information.  Should
the path be printed on each line regardless of the duplication?
Should the path be printed separately on its own line, followed by the
three lines of (mode, oid) pairs?  Or should the format be something
else entirely?

> Granted, we currently only have three stages...

I thought that was true too, but Junio informed me otherwise[1].
Granted, ort does not currently make use of that, but it could.  Not
sure I want to rule it out.

[1] https://lore.kernel.org/git/xmqqr1t89gi8.fsf@gitster.c.googlers.com/

> ...and we can pretty much
> guarantee that at least two of these stages are non-empty (otherwise where
> would be the conflict?).

No, that's not true.  See the paragraph about "conflict types" from
the final patch in this series, where I mentioned three different
types of conflicts that can result in a path having a single higher
order stage.

> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> missing from the second conflict, in the output we would see stages 1, 2,
> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> different conflicts.

I don't understand why you're fixating on the stage here.  Why would
you want to group all the stage 2s together, count them up, and then
determine there are N conflicting files because there are N stage 2's?
 In such a design, you'd have to do the extra post-processing to
recognize the all-zero modes and hashes and toss them.

To me, that seems like more work than just handling the `ls-files -u`
style of output, and doesn't have the advantage of showing things in a
format users may already be familiar with (see above about dropping
stages and maybe also pathname).  Since the `ls-files -u` output is
simply several lines of:
   <mode> <hash> <stage> <filename>
You can get the number of conflicted files by counting the number of
*unique* filenames.  If you want to see which lines are about the same
file, get all the lines with the same filename -- they are sorted by
(<filename>, <stage>) for convenience, much like `ls-files -u` is.


On a different angle, I'm also slightly worried there's an embedded
assumption here, one that I tried to address in the "Mistake to avoid"
section I added in my last patch of this series.  What if you have a
conflicting merge and 0 lines of output in the conflicting info
section?  Is there something about the reason you're asking for three
lines of output per conflicted file which is going to make you
overlook that particular possibility and not handle it?

> But still. It makes me uneasy to have that much variability in
> machine-consumable output.
>
> And who knows, maybe if we're ever implementing more sophisticated merge
> strategies for octopus merges, we might end up with more stages...

Um, if we need to handle more stages, wouldn't that undercut your
request for exactly 3 stages?  Wouldn't it suggest that we would
indeed want to have a flexible number of lines?

> In other words...
>
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  merge-ort.c | 31 +++++++++++++++++++++++++++++++
> >  merge-ort.h | 21 +++++++++++++++++++++
> >  2 files changed, 52 insertions(+)
> >
> > diff --git a/merge-ort.c b/merge-ort.c
> > index b78dde55ad9..5e7cea6cc8f 100644
> > --- a/merge-ort.c
> > +++ b/merge-ort.c
> > @@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
> >       trace2_region_leave("merge", "display messages", opt->repo);
> >  }
> >
> > +void merge_get_conflicted_files(struct merge_result *result,
> > +                             struct string_list *conflicted_files)
> > +{
> > +     struct hashmap_iter iter;
> > +     struct strmap_entry *e;
> > +     struct merge_options_internal *opti = result->priv;
> > +
> > +     strmap_for_each_entry(&opti->conflicted, &iter, e) {
> > +             const char *path = e->key;
> > +             struct conflict_info *ci = e->value;
> > +             int i;
> > +
> > +             VERIFY_CI(ci);
> > +
> > +             for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> > +                     struct stage_info *si;
> > +
> > +                     if (!(ci->filemask & (1ul << i)))
> > +                             continue;
> > +
> > +                     si = xmalloc(sizeof(*si));
> > +                     si->stage = i+1;
> > +                     si->mode = ci->stages[i].mode;
> > +                     oidcpy(&si->oid, &ci->stages[i].oid);
> > +                     string_list_append(conflicted_files, path)->util = si;
> > +             }
> > +     }
> > +     /* string_list_sort() uses a stable sort, so we're good */
> > +     string_list_sort(conflicted_files);
> > +}
> > +
> >  void merge_switch_to_result(struct merge_options *opt,
> >                           struct tree *head,
> >                           struct merge_result *result,
> > diff --git a/merge-ort.h b/merge-ort.h
> > index d643b47cb7c..e635a294ea8 100644
> > --- a/merge-ort.h
> > +++ b/merge-ort.h
> > @@ -2,6 +2,7 @@
> >  #define MERGE_ORT_H
> >
> >  #include "merge-recursive.h"
> > +#include "hash.h"
> >
> >  struct commit;
> >  struct tree;
> > @@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
> >                                  struct merge_result *result,
> >                                  FILE *stream);
> >
> > +struct stage_info {
> > +     struct object_id oid;
> > +     int mode;
> > +     int stage;
> > +};
>
> ... I'd rather not tack this onto a `string_list` but instead have
> something like:
>
>         struct conflict_info {

Nit: "conflict_info" is already taken for another structure; we'd need
a different name.  But it does illustrate the idea just fine.

>                 struct {
>                         struct object_id oid;
>                         int mode;
>                         const char *path;
>                 } stages[3]
>         };
>
> where `path` can be `NULL` to indicate that this stage is missing.

I'll get back to the data structure in a second, but the note about
path being `NULL` is interesting.  I'm presuming that oid and mode are
all-zeros as well, yes?  Now, your original request was that you
wanted a line printed for all three stages.  If we're printing oid,
mode, and path for each stage, what do we print for the path one these
lines?  Is it (1) "(null)", (2) "", or (3) the real pathname (via
carefully comparing to surrounding stages to find out what it is)?  If
we're changing the format to only print the name of the path once,
does the code need to loop over the list of conflicts in order to find
out the name (since the first or second stage might have a NULL path
field)?

In regards to the overall data structure and your comment about
string_list: I'm slightly confused.  You say you want to avoid a
string_list, but you've only accounted for there being one conflict in
this data structure.  I don't see how this suggestion avoids the need
for an array or a string_list or some kind of container to hold
multiple of these.

The inclusion of path within the stages array is also interesting --
should there be (up to) three copies of the pathname allocated per
conflict?  Seems a bit wasteful.  Wouldn't it make more sense to have
something like

    struct conflicts {
        struct {
            struct object_id oid;
            unsigned short mode;
        } stages[3]
    };

and then have a string_list storing the pathname -> conflicts mappings
to avoid the pathname duplication?  Or is there something you really
find problematic about the string_list and you really want an array or
other data structure?

Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts

[Elijah Newren]

On Fri, Jan 28, 2022 at 8:57 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
[...]
> > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > index fd7a867de60..041a4ac2785 100644
> > --- a/Documentation/git-merge-tree.txt
> > +++ b/Documentation/git-merge-tree.txt
> > @@ -58,6 +58,7 @@ simply one line:
> >  Whereas for a conflicted merge, the output is by default of the form:
> >
> >       <OID of toplevel tree>
> > +     <Conflicted file list>
> >       <Informational messages>
>
> To distinguish between the list of conflicted files and the informational
> messages, I think it would be good to insert an empty line, as a
> separator, like.

Yes, I agree; that's why I did so.  :-)

[...]
> >       if (o->show_messages) {
> >               printf("\n");
>
> ... it seems that we do this already...

Yep.

[...]
> > diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> > index c34f8e6c1ed..43c9950dedb 100755
> > --- a/t/t4301-merge-tree-real.sh
> > +++ b/t/t4301-merge-tree-real.sh
> > @@ -94,6 +94,8 @@ test_expect_success 'test conflict notices and such' '
> >       #   "whatever" has *both* a modify/delete and a file/directory conflict
> >       cat <<-EOF >expect &&
> >       HASH
> > +     greeting
> > +     whatever~side1
> >
> >       Auto-merging greeting
> >       CONFLICT (content): Merge conflict in greeting
>
> ... as illustrated by the test, too. I guess the documentation should show
> the empty line, too?

It does:

    Informational messages
    ~~~~~~~~~~~~~~~~~~~~~~

    This always starts with a blank line to separate it from the previous
    sections, and then has free-form messages about the merge, such as:

The newline should not be split out separately, because --no-messages
suppresses the newline.  (Without the messages section, the newline
isn't needed.)

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Elijah Newren]

On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> > Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
> > suggestions and comments on v2):
> >
> >  * Significant changes to output format:
> >    * Flags no longer take a filename for additional output; they write to
> >      stdout instead.
> >    * More information included by default when there are conflicts (no need
> >      to request it with additional flags, instead flags can be used to
> >      suppress it).
> >    * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
> >      information -- when there are conflicts. Add a flag to only list
> >      conflicted files if that's preferred.
>
> The above changes seem good to me.

:-)

> >  * Much more thorough manual for git-merge-tree.txt
> >  * Renamed option from --real to --write-tree
> >  * Accept an optional --trivial-merge option to get old style merge-tree
> >    behavior
> >  * Allow both --write-tree and --trivial-merge to be omitted since we can
> >    deduce which from number of arguments
>
> I still think that it might be simpler and cleaner to leave 'git
> merge-tree' alone for now, and just add a new command named for
> example 'git write-merge-tree'. Later we can always add flags to 'git
> merge-tree' or add 'git trivial-merge-tree' as an alias for 'git
> merge-tree', and eventually slowly switch 'git merge-tree' to mean
> only 'git write-merge-tree' if that's where we want to go.

Understood.  Since we can't immediately kill the old merge-tree, I
don't think there's a perfectly clean solution here, and it's totally
understandable that different folks would have different opinions on
which interim choice would be cleanest.  What you suggest is
reasonable, I just personally prefer the path in this series.

> >  * Document exit code when the merge cannot be run (so we can distinguish
> >    other error cases from conflicts)
> >  * testcase cleanups: test_tick, early skip of test when using recursive
> >    backend, variable renames, etc.
> >  * various minor code cleanups
> >  * Add a new --allow-unrelated-histories option (with same meaning as the
> >    one used in git merge)
>
> The above changes seem good to me too.

Thanks for reading over things carefully and for providing many
detailed, helpful comments!

> > Stuff intentionally NOT included, but which others seemed to feel strongly
> > about; they'd need to convince me more on these:
> >
> >  * Any form of diff output[1]
>
> It's not a big issue for me to not include them right now as long as
> it's possible to add cli options later that add them.

My main concern is just that `merge-tree` remain a low-level tool and
have machine-parseable output.  I was a little worried that both you
and Dscho wanted everything on stdout rather than in separate files,
as the <Informational messages> part of the output is rather
free-form.  But since it's at the end, and has a machine-parseable
beginning, it can just be slurped in and we're all good.  The diff
output raises my eyebrow because I'm worried we're losing this
property.  If there are clear usecases for adding more output, and we
can do so without losing this machine-parseable property, I don't have
a problem with adding an option for it.

One analogy we might use here is that `git merge` provides a diffstat
at the end.  What you're asking is more than a diffstat, but might be
considered similar-ish in nature.

> The reason is
> that I think in many cases when there are conflicts, the conflicts
> will be small and the user will want to see them.

I'm a little worried about the assumption here that conflict size is
measurable and visible via diffs.  That might be true in some cases,
but a UI written with that assumption is going to be very confusing
when hitting cases where that assumption does not hold.  For example:

  * What if there is a binary file conflict, or a modify/delete or
rename/delete conflict, or failed-to-merge submodule conflict, or a
file location conflict? (For these, there is no diff relative to the
first parent and hence this conflict would have no diff output for
it)?
  * What if there was a simple file/directory conflict?  A diff would
show a rename (even when neither side did any renames), but not any
conflict markers.
  * What if there was a rename/rename conflict (both sides renamed
same file differently) or a distinct types conflict?  The former
results in three different conflicting files, none of them with
conflict markers, while the latter results in two different
conflicting files both without conflict markers?  Showing individual
per-file diffs totally loses all context here -- it'll show no-diff
for one of the files, and totally new additions for the ones.

Such a problem statement just seems fraught with edge cases to me, and
suggests that the problem statement might be in need of revisiting.

Don't read this as me closing the door on the possibility of diffs;
I'm not trying to do that.  I'm listing my misgivings about how I
think they might be used (i.e. be careful if you're headed down this
path as you might be digging yourself a never-ending support hole).
You can also think of my comments as feedback to consider and address
when you propose a future feature addition for adding diffs.  If/when
you propose such a feature, we'd probably be able to dive more into
specifics and usecases at that time, which may or may not circumvent
my concerns.

> So it would be
> simpler to just have an option to show any conflict right away, rather
> than have the user launch another command (a diff-tree against which
> tree and with which options?).

Um, this part I'm not sure I get.  I thought the reason for the diffs
was performance -- you knew you wanted the diffs, and you wanted it
done as part of the same process.  But why would this be simpler?
Your patch series included three different diffs, and the emails you
pointed me at suggested all kinds of configurability.  That suggests
the merge-tree command would have to take the exact same options the
user would supply to diff, and thus would have to be told all the same
options, right?  I don't see how this removes any complexity at all
for the user.

Unless...is the request in some way similar to merge's diffstat where
there is always a very specific type of diff that is wanted and you
aren't envisioning much flexibility in what kind of diff or what to
diff against -- is that where the simplification comes from?

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Christian Couder]

On Sat, Jan 29, 2022 at 8:04 AM Elijah Newren <newren@gmail.com> wrote:
>
> On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
> <christian.couder@gmail.com> wrote:
> >
> > On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:

> > > Stuff intentionally NOT included, but which others seemed to feel strongly
> > > about; they'd need to convince me more on these:
> > >
> > >  * Any form of diff output[1]
> >
> > It's not a big issue for me to not include them right now as long as
> > it's possible to add cli options later that add them.
>
> My main concern is just that `merge-tree` remain a low-level tool and
> have machine-parseable output.  I was a little worried that both you
> and Dscho wanted everything on stdout rather than in separate files,
> as the <Informational messages> part of the output is rather
> free-form.  But since it's at the end, and has a machine-parseable
> beginning, it can just be slurped in and we're all good.  The diff
> output raises my eyebrow because I'm worried we're losing this
> property.  If there are clear usecases for adding more output, and we
> can do so without losing this machine-parseable property, I don't have
> a problem with adding an option for it.

That's ok for me for now. I will certainly not work on adding options
for diff output without any usecase.

> One analogy we might use here is that `git merge` provides a diffstat
> at the end.  What you're asking is more than a diffstat, but might be
> considered similar-ish in nature.
>
> > The reason is
> > that I think in many cases when there are conflicts, the conflicts
> > will be small and the user will want to see them.
>
> I'm a little worried about the assumption here that conflict size is
> measurable and visible via diffs.  That might be true in some cases,
> but a UI written with that assumption is going to be very confusing
> when hitting cases where that assumption does not hold.  For example:
>
>   * What if there is a binary file conflict, or a modify/delete or
> rename/delete conflict, or failed-to-merge submodule conflict, or a
> file location conflict? (For these, there is no diff relative to the
> first parent and hence this conflict would have no diff output for
> it)?
>   * What if there was a simple file/directory conflict?  A diff would
> show a rename (even when neither side did any renames), but not any
> conflict markers.
>   * What if there was a rename/rename conflict (both sides renamed
> same file differently) or a distinct types conflict?  The former
> results in three different conflicting files, none of them with
> conflict markers, while the latter results in two different
> conflicting files both without conflict markers?  Showing individual
> per-file diffs totally loses all context here -- it'll show no-diff
> for one of the files, and totally new additions for the ones.

In those cases we just tell users that they cannot resolve those
conflicts in the user interface, see the following doc:

https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#conflicts-you-can-resolve-in-the-user-interface

> Such a problem statement just seems fraught with edge cases to me, and
> suggests that the problem statement might be in need of revisiting.

Users understand that some kinds of conflicts cannot yet be resolved
using a user interface. Maybe we will be able to make improvements so
that more kinds of conflicts can be resolved in a UI in the future
though. That's why a flexible and extensible output could help.

> Don't read this as me closing the door on the possibility of diffs;
> I'm not trying to do that.  I'm listing my misgivings about how I
> think they might be used (i.e. be careful if you're headed down this
> path as you might be digging yourself a never-ending support hole).
> You can also think of my comments as feedback to consider and address
> when you propose a future feature addition for adding diffs.  If/when
> you propose such a feature, we'd probably be able to dive more into
> specifics and usecases at that time, which may or may not circumvent
> my concerns.

I know that diffs, or any new single feature, will likely not be a
silver bullet.

> > So it would be
> > simpler to just have an option to show any conflict right away, rather
> > than have the user launch another command (a diff-tree against which
> > tree and with which options?).
>
> Um, this part I'm not sure I get.  I thought the reason for the diffs
> was performance -- you knew you wanted the diffs, and you wanted it
> done as part of the same process.  But why would this be simpler?

In the commit message of 4/12 you show an example of using it in simple scripts:

NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
test $? -eq 0 || die "There were conflicts..."
...

So I think it would be simpler for someone interested in seeing the
conflicts, like a script writer, or maybe someone using it manually
for example as a dry run before performing a merge, to be able to get
them right away from the command rather than to have to use another
command (which means finding the right command, arguments and options
for that) to get them.

> Your patch series included three different diffs, and the emails you
> pointed me at suggested all kinds of configurability.  That suggests
> the merge-tree command would have to take the exact same options the
> user would supply to diff, and thus would have to be told all the same
> options, right?  I don't see how this removes any complexity at all
> for the user.
>
> Unless...is the request in some way similar to merge's diffstat where
> there is always a very specific type of diff that is wanted and you
> aren't envisioning much flexibility in what kind of diff or what to
> diff against -- is that where the simplification comes from?

Well I just think the default diff output could be tailored for the
most likely usecases and options made available later for more
advanced usecases or users.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Sixt]

Just a heckling from the peanut gallery...

Am 29.01.22 um 07:08 schrieb Elijah Newren:
> On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
>> missing from the second conflict, in the output we would see stages 1, 2,
>> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
>> different conflicts.
> 
> I don't understand why you're fixating on the stage here.  Why would
> you want to group all the stage 2s together, count them up, and then
> determine there are N conflicting files because there are N stage 2's?

Looks like you are misunderstanding Dscho's point: When you have two
conflicts, the first with stages 1 and 2, the second with stages 2 and
3, then the 2s occur lumped together when the 4 lines are printed in a
row, and that is the cue to the parser where the new conflict begins.
Dscho did not mean that all N 2s of should be listed together.

-- Hannes

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
>
> Just a heckling from the peanut gallery...
>
> Am 29.01.22 um 07:08 schrieb Elijah Newren:
> > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> >> missing from the second conflict, in the output we would see stages 1, 2,
> >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> >> different conflicts.
> >
> > I don't understand why you're fixating on the stage here.  Why would
> > you want to group all the stage 2s together, count them up, and then
> > determine there are N conflicting files because there are N stage 2's?
>
> Looks like you are misunderstanding Dscho's point: When you have two
> conflicts, the first with stages 1 and 2, the second with stages 2 and
> 3, then the 2s occur lumped together when the 4 lines are printed in a
> row, and that is the cue to the parser where the new conflict begins.
> Dscho did not mean that all N 2s of should be listed together.

Ah, so...I didn't understand his misunderstanding?  Using stages as a
cue to the parser where the new conflict begins is broken; you should
instead check for when the filename listed on a line does not match
the filename on the previous line.  In particular, if one conflict has
stages 1 and 2, and the next conflict has only stage 3, then looking
at stages only might cause you to accidentally lump unrelated
conflicts together.

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Elijah Newren]

Hi Christian,

On Sat, Jan 29, 2022 at 12:18 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 29, 2022 at 8:04 AM Elijah Newren <newren@gmail.com> wrote:
> >
> > On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
> > <christian.couder@gmail.com> wrote:
> > >
> > > On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> > > <gitgitgadget@gmail.com> wrote:
>
> > > > Stuff intentionally NOT included, but which others seemed to feel strongly
> > > > about; they'd need to convince me more on these:
> > > >
> > > >  * Any form of diff output[1]
> > >
> > > It's not a big issue for me to not include them right now as long as
> > > it's possible to add cli options later that add them.
> >
> > My main concern is just that `merge-tree` remain a low-level tool and
> > have machine-parseable output.  I was a little worried that both you
> > and Dscho wanted everything on stdout rather than in separate files,
> > as the <Informational messages> part of the output is rather
> > free-form.  But since it's at the end, and has a machine-parseable
> > beginning, it can just be slurped in and we're all good.  The diff
> > output raises my eyebrow because I'm worried we're losing this
> > property.  If there are clear usecases for adding more output, and we
> > can do so without losing this machine-parseable property, I don't have
> > a problem with adding an option for it.
>
> That's ok for me for now. I will certainly not work on adding options
> for diff output without any usecase.
>
> > One analogy we might use here is that `git merge` provides a diffstat
> > at the end.  What you're asking is more than a diffstat, but might be
> > considered similar-ish in nature.
> >
> > > The reason is
> > > that I think in many cases when there are conflicts, the conflicts
> > > will be small and the user will want to see them.
> >
> > I'm a little worried about the assumption here that conflict size is
> > measurable and visible via diffs.  That might be true in some cases,
> > but a UI written with that assumption is going to be very confusing
> > when hitting cases where that assumption does not hold.  For example:
> >
> >   * What if there is a binary file conflict, or a modify/delete or
> > rename/delete conflict, or failed-to-merge submodule conflict, or a
> > file location conflict? (For these, there is no diff relative to the
> > first parent and hence this conflict would have no diff output for
> > it)?
> >   * What if there was a simple file/directory conflict?  A diff would
> > show a rename (even when neither side did any renames), but not any
> > conflict markers.
> >   * What if there was a rename/rename conflict (both sides renamed
> > same file differently) or a distinct types conflict?  The former
> > results in three different conflicting files, none of them with
> > conflict markers, while the latter results in two different
> > conflicting files both without conflict markers?  Showing individual
> > per-file diffs totally loses all context here -- it'll show no-diff
> > for one of the files, and totally new additions for the ones.
>
> In those cases we just tell users that they cannot resolve those
> conflicts in the user interface, see the following doc:
>
> https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#conflicts-you-can-resolve-in-the-user-interface

So...I think you may have just convinced me that my fears were
justified and that I should probably NAK any attempt to add diffs to
the merge-tree command.  I won't jump to conclusions but you've
provided some pretty strong signal to me against going down that
route.  The list of limitations in the link you provide do mostly
avoid the broken cases I listed above, but it enshrines those
limitations on that webpage as fundamental rather than just as current
implementation shortcomings.  You may not be able to remove those
limitations on that webpage without either expunging the diffs from
the UI or exposing the brokenness of the various cases above.

If you do propose a diff option in the future, come prepared to
discuss how you'll avoid accidentally leading others down into paths
with the same fundamental issues, and/or how the above types of
conflicts might still be meaningfully handled.

Also, the list of limitations you have may not be quite comprehensive
enough to avoid all problems (though it certainly avoids most of
them).  Can I ask a couple clarifying questions about your list of
limitations in that link? :

  * When that page says the file cannot already contain conflict
markers, is the check performed on the version of the file in the two
trees being merged, or is the check performed on the 2nd and 3rd index
stage of the merge result (these are not equivalent checks, even if
they often give the same answer)?
  * When that page says the file must already exist in the same path
on both branches, is the check performed on by checking the path in
the two trees being merged, or is the check performed on the 2nd and
3rd index stage of the merge result (again, these are not equivalent
checks)?

> > Such a problem statement just seems fraught with edge cases to me, and
> > suggests that the problem statement might be in need of revisiting.
>
> Users understand that some kinds of conflicts cannot yet be resolved
> using a user interface. Maybe we will be able to make improvements so
> that more kinds of conflicts can be resolved in a UI in the future
> though. That's why a flexible and extensible output could help.

I think future improvements to handle more conflict types may well
hinge on removing the diff-output-using portion of your interface; I
think it may well be fundamentally incompatible.

I agree we want to leave the output format open for extension, I'm
just saying we have to be careful about what extensions are included
and this one worries me.

> > Don't read this as me closing the door on the possibility of diffs;
> > I'm not trying to do that.  I'm listing my misgivings about how I
> > think they might be used (i.e. be careful if you're headed down this
> > path as you might be digging yourself a never-ending support hole).
> > You can also think of my comments as feedback to consider and address
> > when you propose a future feature addition for adding diffs.  If/when
> > you propose such a feature, we'd probably be able to dive more into
> > specifics and usecases at that time, which may or may not circumvent
> > my concerns.
>
> I know that diffs, or any new single feature, will likely not be a
> silver bullet.

Sure that's fair; not being a silver bullet is fine.  We do need to
avoid providing a kryptonite bullet, though.

> > > So it would be
> > > simpler to just have an option to show any conflict right away, rather
> > > than have the user launch another command (a diff-tree against which
> > > tree and with which options?).
> >
> > Um, this part I'm not sure I get.  I thought the reason for the diffs
> > was performance -- you knew you wanted the diffs, and you wanted it
> > done as part of the same process.  But why would this be simpler?
>
> In the commit message of 4/12 you show an example of using it in simple scripts:
>
> NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> test $? -eq 0 || die "There were conflicts..."
> ...
>
> So I think it would be simpler for someone interested in seeing the
> conflicts, like a script writer, or maybe someone using it manually
> for example as a dry run before performing a merge, to be able to get
> them right away from the command...

Okay, but the command already does that.  When there are conflicts,
NEWTREE won't actually be a tree; it'll be lots of lines of output.
That's (part of) the reason for the exit status check.  So users
already get that information from the command.

> ...rather than to have to use another
> command (which means finding the right command, arguments and options
> for that) to get them.

As for finding the right arguments and options...

> > Your patch series included three different diffs, and the emails you
> > pointed me at suggested all kinds of configurability.  That suggests
> > the merge-tree command would have to take the exact same options the
> > user would supply to diff, and thus would have to be told all the same
> > options, right?  I don't see how this removes any complexity at all
> > for the user.
> >
> > Unless...is the request in some way similar to merge's diffstat where
> > there is always a very specific type of diff that is wanted and you
> > aren't envisioning much flexibility in what kind of diff or what to
> > diff against -- is that where the simplification comes from?
>
> Well I just think the default diff output could be tailored for the
> most likely usecases and options made available later for more
> advanced usecases or users.

Ah, so this may go against your earlier comments at [1] about a
merge-tree on steroids and a huge array of diff options, or against
your comments about diffs not being provided by default (also [1]).
Because if you have that huge range of diff options, and the diffs are
not provided by default, then it's not clear how you've simplified
things because users would still need to figure out the right
arguments and options to pass, it's just that the user would have to
pass all (or maybe just most?) of those arguments and options to
merge-tree instead of to diff.  Or is the simpler UI you're discussing
really just about not needing to include 1 argument, the name of the
new toplevel tree, since that single argument could be implicit?

I'm having a hard time buying the "simpler UI for script writers"
angle of argument here, especially for script writers who should fully
be able to look up the appropriate commands and use them.  Your
earlier arguments about performance being important (having both the
merge & the diff in the same process) seemed much more convincing to
me.  But maybe I'm still just missing something about your "simpler"
angle?

[1] https://lore.kernel.org/git/CAP8UFD0wKnAg5oyMWchXysPTg3K9Vb4M1tRcPzPE81QM903pYg@mail.gmail.com/

[PATCH v2 00/13] In-core git merge-tree ("Server side merges")

[Elijah Newren via GitGitGadget]

Note: Depends on en/remerge-diff

(Note2: This is a continuation of my series at [0], but I can't change the
base repository for a pull request in GitHub so I had to open a new PR and
that makes it look like a new series. [0]
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/)

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs.

== Updates Log ==

Stuff NOT included that reviewers brought up in the last round:

 * Very generic (mode, oid, stage, filename) printing formatting[1]
 * Always printing 3 stages for each filename with conflicts[2] [1]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [2]
   https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/

Updates since v3 (which looks like v1 due to the re-submit; thanks to René,
Ævar, Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
suggestions and comments on v2):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since v1 (thanks to Johannes Altmanninger and Fabian for suggestions
on v1):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-ort: allow update messages to be written to different file
    stream
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

Johannes Schindelin (1):
  diff: allow diff_warn_rename_limit to write somewhere besides stdout

 Documentation/git-merge-tree.txt | 179 ++++++++++++++++++++++++++++---
 builtin/merge-tree.c             | 162 +++++++++++++++++++++++++---
 diff.c                           |  20 ++--
 diff.h                           |   3 +-
 git.c                            |   2 +-
 merge-ort.c                      | 109 ++++++++++++-------
 merge-ort.h                      |  30 ++++++
 merge-recursive.c                |   3 +-
 t/t4301-merge-tree-write-tree.sh | 164 ++++++++++++++++++++++++++++
 9 files changed, 603 insertions(+), 69 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v1:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  65fdae9ddba !  3:  63f42df21ae merge-tree: add option parsing and initial shell for real merge function
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
       }
       
      +struct merge_tree_options {
     -+	int real;
     -+	int trivial;
     ++	int mode;
      +};
      +
      +static int real_merge(struct merge_tree_options *o,
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
      +		NULL
      +	};
      +	struct option mt_options[] = {
     -+		OPT_BOOL(0, "write-tree", &o.real,
     -+			 N_("do a real merge instead of a trivial merge")),
     -+		OPT_BOOL(0, "trivial-merge", &o.trivial,
     -+			 N_("do a trivial merge only")),
     ++		OPT_CMDMODE(0, "write-tree", &o.mode,
     ++			    N_("do a real merge instead of a trivial merge"),
     ++			    'w'),
     ++		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     ++			    N_("do a trivial merge only"), 't'),
      +		OPT_END()
      +	};
      +
     -+	/* Check for a request for basic help */
     -+	if (argc == 2 && !strcmp(argv[1], "-h"))
     -+		usage_with_options(merge_tree_usage, mt_options);
     -+
      +	/* Parse arguments */
      +	argc = parse_options(argc, argv, prefix, mt_options,
     -+			     merge_tree_usage, 0);
     -+	if (o.real && o.trivial)
     -+		die(_("--write-tree and --trivial-merge are incompatible"));
     -+	if (o.real || o.trivial) {
     -+		expected_remaining_argc = (o.real ? 2 : 3);
     ++			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     ++	if (o.mode) {
     ++		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
      +		if (argc != expected_remaining_argc)
      +			usage_with_options(merge_tree_usage, mt_options);
      +	} else {
      +		if (argc < 2 || argc > 3)
      +			usage_with_options(merge_tree_usage, mt_options);
     -+		o.real = (argc == 2);
     ++		o.mode = (argc == 2 ? 'w' : 't');
      +	}
      +
      +	/* Do the relevant type of merge */
     -+	if (o.real)
     ++	if (o.mode == 'w')
      +		return real_merge(&o, argv[0], argv[1]);
      +	else
      +		return trivial_merge(argv[0], argv[1], argv[2]);
  4:  05bd17686e1 !  4:  02c29f920d0 merge-tree: implement real merges
     @@ Commit message
      
          The only output is:
            - the toplevel resulting tree printed on stdout
     -      - exit status of 0 (clean) or 1 (conflicts present)
     +      - exit status of 0 (clean), 1 (conflicts present), anything else
     +        (merge could not be performed; unknown if clean or conflicted)
      
          This output is meant to be used by some higher level script, perhaps in
          a sequence of steps like this:
     @@ Commit message
          preliminary implementation, but subsequent commits will add that
          ability.
      
     +    This also marks the traditional trivial merge of merge-tree as
     +    deprecated.  The trivial merge not only had limited applicability, the
     +    output format was also difficult to work with (and its format
     +    undocumented), and will generally be less performant than real merges.
     +
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
       [verse]
      -'git merge-tree' <base-tree> <branch1> <branch2>
      +'git merge-tree' [--write-tree] <branch1> <branch2>
     -+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
     ++'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
       DESCRIPTION
       -----------
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +Performs a merge, but does not make any new commits and does not read
      +from or write to either the working tree or index.
      +
     -+The first form will merge the two branches, doing a full recursive
     -+merge with rename detection.  The rest of this manual (other than the
     -+next paragraph) describes the first form in more detail -- including
     -+options, output format, exit status, and usage notes.
     -+
     -+The second form is deprecated; it is kept for backward compatibility
     -+reasons but may be deleted in the future.  It will only do a trivial
     -+merge.  It reads three tree-ish, and outputs trivial merge results and
     -+conflicting stages to the standard output in a semi-diff format.
     -+Since this was designed for higher level scripts to consume and merge
     -+the results back into the index, it omits entries that match
     -+<branch1>.  The result of this second form is is similar to what
     -+three-way 'git read-tree -m' does, but instead of storing the results
     -+in the index, the command outputs the entries to the standard output.
     -+This form not only has limited applicability, the output format is
     -+also difficult to work with, and it will generally be less performant
     -+than the first form even on successful merges (especially if working
     -+in large repositories).  The remainder of this manual will only
     -+discuss the first form.
     ++The second form is deprecated and supported only for backward
     ++compatibility.  It will likely be removed in the future, and will not
     ++be discussed further in this manual.
     ++
     ++The first form will merge the two branches, doing a real merge.  A real
     ++merge is distinguished from a trivial merge in that it includes:
     ++
     ++  * three way content merges of individual files
     ++  * rename detection
     ++  * proper directory/file conflict handling
     ++  * recursive ancestor consolidation (i.e. when there is more than one
     ++    merge base, creating a virtual merge base by merging the merge bases)
     ++  * etc.
     ++
     ++After the merge completes, it will create a new toplevel tree object.
     ++See `OUTPUT` below for details.
      +
      +OUTPUT
      +------
     @@ builtin/merge-tree.c: struct merge_tree_options {
      +
      +	parent1 = get_merge_parent(branch1);
      +	if (!parent1)
     -+		help_unknown_ref(branch1, "merge",
     ++		help_unknown_ref(branch1, "merge-tree",
      +				 _("not something we can merge"));
      +
      +	parent2 = get_merge_parent(branch2);
      +	if (!parent2)
     -+		help_unknown_ref(branch2, "merge",
     ++		help_unknown_ref(branch2, "merge-tree",
      +				 _("not something we can merge"));
      +
      +	init_merge_options(&opt, the_repository);
     -+	/*
     -+	 * TODO: Support subtree and other -X options?
     -+	if (use_strategies_nr == 1 &&
     -+	    !strcmp(use_strategies[0]->name, "subtree"))
     -+		opt.subtree_shift = "";
     -+	for (x = 0; x < xopts_nr; x++)
     -+		if (parse_merge_opt(&opt, xopts[x]))
     -+			die(_("Unknown strategy option: -X%s"), xopts[x]);
     -+	*/
      +
      +	opt.show_rename_progress = 0;
      +
     -+	opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
     -+	opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */
     ++	opt.branch1 = branch1;
     ++	opt.branch2 = branch2;
      +
      +	/*
      +	 * Get the merge bases, in reverse order; see comment above
     @@ builtin/merge-tree.c: struct merge_tree_options {
      +		commit_list_insert(j->item, &merge_bases);
      +
      +	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
     -+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
      +	if (result.clean < 0)
      +		die(_("failure to merge"));
     -+	else if (!result.clean)
     -+		printf(_("Conflicts!\n"));
     ++	puts(oid_to_hex(&result.tree->object.oid));
      +	merge_finalize(&opt, &result);
      +	return !result.clean; /* result.clean < 0 handled above */
       }
       
       int cmd_merge_tree(int argc, const char **argv, const char *prefix)
      
     - ## t/t4301-merge-tree-real.sh (new) ##
     + ## t/t4301-merge-tree-write-tree.sh (new) ##
      @@
      +#!/bin/sh
      +
     @@ t/t4301-merge-tree-real.sh (new)
      +. ./test-lib.sh
      +
      +# This test is ort-specific
     -+test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {
     ++if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
     ++then
      +	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
      +	test_done
     -+}
     ++fi
      +
      +test_expect_success setup '
      +	test_write_lines 1 2 3 4 5 >numbers &&
  -:  ----------- >  5:  6fb4f4580a5 diff: allow diff_warn_rename_limit to write somewhere besides stdout
  5:  095aa266c2b !  6:  28368c03898 merge-ort: split out a separate display_update_messages() function
     @@ Metadata
       ## Commit message ##
          merge-ort: split out a separate display_update_messages() function
      
     -    No functional changes included in this patch; it's just a preparatory
     -    step to allow the printed messages to be handled differently by other
     -    callers, such as in `git merge-tree --write-tree`.
     +    This patch includes no new code; it simply moves a bunch of lines into a
     +    new function.  As such, there are no functional changes.  This is just a
     +    preparatory step to allow the printed messages to be handled differently
     +    by other callers, such as in `git merge-tree --write-tree`.
     +
     +    (Patch best viewed with
     +         --color-moved --color-moved-ws=allow-indentation-change
     +     to see that it is a simple code movement.)
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
      +
      +	/* Also include needed rename limit adjustment now */
      +	diff_warn_rename_limit("merge.renamelimit",
     -+			       opti->renames.needed_limit, 0);
     ++			       opti->renames.needed_limit, 0, stdout);
      +
      +	trace2_region_leave("merge", "display messages", opt->repo);
      +}
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
      -
      -		/* Also include needed rename limit adjustment now */
      -		diff_warn_rename_limit("merge.renamelimit",
     --				       opti->renames.needed_limit, 0);
     +-				       opti->renames.needed_limit, 0, stdout);
      -
      -		trace2_region_leave("merge", "display messages", opt->repo);
      -	}
  6:  e3ef17eb46f !  7:  593d0c00b57 merge-ort: allow update messages to be written to different file stream
     @@ merge-ort.c: void merge_display_update_messages(struct merge_options *opt,
       		struct strbuf *sb = olist.items[i].util;
       
      -		printf("%s", sb->buf);
     -+		fprintf(stream, "%s", sb->buf);
     ++		strbuf_write(sb, stream);
       	}
       	string_list_clear(&olist, 0);
       
     + 	/* Also include needed rename limit adjustment now */
     + 	diff_warn_rename_limit("merge.renamelimit",
     +-			       opti->renames.needed_limit, 0, stdout);
     ++			       opti->renames.needed_limit, 0, stream);
     + 
     + 	trace2_region_leave("merge", "display messages", opt->repo);
     + }
      @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
       	}
       
  7:  2f296aeeefb !  8:  d0d30e92ecd merge-tree: support including merge messages in output
     @@ Commit message
          When running `git merge-tree --write-tree`, we previously would only
          return an exit status reflecting the cleanness of a merge, and print out
          the toplevel tree of the resulting merge.  Merges also have
     -    informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
     -    ...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
     -    conflicts occur (such as file/directory, modify/delete, add/add with
     -    differing modes, rename/rename (1to2), etc.), these informational
     -    messages are often the only notification since these conflicts are not
     -    representable in the contents of the file.
     +    informational messages, such as:
     +      * "Auto-merging <PATH>"
     +      * "CONFLICT (content): ..."
     +      * "CONFLICT (file/directory)"
     +      * etc.
     +    In fact, when non-content conflicts occur (such as file/directory,
     +    modify/delete, add/add with differing modes, rename/rename (1to2),
     +    etc.), these informational messages may be the only notification the
     +    user gets since these conflicts are not representable in the contents
     +    of the file.
      
          Add a --[no-]messages option so that callers can request these messages
          be included at the end of the output.  Include such messages by default
     @@ Documentation/git-merge-tree.txt: git-merge-tree - Perform merge without touchin
       [verse]
      -'git merge-tree' [--write-tree] <branch1> <branch2>
      +'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
     - 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
     + 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
       DESCRIPTION
     -@@ Documentation/git-merge-tree.txt: than the first form even on successful merges (especially if working
     - in large repositories).  The remainder of this manual will only
     - discuss the first form.
     +@@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in that it includes:
     + After the merge completes, it will create a new toplevel tree object.
     + See `OUTPUT` below for details.
       
      +OPTIONS
      +-------
     @@ Documentation/git-merge-tree.txt: be used as a part of a series of steps such as
      
       ## builtin/merge-tree.c ##
      @@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     + 
       struct merge_tree_options {
     - 	int real;
     - 	int trivial;
     + 	int mode;
      +	int show_messages;
       };
       
       static int real_merge(struct merge_tree_options *o,
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
     - 		commit_list_insert(j->item, &merge_bases);
     - 
       	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
     --	printf("%s\n", oid_to_hex(&result.tree->object.oid));
     -+
       	if (result.clean < 0)
       		die(_("failure to merge"));
     --	else if (!result.clean)
     --		printf(_("Conflicts!\n"));
      +
      +	if (o->show_messages == -1)
      +		o->show_messages = !result.clean;
      +
     -+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
     + 	puts(oid_to_hex(&result.tree->object.oid));
      +	if (o->show_messages) {
      +		printf("\n");
      +		merge_display_update_messages(&opt, &result, stdout);
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		NULL
       	};
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			 N_("do a real merge instead of a trivial merge")),
     - 		OPT_BOOL(0, "trivial-merge", &o.trivial,
     - 			 N_("do a trivial merge only")),
     + 			    'w'),
     + 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     + 			    N_("do a trivial merge only"), 't'),
      +		OPT_BOOL(0, "messages", &o.show_messages,
      +			 N_("also show informational/conflict messages")),
       		OPT_END()
       	};
       
     -@@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 		usage_with_options(merge_tree_usage, mt_options);
     - 
       	/* Parse arguments */
      +	original_argc = argc;
       	argc = parse_options(argc, argv, prefix, mt_options,
     - 			     merge_tree_usage, 0);
     - 	if (o.real && o.trivial)
     - 		die(_("--write-tree and --trivial-merge are incompatible"));
     -+	if (!o.real && original_argc < argc)
     -+		die(_("--write-tree must be specified if any other options are"));
     - 	if (o.real || o.trivial) {
     - 		expected_remaining_argc = (o.real ? 2 : 3);
     - 		if (argc != expected_remaining_argc)
     + 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     + 	if (o.mode) {
     +@@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     + 			usage_with_options(merge_tree_usage, mt_options);
     + 		o.mode = (argc == 2 ? 'w' : 't');
     + 	}
     ++	if (o.mode == 't' && original_argc < argc)
     ++		die(_("--trivial-merge is incompatible with all other options"));
     + 
     + 	/* Do the relevant type of merge */
     + 	if (o.mode == 'w')
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Barf on too many arguments' '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
       	grep "^usage: git merge-tree" expect
       '
       
  8:  35e0ed9271a !  9:  9c2334ae9f2 merge-ort: provide a merge_get_conflicted_files() helper function
     @@ Commit message
          merge-ort: provide a merge_get_conflicted_files() helper function
      
          After a merge, this function allows the user to extract the same
     -    information that would be printed by `ls-files -u` -- conflicted
     +    information that would be printed by `ls-files -u`, which means
          files with their mode, oid, and stage.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
  9:  fcbb087fa88 ! 10:  243134dc247 merge-tree: provide a list of which files have conflicts
     @@ builtin/merge-tree.c: struct merge_tree_options {
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		o->show_messages = !result.clean;
       
     - 	printf("%s\n", oid_to_hex(&result.tree->object.oid));
     + 	puts(oid_to_hex(&result.tree->object.oid));
      +	if (!result.clean) {
      +		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
      +		const char *last = NULL;
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
       
       	/* Do the relevant type of merge */
     - 	if (o.real)
     + 	if (o.mode == 'w')
      -		return real_merge(&o, argv[0], argv[1]);
      +		return real_merge(&o, argv[0], argv[1], prefix);
       	else
       		return trivial_merge(argv[0], argv[1], argv[2]);
       }
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such' '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
       	#   "whatever" has *both* a modify/delete and a file/directory conflict
       	cat <<-EOF >expect &&
       	HASH
     @@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such'
       
       	Auto-merging greeting
       	CONFLICT (content): Merge conflict in greeting
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
       	test_cmp expect actual
       '
       
 10:  050add3e498 ! 11:  c322e4c6938 merge-tree: provide easy access to `ls-files -u` style info
     @@ Commit message
          Much like `git merge` updates the index with information of the form
              (mode, oid, stage, name)
          provide this output for conflicted files for merge-tree as well.
     -    Provide an --exclude-oids-and-modes option for users to exclude the
     -    mode, oid, and stage and only get the list of conflicted filenames.
     +    Provide an --exclude-modes-oids-stages/-l option for users to exclude
     +    the mode, oid, and stage and only get the list of conflicted filenames.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: discuss the first form.
     +@@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
       OPTIONS
       -------
       
     @@ Documentation/git-merge-tree.txt: plumbing commands since the possibility of mer
       Part of the linkgit:git[1] suite
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: struct merge_tree_options {
     - 	int real;
     - 	int trivial;
     +@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     + struct merge_tree_options {
     + 	int mode;
       	int show_messages;
     -+	int exclude_oids_and_modes;
     ++	int exclude_modes_oids_stages;
       };
       
       static int real_merge(struct merge_tree_options *o,
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			const char *name = conflicted_files.items[i].string;
      -			if (last && !strcmp(last, name))
      +			struct stage_info *c = conflicted_files.items[i].util;
     -+			if (!o->exclude_oids_and_modes)
     ++			if (!o->exclude_modes_oids_stages)
      +				printf("%06o %s %d\t",
      +				       c->mode, oid_to_hex(&c->oid), c->stage);
      +			else if (last && !strcmp(last, name))
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			write_name_quoted_relative(
       				name, prefix, stdout, line_termination);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			 N_("do a trivial merge only")),
     + 			    N_("do a trivial merge only"), 't'),
       		OPT_BOOL(0, "messages", &o.show_messages,
       			 N_("also show informational/conflict messages")),
     -+		OPT_BOOL_F(0, "exclude-oids-and-modes",
     -+			   &o.exclude_oids_and_modes,
     -+			   N_("list conflicted files without oids and modes"),
     ++		OPT_BOOL_F('l', "exclude-modes-oids-stages",
     ++			   &o.exclude_modes_oids_stages,
     ++			   N_("list conflicted files without modes/oids/stages"),
      +			   PARSE_OPT_NONEG),
       		OPT_END()
       	};
       
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Content merge and a few conflicts' '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Content merge and a few conflicts' '
       	expected_tree=$(cat .git/AUTO_MERGE) &&
       
       	# We will redo the merge, while we are still in a conflicted state!
     @@ t/t4301-merge-tree-real.sh: test_expect_success 'Content merge and a few conflic
       	test_when_finished "git reset --hard" &&
       
       	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Barf on too many arguments' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
       '
       
       test_expect_success 'test conflict notices and such' '
      -	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --exclude-oids-and-modes side1 side2 >out &&
     ++	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
       	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
       
       	# Expected results:
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
       '
       
       test_expect_success 'Just the conflicted files without the messages' '
      -	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-oids-and-modes side1 side2 >out &&
     ++	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
       	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
       
       	test_write_lines HASH greeting whatever~side1 >expect &&
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Just the conflicted files without the messages' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Just the conflicted files without the messages' '
       	test_cmp expect actual
       '
       
 11:  ba8a50f03cb ! 12:  25677d5038c merge-tree: add a --allow-unrelated-histories flag
     @@ Documentation/git-merge-tree.txt: OPTIONS
      
       ## builtin/merge-tree.c ##
      @@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     + 
       struct merge_tree_options {
     - 	int real;
     - 	int trivial;
     + 	int mode;
      +	int allow_unrelated_histories;
       	int show_messages;
     - 	int exclude_oids_and_modes;
     + 	int exclude_modes_oids_stages;
       };
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	 * merge_incore_recursive in merge-ort.h
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	for (j = common; j; j = j->next)
       		commit_list_insert(j->item, &merge_bases);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			   &o.exclude_oids_and_modes,
     - 			   N_("list conflicted files without oids and modes"),
     + 			   &o.exclude_modes_oids_stages,
     + 			   N_("list conflicted files without modes/oids/stages"),
       			   PARSE_OPT_NONEG),
      +		OPT_BOOL_F(0, "allow-unrelated-histories",
      +			   &o.allow_unrelated_histories,
     @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char
       	};
       
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success setup '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
       	>whatever/empty &&
       	git add numbers greeting whatever/empty &&
       	test_tick &&
     @@ t/t4301-merge-tree-real.sh: test_expect_success setup '
       '
       
       test_expect_success 'Content merge and a few conflicts' '
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Check conflicted oids and modes without messages' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and modes without messages' '
       	test_cmp conflicted-file-info actual
       '
       
 12:  4123209cafc = 13:  e7c63425a0e git-merge-tree.txt: add a section on potentional usage mistakes

-- 
gitgitgadget

[PATCH v2 01/13] merge-tree: rename merge_trees() to trivial_merge_trees()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget

[PATCH v2 02/13] merge-tree: move logic for existing merge into new function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget

[PATCH v2 03/13] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 61 +++++++++++++++++++++++++++++++++++++-------
 git.c                |  2 +-
 2 files changed, 53 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..e98ec8a9f1d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,51 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    'w'),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), 't'),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	if (o.mode) {
+		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
+		if (argc != expected_remaining_argc)
+			usage_with_options(merge_tree_usage, mt_options);
+	} else {
+		if (argc < 2 || argc > 3)
+			usage_with_options(merge_tree_usage, mt_options);
+		o.mode = (argc == 2 ? 'w' : 't');
+	}
+
+	/* Do the relevant type of merge */
+	if (o.mode == 'w')
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget

[PATCH v2 04/13] merge-tree: implement real merges

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability.

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 71 +++++++++++++++++++++-----
 builtin/merge-tree.c             | 44 +++++++++++++++-
 t/t4301-merge-tree-write-tree.sh | 88 ++++++++++++++++++++++++++++++++
 3 files changed, 190 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..569485815a0 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,73 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The second form is deprecated and supported only for backward
+compatibility.  It will likely be removed in the future, and will not
+be discussed further in this manual.
+
+The first form will merge the two branches, doing a real merge.  A real
+merge is distinguished from a trivial merge in that it includes:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, it will create a new toplevel tree object.
+See `OUTPUT` below for details.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1.
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
+be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+However, it does not quite fit into the same category of low-level
+plumbing commands since the possibility of merge conflicts give it a
+much higher chance of the command not succeeding.
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index e98ec8a9f1d..d14c9f6e44e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -392,7 +395,46 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..66c3eaf2021
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,88 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(cat .git/AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	cat tmp | sed -e s/HEAD/side1/ >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget

[PATCH v2 05/13] diff: allow diff_warn_rename_limit to write somewhere besides stdout

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <Johannes.Schindelin@gmx.de>

diff_warn_rename_limit() is hardcoded to write to stdout.  Make it
accept an output location parameter to make it more flexible.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 diff.c            | 20 ++++++++++++++------
 diff.h            |  3 ++-
 merge-ort.c       |  2 +-
 merge-recursive.c |  3 ++-
 4 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/diff.c b/diff.c
index 1bfb01c18ec..6952035046f 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,17 +6377,25 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out)
 {
-	fflush(stdout);
+	const char *fmt = NULL;
+
 	if (degraded_cc)
-		warning(_(degrade_cc_to_c_warning));
+		fmt = _(degrade_cc_to_c_warning);
 	else if (needed)
-		warning(_(rename_limit_warning));
+		fmt = _(rename_limit_warning);
 	else
 		return;
 	if (0 < needed)
-		warning(_(rename_limit_advice), varname, needed);
+		fmt = _(rename_limit_advice);
+
+	fflush(out);
+	if (out == stdout)
+		warning(fmt, varname, needed);
+	else
+		fprintf(out, fmt, varname, needed);
 }
 
 static void create_filepairs_for_header_only_notifications(struct diff_options *o)
@@ -6870,7 +6878,7 @@ int diff_result_code(struct diff_options *opt, int status)
 
 	diff_warn_rename_limit("diff.renameLimit",
 			       opt->needed_rename_limit,
-			       opt->degraded_cc_to_c);
+			       opt->degraded_cc_to_c, stdout);
 	if (!opt->flags.exit_with_status &&
 	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
 		return status;
diff --git a/diff.h b/diff.h
index ce9e2cf2e4f..40c5b78fb0a 100644
--- a/diff.h
+++ b/diff.h
@@ -597,7 +597,8 @@ void diffcore_fix_diff_index(void);
 int diff_queue_is_empty(struct diff_options *o);
 void diff_flush(struct diff_options*);
 void diff_free(struct diff_options*);
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out);
 
 /* diff-raw status letters */
 #define DIFF_STATUS_ADDED		'A'
diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..65618048b59 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4305,7 +4305,7 @@ void merge_switch_to_result(struct merge_options *opt,
 
 		/* Also include needed rename limit adjustment now */
 		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
+				       opti->renames.needed_limit, 0, stdout);
 
 		trace2_region_leave("merge", "display messages", opt->repo);
 	}
diff --git a/merge-recursive.c b/merge-recursive.c
index 9ec1e6d043a..d2eeca9fa20 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3738,7 +3738,8 @@ static void merge_finalize(struct merge_options *opt)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->priv->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0,
+				       stdout);
 	FREE_AND_NULL(opt->priv);
 }
 
-- 
gitgitgadget

[PATCH v2 06/13] merge-ort: split out a separate display_update_messages() function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 77 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 36 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 65618048b59..1ada3198390 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0, stdout);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4273,42 +4312,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
 
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0, stdout);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v2 07/13] merge-ort: allow update messages to be written to different file stream

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This modifies the new display_update_messages() function to allow
printing to somewhere other than stdout.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 9 +++++----
 merge-ort.h | 3 ++-
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 1ada3198390..d28d1721d14 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 }
 
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result)
+				   struct merge_result *result,
+				   FILE *stream)
 {
 	struct merge_options_internal *opti = result->priv;
 	struct hashmap_iter iter;
@@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
 	for (i = 0; i < olist.nr; ++i) {
 		struct strbuf *sb = olist.items[i].util;
 
-		printf("%s", sb->buf);
+		strbuf_write(sb, stream);
 	}
 	string_list_clear(&olist, 0);
 
 	/* Also include needed rename limit adjustment now */
 	diff_warn_rename_limit("merge.renamelimit",
-			       opti->renames.needed_limit, 0, stdout);
+			       opti->renames.needed_limit, 0, stream);
 
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
@@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
 	}
 
 	if (display_update_msgs)
-		merge_display_update_messages(opt, result);
+		merge_display_update_messages(opt, result, stdout);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..d643b47cb7c 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
  * so only call this when bypassing merge_switch_to_result().
  */
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result);
+				   struct merge_result *result,
+				   FILE *stream);
 
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
-- 
gitgitgadget

[PATCH v2 08/13] merge-tree: support including merge messages in output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 19 ++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 21 +++++++++++++++
 3 files changed, 76 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 569485815a0..42e0f8f6183 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
@@ -35,17 +35,47 @@ merge is distinguished from a trivial merge in that it includes:
 After the merge completes, it will create a new toplevel tree object.
 See `OUTPUT` below for details.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
+
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -69,7 +99,8 @@ be used as a part of a series of steps such as
 
 However, it does not quite fit into the same category of low-level
 plumbing commands since the possibility of merge conflicts give it a
-much higher chance of the command not succeeding.
+much higher chance of the command not succeeding (and NEWTREE containing
+a bunch of stuff other than just a toplevel tree).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d14c9f6e44e..6a556ab1c9c 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -390,6 +390,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -432,18 +433,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result, stdout);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -453,10 +463,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    'w'),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), 't'),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	if (o.mode) {
@@ -468,6 +481,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			usage_with_options(merge_tree_usage, mt_options);
 		o.mode = (argc == 2 ? 'w' : 't');
 	}
+	if (o.mode == 't' && original_argc < argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 66c3eaf2021..e2255711f9c 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -85,4 +85,25 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v2 09/13] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index d28d1721d14..c4ce6027dc4 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index d643b47cb7c..e635a294ea8 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result,
 				   FILE *stream);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v2 10/13] merge-tree: provide a list of which files have conflicts

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 42e0f8f6183..160e8f44b62 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -55,6 +55,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -66,6 +67,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 6a556ab1c9c..54dae018203 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -394,7 +397,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -438,6 +442,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result, stdout);
@@ -486,7 +506,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index e2255711f9c..7113d060bc5 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -95,6 +95,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -106,4 +108,13 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide an --exclude-modes-oids-stages/-l option for users to exclude
the mode, oid, and stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 30 ++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++++--
 3 files changed, 58 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 160e8f44b62..55bb7bc61c1 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -38,6 +38,11 @@ See `OUTPUT` below for details.
 OPTIONS
 -------
 
+--exclude-oids-and-modes::
+	Instead of writing a list of (mode, oid, stage, path) tuples
+	to output for conflicted files, just provide a list of
+	filenames with conflicts.
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -55,7 +60,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -67,18 +72,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--exclude-oids-and-modes` option is passed, the mode, object, and
+stage will be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -110,6 +120,14 @@ plumbing commands since the possibility of merge conflicts give it a
 much higher chance of the command not succeeding (and NEWTREE containing
 a bunch of stuff other than just a toplevel tree).
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 54dae018203..dc52cd02dce 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -394,6 +394,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int exclude_modes_oids_stages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->exclude_modes_oids_stages)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), 't'),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F('l', "exclude-modes-oids-stages",
+			   &o.exclude_modes_oids_stages,
+			   N_("list conflicted files without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 7113d060bc5..1572f460da0 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -47,6 +47,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(cat .git/AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -86,7 +87,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -109,7 +110,7 @@ test_expect_success 'test conflict notices and such' '
 '
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -117,4 +118,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v2 12/13] merge-tree: add a --allow-unrelated-histories flag

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 55bb7bc61c1..d35710c81d5 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -49,6 +49,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index dc52cd02dce..cca5075d521 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -393,6 +393,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int exclude_modes_oids_stages;
 };
@@ -430,7 +431,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -494,6 +495,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.exclude_modes_oids_stages,
 			   N_("list conflicted files without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 1572f460da0..996bdfaab7d 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -38,7 +38,13 @@ test_expect_success setup '
 	>whatever/empty &&
 	git add numbers greeting whatever/empty &&
 	test_tick &&
-	git commit -m other-modifications
+	git commit -m other-modifications &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Content merge and a few conflicts' '
@@ -139,4 +145,20 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v2 13/13] git-merge-tree.txt: add a section on potentional usage mistakes

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index d35710c81d5..b97ddc58a7a 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -133,6 +133,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")

[Elijah Newren]

On Sat, Jan 29, 2022 at 9:43 AM Elijah Newren <newren@gmail.com> wrote:
>
> Hi Christian,
>
> On Sat, Jan 29, 2022 at 12:18 AM Christian Couder
> <christian.couder@gmail.com> wrote:
> >
> > On Sat, Jan 29, 2022 at 8:04 AM Elijah Newren <newren@gmail.com> wrote:
> > >
> > > On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
> > > <christian.couder@gmail.com> wrote:
[...]
> > > > The reason is
> > > > that I think in many cases when there are conflicts, the conflicts
> > > > will be small and the user will want to see them.
> > >
> > > I'm a little worried about the assumption here that conflict size is
> > > measurable and visible via diffs.  That might be true in some cases,
> > > but a UI written with that assumption is going to be very confusing
> > > when hitting cases where that assumption does not hold.  For example:
> > >
> > >   * What if there is a binary file conflict, or a modify/delete or
> > > rename/delete conflict, or failed-to-merge submodule conflict, or a
> > > file location conflict? (For these, there is no diff relative to the
> > > first parent and hence this conflict would have no diff output for
> > > it)?
> > >   * What if there was a simple file/directory conflict?  A diff would
> > > show a rename (even when neither side did any renames), but not any
> > > conflict markers.
> > >   * What if there was a rename/rename conflict (both sides renamed
> > > same file differently) or a distinct types conflict?  The former
> > > results in three different conflicting files, none of them with
> > > conflict markers, while the latter results in two different
> > > conflicting files both without conflict markers?  Showing individual
> > > per-file diffs totally loses all context here -- it'll show no-diff
> > > for one of the files, and totally new additions for the ones.
> >
> > In those cases we just tell users that they cannot resolve those
> > conflicts in the user interface, see the following doc:
> >
> > https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#conflicts-you-can-resolve-in-the-user-interface
>
> So...I think you may have just convinced me that my fears were
> justified and that I should probably NAK any attempt to add diffs to
> the merge-tree command.  I won't jump to conclusions but you've
> provided some pretty strong signal to me against going down that
> route.  The list of limitations in the link you provide do mostly
> avoid the broken cases I listed above, but it enshrines those
> limitations on that webpage as fundamental rather than just as current
> implementation shortcomings.  You may not be able to remove those
> limitations on that webpage without either expunging the diffs from
> the UI or exposing the brokenness of the various cases above.
>
> If you do propose a diff option in the future, come prepared to
> discuss how you'll avoid accidentally leading others down into paths
> with the same fundamental issues, and/or how the above types of
> conflicts might still be meaningfully handled.

Actually, after having a few extra days to think about it, I thought
of something that should have been obvious to me, given my other
in-flight series that this depends upon...

If you used the same trick that remerge-diff does to include the
CONFLICT (and related messages) headers in the diff, then the kinds of
conflicts that are normally either invisible or misleading/confusing
to show via a diff would suddenly have the extra notices needed to
explain them, and make this problem tractable.

Further, it'd only make sense to do the special diff as part of the
merge-tree process, since it has the conflict messages strmap needed
to do this.

And there's not all that much work that would be needed to take
advantage of this, especially since this series already depends upon
the remerge-diff series.

So, maybe this is fine after all.

> Also, the list of limitations you have may not be quite comprehensive
> enough to avoid all problems (though it certainly avoids most of
> them).  Can I ask a couple clarifying questions about your list of
> limitations in that link? :
>
>   * When that page says the file cannot already contain conflict
> markers, is the check performed on the version of the file in the two
> trees being merged, or is the check performed on the 2nd and 3rd index
> stage of the merge result (these are not equivalent checks, even if
> they often give the same answer)?
>   * When that page says the file must already exist in the same path
> on both branches, is the check performed on by checking the path in
> the two trees being merged, or is the check performed on the 2nd and
> 3rd index stage of the merge result (again, these are not equivalent
> checks)?

I am still curious about this either way.

[PATCH v3 00/15] In-core git merge-tree ("Server side merges")

[Elijah Newren via GitGitGadget]

Note: Depends on en/remerge-diff (to avoid a small textual conflict). Still
under heavy discussion, though, so no need to pick up yet.

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs.

== Quick Overview ==

 * Patches 1-2: preparatory cleanups
 * Patches 3-4: implement basic real merges
 * Patches 5-9: include informational messages ("CONFLICT" messages and
   such) in output
   * to be honest, patches 5, 6, & 8 may be less relevant since we're now
     including these messages on stdout anyway...
 * Patches 10-13: add ability to include ls-files -u style of info in the
   output
 * Patch 14: support --allow-unrelated-histories
 * Patch 15: augment the manual with potential usage mistakes

== Updates Log ==

Updates since v2 (or v4, if you include the rounds at
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):

 * Improved patches from Dscho for the diff_warn_rename_limit() handling
 * Add a -z option for NUL-terminated conflict info lines (so that filenames
   do not have to be quoted)

Stuff NOT included that reviewers brought up in earlier rounds:

 * Very generic (mode, oid, stage, filename) printing formatting[1]
 * Always printing 3 stages for each filename with conflicts[2] [1]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [2]
   https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/

Updates since v1 (or v3 depending on how you count; thanks to René, Ævar,
Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

Updates since original submission v2 (thanks to Christian, Dscho, Ramsay,
and René for suggestions and comments):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since original submission v1 (thanks to Johannes Altmanninger and
Fabian for suggestions):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (13):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-ort: allow update messages to be written to different file
    stream
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

Johannes Schindelin (2):
  Introduce a variant of the `warning()` function that takes a `FILE *`
  diff: allow diff_warn_rename_limit to write somewhere besides stderr

 Documentation/git-merge-tree.txt | 192 +++++++++++++++++++++++++++--
 builtin/merge-tree.c             | 164 +++++++++++++++++++++++--
 diff.c                           |  13 +-
 diff.h                           |   3 +-
 git-compat-util.h                |   1 +
 git.c                            |   2 +-
 merge-ort.c                      | 109 +++++++++++------
 merge-ort.h                      |  30 +++++
 merge-recursive.c                |   3 +-
 t/t4301-merge-tree-write-tree.sh | 204 +++++++++++++++++++++++++++++++
 usage.c                          |  14 +++
 11 files changed, 667 insertions(+), 68 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v2:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  63f42df21ae =  3:  63f42df21ae merge-tree: add option parsing and initial shell for real merge function
  4:  02c29f920d0 =  4:  02c29f920d0 merge-tree: implement real merges
  -:  ----------- >  5:  290b42846b5 Introduce a variant of the `warning()` function that takes a `FILE *`
  5:  6fb4f4580a5 !  6:  2083fbe9b2e diff: allow diff_warn_rename_limit to write somewhere besides stdout
     @@ Metadata
      Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
      
       ## Commit message ##
     -    diff: allow diff_warn_rename_limit to write somewhere besides stdout
     +    diff: allow diff_warn_rename_limit to write somewhere besides stderr
      
     -    diff_warn_rename_limit() is hardcoded to write to stdout.  Make it
     -    accept an output location parameter to make it more flexible.
     +    diff_warn_rename_limit() is hardcoded to write to stderr.  Make it
     +    accept a file stream parameter to make it more flexible.
      
          Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
          Signed-off-by: Elijah Newren <newren@gmail.com>
     @@ diff.c: static const char rename_limit_advice[] =
      +void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
      +			    FILE *out)
       {
     --	fflush(stdout);
     -+	const char *fmt = NULL;
     -+
     + 	fflush(stdout);
       	if (degraded_cc)
      -		warning(_(degrade_cc_to_c_warning));
     -+		fmt = _(degrade_cc_to_c_warning);
     ++		warning_fp(out, _(degrade_cc_to_c_warning));
       	else if (needed)
      -		warning(_(rename_limit_warning));
     -+		fmt = _(rename_limit_warning);
     ++		warning_fp(out, _(rename_limit_warning));
       	else
       		return;
     ++
     ++
       	if (0 < needed)
      -		warning(_(rename_limit_advice), varname, needed);
     -+		fmt = _(rename_limit_advice);
     -+
     -+	fflush(out);
     -+	if (out == stdout)
     -+		warning(fmt, varname, needed);
     -+	else
     -+		fprintf(out, fmt, varname, needed);
     ++		warning_fp(out, _(rename_limit_advice), varname, needed);
       }
       
       static void create_filepairs_for_header_only_notifications(struct diff_options *o)
     @@ diff.c: int diff_result_code(struct diff_options *opt, int status)
       	diff_warn_rename_limit("diff.renameLimit",
       			       opt->needed_rename_limit,
      -			       opt->degraded_cc_to_c);
     -+			       opt->degraded_cc_to_c, stdout);
     ++			       opt->degraded_cc_to_c, stderr);
       	if (!opt->flags.exit_with_status &&
       	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
       		return status;
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
       		/* Also include needed rename limit adjustment now */
       		diff_warn_rename_limit("merge.renamelimit",
      -				       opti->renames.needed_limit, 0);
     -+				       opti->renames.needed_limit, 0, stdout);
     ++				       opti->renames.needed_limit, 0, stderr);
       
       		trace2_region_leave("merge", "display messages", opt->repo);
       	}
     @@ merge-recursive.c: static void merge_finalize(struct merge_options *opt)
       		diff_warn_rename_limit("merge.renamelimit",
      -				       opt->priv->needed_rename_limit, 0);
      +				       opt->priv->needed_rename_limit, 0,
     -+				       stdout);
     ++				       stderr);
       	FREE_AND_NULL(opt->priv);
       }
       
  6:  28368c03898 !  7:  1be858e6aa6 merge-ort: split out a separate display_update_messages() function
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
      +
      +	/* Also include needed rename limit adjustment now */
      +	diff_warn_rename_limit("merge.renamelimit",
     -+			       opti->renames.needed_limit, 0, stdout);
     ++			       opti->renames.needed_limit, 0, stderr);
      +
      +	trace2_region_leave("merge", "display messages", opt->repo);
      +}
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
      -
      -		/* Also include needed rename limit adjustment now */
      -		diff_warn_rename_limit("merge.renamelimit",
     --				       opti->renames.needed_limit, 0, stdout);
     +-				       opti->renames.needed_limit, 0, stderr);
      -
      -		trace2_region_leave("merge", "display messages", opt->repo);
      -	}
  7:  593d0c00b57 !  8:  04c3bdc44d2 merge-ort: allow update messages to be written to different file stream
     @@ Commit message
          merge-ort: allow update messages to be written to different file stream
      
          This modifies the new display_update_messages() function to allow
     -    printing to somewhere other than stdout.
     +    printing to somewhere other than stdout.  It also consolidates the
     +    location of the diff_warn_rename_limit() message with the rest of the
     +    CONFLICT and other update messages to all go to the same stream.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ merge-ort.c: void merge_display_update_messages(struct merge_options *opt,
       
       	/* Also include needed rename limit adjustment now */
       	diff_warn_rename_limit("merge.renamelimit",
     --			       opti->renames.needed_limit, 0, stdout);
     +-			       opti->renames.needed_limit, 0, stderr);
      +			       opti->renames.needed_limit, 0, stream);
       
       	trace2_region_leave("merge", "display messages", opt->repo);
  8:  d0d30e92ecd =  9:  c8ed002408d merge-tree: support including merge messages in output
  9:  9c2334ae9f2 = 10:  1c2a3f5ef63 merge-ort: provide a merge_get_conflicted_files() helper function
 10:  243134dc247 = 11:  9c2389eef0e merge-tree: provide a list of which files have conflicts
 11:  c322e4c6938 = 12:  2188a8ca1e7 merge-tree: provide easy access to `ls-files -u` style info
  -:  ----------- > 13:  52339b396fa merge-tree: allow `ls-files -u` style info to be NUL terminated
 12:  25677d5038c ! 14:  c854ecb5f4a merge-tree: add a --allow-unrelated-histories flag
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
       '
       
       test_expect_success 'Content merge and a few conflicts' '
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and modes without messages' '
     - 	test_cmp conflicted-file-info actual
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'NUL terminated conflicted file "lines"' '
     + 	test_cmp expect actual
       '
       
      +test_expect_success 'error out by default for unrelated histories' '
 13:  e7c63425a0e = 15:  bc8591bbb63 git-merge-tree.txt: add a section on potentional usage mistakes

-- 
gitgitgadget

[PATCH v3 01/15] merge-tree: rename merge_trees() to trivial_merge_trees()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget

[PATCH v3 02/15] merge-tree: move logic for existing merge into new function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget

[PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 61 +++++++++++++++++++++++++++++++++++++-------
 git.c                |  2 +-
 2 files changed, 53 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..e98ec8a9f1d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,51 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    'w'),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), 't'),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	if (o.mode) {
+		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
+		if (argc != expected_remaining_argc)
+			usage_with_options(merge_tree_usage, mt_options);
+	} else {
+		if (argc < 2 || argc > 3)
+			usage_with_options(merge_tree_usage, mt_options);
+		o.mode = (argc == 2 ? 'w' : 't');
+	}
+
+	/* Do the relevant type of merge */
+	if (o.mode == 'w')
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget

[PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability.

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 71 +++++++++++++++++++++-----
 builtin/merge-tree.c             | 44 +++++++++++++++-
 t/t4301-merge-tree-write-tree.sh | 88 ++++++++++++++++++++++++++++++++
 3 files changed, 190 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..569485815a0 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,73 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The second form is deprecated and supported only for backward
+compatibility.  It will likely be removed in the future, and will not
+be discussed further in this manual.
+
+The first form will merge the two branches, doing a real merge.  A real
+merge is distinguished from a trivial merge in that it includes:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, it will create a new toplevel tree object.
+See `OUTPUT` below for details.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1.
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
+be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+However, it does not quite fit into the same category of low-level
+plumbing commands since the possibility of merge conflicts give it a
+much higher chance of the command not succeeding.
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index e98ec8a9f1d..d14c9f6e44e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -392,7 +395,46 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..66c3eaf2021
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,88 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(cat .git/AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	cat tmp | sed -e s/HEAD/side1/ >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget

[PATCH v3 05/15] Introduce a variant of the `warning()` function that takes a `FILE *`

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

We are about to teach `diff_warn_rename_limit()` to write into a file
instead of `stderr`. That function wants to call `warning()` when
writing to `stderr`, though, allowing for the `warn_routine` to be
overridden.

Let's introduce a helper for that.

Note: Since there is currently no need to provide similar functions for
`error()` or `die()`, let alone for the `_errno` variants, we will leave
that to a date when the need for those should arise, if ever.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-compat-util.h |  1 +
 usage.c           | 14 ++++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index d70ce142861..64ba60e5c71 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,6 +475,7 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
+void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index c7d233b0de9..0bfd2c603c0 100644
--- a/usage.c
+++ b/usage.c
@@ -253,6 +253,20 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
+void warning_fp(FILE *out, const char *warn, ...)
+{
+	va_list params;
+
+	va_start(params, warn);
+	if (out == stderr)
+		warn_routine(warn, params);
+	else {
+		vfprintf(out, warn, params);
+		fputc('\n', out);
+	}
+	va_end(params);
+}
+
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 
-- 
gitgitgadget

[PATCH v3 06/15] diff: allow diff_warn_rename_limit to write somewhere besides stderr

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <Johannes.Schindelin@gmx.de>

diff_warn_rename_limit() is hardcoded to write to stderr.  Make it
accept a file stream parameter to make it more flexible.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 diff.c            | 13 ++++++++-----
 diff.h            |  3 ++-
 merge-ort.c       |  2 +-
 merge-recursive.c |  3 ++-
 4 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index 1bfb01c18ec..28368110147 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,17 +6377,20 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out)
 {
 	fflush(stdout);
 	if (degraded_cc)
-		warning(_(degrade_cc_to_c_warning));
+		warning_fp(out, _(degrade_cc_to_c_warning));
 	else if (needed)
-		warning(_(rename_limit_warning));
+		warning_fp(out, _(rename_limit_warning));
 	else
 		return;
+
+
 	if (0 < needed)
-		warning(_(rename_limit_advice), varname, needed);
+		warning_fp(out, _(rename_limit_advice), varname, needed);
 }
 
 static void create_filepairs_for_header_only_notifications(struct diff_options *o)
@@ -6870,7 +6873,7 @@ int diff_result_code(struct diff_options *opt, int status)
 
 	diff_warn_rename_limit("diff.renameLimit",
 			       opt->needed_rename_limit,
-			       opt->degraded_cc_to_c);
+			       opt->degraded_cc_to_c, stderr);
 	if (!opt->flags.exit_with_status &&
 	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
 		return status;
diff --git a/diff.h b/diff.h
index ce9e2cf2e4f..40c5b78fb0a 100644
--- a/diff.h
+++ b/diff.h
@@ -597,7 +597,8 @@ void diffcore_fix_diff_index(void);
 int diff_queue_is_empty(struct diff_options *o);
 void diff_flush(struct diff_options*);
 void diff_free(struct diff_options*);
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out);
 
 /* diff-raw status letters */
 #define DIFF_STATUS_ADDED		'A'
diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..46e72b62880 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4305,7 +4305,7 @@ void merge_switch_to_result(struct merge_options *opt,
 
 		/* Also include needed rename limit adjustment now */
 		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
+				       opti->renames.needed_limit, 0, stderr);
 
 		trace2_region_leave("merge", "display messages", opt->repo);
 	}
diff --git a/merge-recursive.c b/merge-recursive.c
index 9ec1e6d043a..01ca82773cc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3738,7 +3738,8 @@ static void merge_finalize(struct merge_options *opt)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->priv->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0,
+				       stderr);
 	FREE_AND_NULL(opt->priv);
 }
 
-- 
gitgitgadget

[PATCH v3 07/15] merge-ort: split out a separate display_update_messages() function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 77 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 36 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 46e72b62880..82d2faf5bf9 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0, stderr);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4273,42 +4312,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
 
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0, stderr);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This modifies the new display_update_messages() function to allow
printing to somewhere other than stdout.  It also consolidates the
location of the diff_warn_rename_limit() message with the rest of the
CONFLICT and other update messages to all go to the same stream.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 9 +++++----
 merge-ort.h | 3 ++-
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 82d2faf5bf9..d28d1721d14 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 }
 
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result)
+				   struct merge_result *result,
+				   FILE *stream)
 {
 	struct merge_options_internal *opti = result->priv;
 	struct hashmap_iter iter;
@@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
 	for (i = 0; i < olist.nr; ++i) {
 		struct strbuf *sb = olist.items[i].util;
 
-		printf("%s", sb->buf);
+		strbuf_write(sb, stream);
 	}
 	string_list_clear(&olist, 0);
 
 	/* Also include needed rename limit adjustment now */
 	diff_warn_rename_limit("merge.renamelimit",
-			       opti->renames.needed_limit, 0, stderr);
+			       opti->renames.needed_limit, 0, stream);
 
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
@@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
 	}
 
 	if (display_update_msgs)
-		merge_display_update_messages(opt, result);
+		merge_display_update_messages(opt, result, stdout);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..d643b47cb7c 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
  * so only call this when bypassing merge_switch_to_result().
  */
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result);
+				   struct merge_result *result,
+				   FILE *stream);
 
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
-- 
gitgitgadget

[PATCH v3 09/15] merge-tree: support including merge messages in output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 19 ++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 21 +++++++++++++++
 3 files changed, 76 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 569485815a0..42e0f8f6183 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
@@ -35,17 +35,47 @@ merge is distinguished from a trivial merge in that it includes:
 After the merge completes, it will create a new toplevel tree object.
 See `OUTPUT` below for details.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
+
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -69,7 +99,8 @@ be used as a part of a series of steps such as
 
 However, it does not quite fit into the same category of low-level
 plumbing commands since the possibility of merge conflicts give it a
-much higher chance of the command not succeeding.
+much higher chance of the command not succeeding (and NEWTREE containing
+a bunch of stuff other than just a toplevel tree).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d14c9f6e44e..6a556ab1c9c 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -390,6 +390,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -432,18 +433,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result, stdout);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -453,10 +463,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    'w'),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), 't'),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	if (o.mode) {
@@ -468,6 +481,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			usage_with_options(merge_tree_usage, mt_options);
 		o.mode = (argc == 2 ? 'w' : 't');
 	}
+	if (o.mode == 't' && original_argc < argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 66c3eaf2021..e2255711f9c 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -85,4 +85,25 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v3 10/15] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index d28d1721d14..c4ce6027dc4 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index d643b47cb7c..e635a294ea8 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result,
 				   FILE *stream);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v3 11/15] merge-tree: provide a list of which files have conflicts

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 42e0f8f6183..160e8f44b62 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -55,6 +55,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -66,6 +67,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 6a556ab1c9c..54dae018203 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -394,7 +397,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -438,6 +442,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result, stdout);
@@ -486,7 +506,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index e2255711f9c..7113d060bc5 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -95,6 +95,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -106,4 +108,13 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide an --exclude-modes-oids-stages/-l option for users to exclude
the mode, oid, and stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 30 ++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++++--
 3 files changed, 58 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 160e8f44b62..55bb7bc61c1 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -38,6 +38,11 @@ See `OUTPUT` below for details.
 OPTIONS
 -------
 
+--exclude-oids-and-modes::
+	Instead of writing a list of (mode, oid, stage, path) tuples
+	to output for conflicted files, just provide a list of
+	filenames with conflicts.
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -55,7 +60,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -67,18 +72,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--exclude-oids-and-modes` option is passed, the mode, object, and
+stage will be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -110,6 +120,14 @@ plumbing commands since the possibility of merge conflicts give it a
 much higher chance of the command not succeeding (and NEWTREE containing
 a bunch of stuff other than just a toplevel tree).
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 54dae018203..dc52cd02dce 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -394,6 +394,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int exclude_modes_oids_stages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->exclude_modes_oids_stages)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), 't'),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F('l', "exclude-modes-oids-stages",
+			   &o.exclude_modes_oids_stages,
+			   N_("list conflicted files without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 7113d060bc5..1572f460da0 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -47,6 +47,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(cat .git/AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -86,7 +87,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -109,7 +110,7 @@ test_expect_success 'test conflict notices and such' '
 '
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -117,4 +118,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v3 13/15] merge-tree: allow `ls-files -u` style info to be NUL terminated

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much as `git ls-files` has a -z option, let's add one to merge-tree so
that the conflict-info section can be NUL terminated (and avoid quoting
of unusual filenames).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 21 +++++++++++++----
 builtin/merge-tree.c             |  4 +++-
 t/t4301-merge-tree-write-tree.sh | 40 ++++++++++++++++++++++++++++++++
 3 files changed, 60 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 55bb7bc61c1..02f766716f9 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -38,6 +38,12 @@ See `OUTPUT` below for details.
 OPTIONS
 -------
 
+-z::
+	Do not quote filenames in the <Conflicted file info> section,
+	and end each filename with a NUL character rather than
+	newline.  Also begin the messages section with a NUL character
+	instead of a newline.  See OUTPUT below for more information.
+
 --exclude-oids-and-modes::
 	Instead of writing a list of (mode, oid, stage, path) tuples
 	to output for conflicted files, just provide a list of
@@ -70,7 +76,8 @@ OID of toplevel tree
 
 This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
-files within this tree may have embedded conflict markers.
+files within this tree may have embedded conflict markers.  This section
+is always followed by a newline.
 
 Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
@@ -82,19 +89,25 @@ This is a sequence of lines with the format
 The filename will be quoted as explained for the configuration
 variable `core.quotePath` (see linkgit:git-config[1]).  However, if
 the `--exclude-oids-and-modes` option is passed, the mode, object, and
-stage will be omitted.
+stage will be omitted.  If `-z` is passed, the "lines" are terminated
+by a NUL character instead of a newline character.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
-This always starts with a blank line to separate it from the previous
-sections, and then has free-form messages about the merge, such as:
+This always starts with a blank line (or NUL if `-z` is passed) to
+separate it from the previous sections, and then has free-form
+messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
   * "Failed to merge submodule <submodule> (<reason>)"
   * "Warning: cannot merge binary files: <filename>"
 
+Note that these free-form messages will never have a NUL character
+in or between them, even if -z is passed.  It is simply a large block
+of text taking up the remainder of the output.
+
 EXIT STATUS
 -----------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index dc52cd02dce..7e55f0fa301 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -464,7 +464,7 @@ static int real_merge(struct merge_tree_options *o,
 		string_list_clear(&conflicted_files, 1);
 	}
 	if (o->show_messages) {
-		printf("\n");
+		putchar(line_termination);
 		merge_display_update_messages(&opt, &result, stdout);
 	}
 	merge_finalize(&opt, &result);
@@ -490,6 +490,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), 't'),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_SET_INT('z', NULL, &line_termination,
+			    N_("separate paths with the NUL character"), '\0'),
 		OPT_BOOL_F('l', "exclude-modes-oids-stages",
 			   &o.exclude_modes_oids_stages,
 			   N_("list conflicted files without modes/oids/stages"),
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 1572f460da0..f89d87c26b7 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -139,4 +139,44 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'NUL terminated conflicted file "lines"' '
+	git checkout -b tweak1 side1 &&
+	test_write_lines zero 1 2 3 4 5 6 >numbers &&
+	git add numbers &&
+	git mv numbers "Αυτά μου φαίνονται κινέζικα" &&
+	git commit -m "Renamed numbers" &&
+
+	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
+	echo HASH >expect &&
+
+	q_to_tab <<-EOF | lf_to_nul >>expect &&
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~tweak1
+	100644 HASH 2Qwhatever~tweak1
+	100644 HASH 1QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 2QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 3QΑυτά μου φαίνονται κινέζικα
+
+	EOF
+
+	cat <<-EOF >>expect &&
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
+	CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
+	Auto-merging Αυτά μου φαίνονται κινέζικα
+	CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v3 14/15] merge-tree: add a --allow-unrelated-histories flag

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 02f766716f9..e6a9ff2768b 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -55,6 +55,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 7e55f0fa301..58c0ddc5a32 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -393,6 +393,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int exclude_modes_oids_stages;
 };
@@ -430,7 +431,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -496,6 +497,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.exclude_modes_oids_stages,
 			   N_("list conflicted files without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index f89d87c26b7..4de089d976d 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -38,7 +38,13 @@ test_expect_success setup '
 	>whatever/empty &&
 	git add numbers greeting whatever/empty &&
 	test_tick &&
-	git commit -m other-modifications
+	git commit -m other-modifications &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Content merge and a few conflicts' '
@@ -179,4 +185,20 @@ test_expect_success 'NUL terminated conflicted file "lines"' '
 	test_cmp expect actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v3 15/15] git-merge-tree.txt: add a section on potentional usage mistakes

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index e6a9ff2768b..6a2ed475106 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -146,6 +146,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> @@ -392,7 +395,46 @@ struct merge_tree_options {
>  static int real_merge(struct merge_tree_options *o,
>  		      const char *branch1, const char *branch2)
>  {
> -	die(_("real merges are not yet implemented"));
> +	struct commit *parent1, *parent2;
> +	struct commit_list *common;
> +	struct commit_list *merge_bases = NULL;
> +	struct commit_list *j;
> +	struct merge_options opt;
> +	struct merge_result result = { 0 };
> +
> +	parent1 = get_merge_parent(branch1);
> +	if (!parent1)
> +		help_unknown_ref(branch1, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	parent2 = get_merge_parent(branch2);
> +	if (!parent2)
> +		help_unknown_ref(branch2, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	init_merge_options(&opt, the_repository);
> +
> +	opt.show_rename_progress = 0;
> +
> +	opt.branch1 = branch1;
> +	opt.branch2 = branch2;
> +
> +	/*
> +	 * Get the merge bases, in reverse order; see comment above
> +	 * merge_incore_recursive in merge-ort.h
> +	 */
> +	common = get_merge_bases(parent1, parent2);
> +	if (!common)
> +		die(_("refusing to merge unrelated histories"));

It appears to me that "merge-tree" in this mode, with the above
code, cannot be used as a workhorse to implement server-side
cherry-pick (or revert), which needs to allow the user to specify an
arbitrary "common ancestor", instead of computing on its own.

To replay the change made by commit A on top of commit X (i.e.
"cherry-pick A on X"), we have to be able to say "compute the
three-way merge between A and X, pretending as if A^ were their
common ancestor".  The story is the same for revert---we compute
three-way merge between A^ and X, pretending as if A were their
common ancestor.

The above interface into this function, sadly, does not seem to
allow such a request, unless I am missing something.

And if I am correct, it is a shame---after all, the point of the
merge-trees command is to take three trees and run a three-way
merge, and not being able to merge three "trees" and require
"commits" makes this mode much less useful than its potential.

Re: [PATCH v2 03/13] merge-tree: add option parsing and initial shell for real merge function

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +struct merge_tree_options {
> +	int mode;
> +};

> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_CMDMODE(0, "write-tree", &o.mode,
> +			    N_("do a real merge instead of a trivial merge"),
> +			    'w'),

Given the length of the second line in the usage[] array, it would
make more sense to have "'w')," on the same line to make it match
better with the following one.

> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
> +			    N_("do a trivial merge only"), 't'),
> +		OPT_END()
> +	};
> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +	if (o.mode) {
> +		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.mode = (argc == 2 ? 'w' : 't');
> +	}

We are not planning to have tons of different command modes, but the
above looks quite brittle in assuming that 'w' and 't' are the only
ones, and not having an easy way to extend the part without major
rewrite when the assumption will have to be broken.  I wonder:

	switch (o.cmd_mode) {
        default:
		BUG("unexpected cmdmode %c", o.cmd_mode);
	case 0:
		switch (argc) {
		default:
			usage_with_options(merge_tree_usage, mt_options);
		case 2: 
			o.cmd_mode = 'w';
			break;
		case 3:
			o.cmd_mode = 't';
			break;
		}
                expected_remaining_argc = argc;
		break;
	case 'w':
		expected_remaining_argc = 2;
                break;
	case 't':
		expected_remaining_argc = 3;
 		break;
	}

        if (argc != expected_remaining_argc)
		usage_with_options(merge_tree_usage, mt_options);

even though it is a tad longer with more boilerplate, is easlier to
manage.

> +	/* Do the relevant type of merge */
> +	if (o.mode == 'w')
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/git.c b/git.c
> index 5ff21be21f3..6090a1289db 100644
> --- a/git.c
> +++ b/git.c
> @@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
>  	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
> -	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
> +	{ "merge-tree", cmd_merge_tree, RUN_SETUP },

This affects git_support_parseopt_helper function in the completion
script, but it by itself is very unlikely to break existing tests on
the completion, as the bit only affects how the command line options
are completed, and the command didn't have any command line options
to be tested before this series ;-)

>  	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
>  	{ "mktree", cmd_mktree, RUN_SETUP },
>  	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },

Re: [PATCH v2 04/13] merge-tree: implement real merges

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Elijah Newren <newren@gmail.com>
>
> This adds the ability to perform real merges rather than just trivial
> merges (meaning handling three way content merges, recursive ancestor
> consolidation, renames, proper directory/file conflict handling, and so
> forth).  However, unlike `git merge`, the working tree and index are
> left alone and no branch is updated.
>
> The only output is:
>   - the toplevel resulting tree printed on stdout
>   - exit status of 0 (clean), 1 (conflicts present), anything else
>     (merge could not be performed; unknown if clean or conflicted)
>
> This output is meant to be used by some higher level script, perhaps in
> a sequence of steps like this:
>
>    NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
>    test $? -eq 0 || die "There were conflicts..."
>    NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
>    git update-ref $BRANCH1 $NEWCOMMIT

It is unclear what NEWTREE has, if anything meaningful, when the
command exited with non-zero status.  Let's read on.

> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 58731c19422..569485815a0 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -3,26 +3,73 @@ git-merge-tree(1)
>  
>  NAME
>  ----
> -git-merge-tree - Show three-way merge without touching index
> +git-merge-tree - Perform merge without touching index or working tree

OK.  It is interesting that both apply equally to either command
mode ;-)

> +Performs a merge, but does not make any new commits and does not read
> +from or write to either the working tree or index.
> +
> +The second form is deprecated and supported only for backward
> +compatibility.  It will likely be removed in the future, and will not
> +be discussed further in this manual.

This, especially the deletion of the original description on what
trivial merge does, may be premature, especially if it is still
"supported for backward compatibility".

> +The first form will merge the two branches, doing a real merge.  A real
> +merge is distinguished from a trivial merge in that it includes:

I think this is worth keeping, simply because I do not think you
should entirely omit description on the trivial one.

But if we were to remove the description on the trivial one, then it
is totally meaningless to label the following as "... distinguished
from a trivial merge in that".  The list of things the real merge
command mode does (below) is of course worth having.

> +  * three way content merges of individual files
> +  * rename detection
> +  * proper directory/file conflict handling
> +  * recursive ancestor consolidation (i.e. when there is more than one
> +    merge base, creating a virtual merge base by merging the merge bases)
> +  * etc.
> +
> +After the merge completes, it will create a new toplevel tree object.
> +See `OUTPUT` below for details.
> +
> +OUTPUT
> +------
> +
> +For either a successful or conflicted merge, the output from
> +git-merge-tree is simply one line:
> +
> +	<OID of toplevel tree>
> +
> +The printed tree object corresponds to what would be checked out in
> +the working tree at the end of `git merge`, and thus may have files
> +with conflict markers in them.

So we would leave cruft in the object store, but that is very much
on purpose, and the expectation is that the user would commit-tree
the tree object and reference it with a ref soon enough before
garbage collection prunes them, just like how write-tree is meant to
be used.  OK.

> +EXIT STATUS
> +-----------
> +
> +For a successful, non-conflicted merge, the exit status is 0.  When the
> +merge has conflicts, the exit status is 1.  If the merge is not able to
> +complete (or start) due to some kind of error, the exit status is
> +something other than 0 or 1.

And the output given to the standard output stream, when the command
exits with status higher than 1, is...?  "unspecified" is of course
an acceptable answer, of course, but I am wondering if it is worth
spelling out in the doc.

> +USAGE NOTES
> +-----------
> +
> +git-merge-tree was written to be low-level plumbing, similar to
> +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could

A notable omission in the above list is 'write-tree'.

> +be used as a part of a series of steps such as
> +
> +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> +       test $? -eq 0 || die "There were conflicts..."
> +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> +       git update-ref $BRANCH1 $NEWCOMMIT
> +
> +However, it does not quite fit into the same category of low-level
> +plumbing commands since the possibility of merge conflicts give it a
> +much higher chance of the command not succeeding.

I am not sure if that is a fair categorization.  It is a fine
building block at the lowest level of the tool hierarchy.  The
primary thing that differentiates plumbing from Porcelain is that
the former is a better fit for scripting: doing one thing and one
thing well with minimum and stable UI.  The complexity of that one
thing it does has much less to do with the categorization, and the
rate at which users may use the command in a failure-inducing
situation has nothing to do with it.  The write-tree plumbing
command will reliably fail with 100% chance when the index used as
its input is unmerged.

> @@ -392,7 +395,46 @@ struct merge_tree_options {
>  static int real_merge(struct merge_tree_options *o,
>  		      const char *branch1, const char *branch2)
>  {
> -	die(_("real merges are not yet implemented"));
> +	struct commit *parent1, *parent2;
> +	struct commit_list *common;
> +	struct commit_list *merge_bases = NULL;
> +	struct commit_list *j;
> +	struct merge_options opt;
> +	struct merge_result result = { 0 };
> +
> +	parent1 = get_merge_parent(branch1);
> +	if (!parent1)
> +		help_unknown_ref(branch1, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	parent2 = get_merge_parent(branch2);
> +	if (!parent2)
> +		help_unknown_ref(branch2, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	init_merge_options(&opt, the_repository);
> +
> +	opt.show_rename_progress = 0;
> +
> +	opt.branch1 = branch1;
> +	opt.branch2 = branch2;
> +
> +	/*
> +	 * Get the merge bases, in reverse order; see comment above
> +	 * merge_incore_recursive in merge-ort.h
> +	 */
> +	common = get_merge_bases(parent1, parent2);
> +	if (!common)
> +		die(_("refusing to merge unrelated histories"));
> +	for (j = common; j; j = j->next)
> +		commit_list_insert(j->item, &merge_bases);
> +
> +	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> +	if (result.clean < 0)
> +		die(_("failure to merge"));
> +	puts(oid_to_hex(&result.tree->object.oid));
> +	merge_finalize(&opt, &result);
> +	return !result.clean; /* result.clean < 0 handled above */
>  }

The implementation is rather straight-forward, if you know how to
drive the merge_incore_recursive() helper ;-)

> +test_expect_success setup '
> +	test_write_lines 1 2 3 4 5 >numbers &&
> +	echo hello >greeting &&
> +	echo foo >whatever &&
> +	git add numbers greeting whatever &&
> +	test_tick &&
> +	git commit -m initial &&
> +
> +	git branch side1 &&
> +	git branch side2 &&
> +
> +	git checkout side1 &&
> +	test_write_lines 1 2 3 4 5 6 >numbers &&
> +	echo hi >greeting &&
> +	echo bar >whatever &&
> +	git add numbers greeting whatever &&
> +	test_tick &&
> +	git commit -m modify-stuff &&
> +
> +	git checkout side2 &&
> +	test_write_lines 0 1 2 3 4 5 >numbers &&
> +	echo yo >greeting &&
> +	git rm whatever &&
> +	mkdir whatever &&
> +	>whatever/empty &&
> +	git add numbers greeting whatever/empty &&
> +	test_tick &&
> +	git commit -m other-modifications
> +'
> +
> +test_expect_success 'Content merge and a few conflicts' '
> +	git checkout side1^0 &&
> +	test_must_fail git merge side2 &&
> +	expected_tree=$(cat .git/AUTO_MERGE) &&
> +
> +	# We will redo the merge, while we are still in a conflicted state!
> +	test_when_finished "git reset --hard" &&
> +
> +	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
> +	actual_tree=$(head -n 1 RESULT) &&
> +
> +	# Due to differences of e.g. "HEAD" vs "side1", the results will not
> +	# exactly match.  Dig into individual files.
> +
> +	# Numbers should have three-way merged cleanly
> +	test_write_lines 0 1 2 3 4 5 6 >expect &&
> +	git show ${actual_tree}:numbers >actual &&
> +	test_cmp expect actual &&
> +
> +	# whatever and whatever~<branch> should have same HASHES
> +	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
> +	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
> +	test_cmp expect actual &&
> +
> +	# greeting should have a merge conflict
> +	git show ${expected_tree}:greeting >tmp &&
> +	cat tmp | sed -e s/HEAD/side1/ >expect &&
> +	git show ${actual_tree}:greeting >actual &&
> +	test_cmp expect actual
> +'

It is somewhat sad that we need to reivent merge test cases over and
over, instead of easily reuse an existing one by replacing

	git checkout one &&
	git merge two

with

	git checkout one &&
	T=$(git merge-tree HEAD two) &&
	C=$(git commit-tree $T -p HEAD -p two) &&
	git reset --hard $C

;-)

> +> +test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
> +	# Mis-spell with single "s" instead of double "s"
> +	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
> +
> +	grep "error: unknown option.*mesages" expect
> +'
> +
> +test_expect_success 'Barf on too many arguments' '
> +	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
> +
> +	grep "^usage: git merge-tree" expect
> +'

Re: [PATCH v2 08/13] merge-tree: support including merge messages in output

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +By default, for a successful merge, the output from git-merge-tree is
> +simply one line:
> +
> +	<OID of toplevel tree>
> +
> +Whereas for a conflicted merge, the output is by default of the form:
>  
>  	<OID of toplevel tree>
> +	<Informational messages>

Sounds useful.  This made me wonder, as the only shuffling of the
output destination in the past few steps were to send the output to
some "FILE *", how you send the findings you make while coming up
with the result _after_ the result.  It turns out that the ORT
machinery already buffers these findings in a strbuf per path, so
there is no trouble doing so ;-)

It still makes me wonder how the "send rename warnings to the
standard output stream, instead of the standard error stream" change
interacts with this change, though.  That needs to be done way
before you finish computing the result, and it does not seem to be
buffered in-core, like per-path conflict information messages.

Re: [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).

Makes sense.  Ideally things like this should be discoverable by
inspecting the tree object shown as the result of the (conflicted)
merge, but since the design of the output is to show only a single
tree, there is nowhere to store such an extra piece of information
per path (grepping for markers in blobs of course does not count).

I guess an alternative to show four trees when conflicted instead of
one (i.e. the primary tree may either contain only the cleanly
merged paths _or_ also blobs with conflict markers for conflicted
paths; the three other trees record three stages that would be in
the index, if we were performing the same merge using the index),
but a machine-parseable list of paths is fine.

> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;

OK.  The iteration used here makes casual readers wonder why the
helper doesn't make paths unique, but the string list item holds
in its util pointer a pointer to a structure with <stage, mode, oid>
tuple, so it is natural to make the consumer, who wants uniquified
list, responsible for deduping, like this loop.

> +		}
> +		string_list_clear(&conflicted_files, 1);

And the stage-info structure associated with these paths are
deallocated with this call.  Good.

> +	}

Re: [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).

Makes sense.  Ideally things like this should be discoverable by
inspecting the tree object shown as the result of the (conflicted)
merge, but since the design of the output is to show only a single
tree, there is nowhere to store such an extra piece of information
per path (grepping for markers in blobs of course does not count).

I guess an alternative to show four trees when conflicted instead of
one (i.e. the primary tree may either contain only the cleanly
merged paths _or_ also blobs with conflict markers for conflicted
paths; the three other trees record three stages that would be in
the index, if we were performing the same merge using the index),
but a machine-parseable list of paths is fine.

> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;

OK.  The iteration used here makes casual readers wonder why the
helper doesn't make paths unique, but the string list item holds
in its util pointer a pointer to a structure with <stage, mode, oid>
tuple, so it is natural to make the consumer, who wants uniquified
list, responsible for deduping, like this loop.

> +		}
> +		string_list_clear(&conflicted_files, 1);

And the stage-info structure associated with these paths are
deallocated with this call.  Good.

> +	}

Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
>  		merge_get_conflicted_files(&result, &conflicted_files);
>  		for (i = 0; i < conflicted_files.nr; i++) {
>  			const char *name = conflicted_files.items[i].string;
> -			if (last && !strcmp(last, name))
> +			struct stage_info *c = conflicted_files.items[i].util;
> +			if (!o->exclude_modes_oids_stages)
> +				printf("%06o %s %d\t",
> +				       c->mode, oid_to_hex(&c->oid), c->stage);
> +			else if (last && !strcmp(last, name))
>  				continue;
>  			write_name_quoted_relative(
>  				name, prefix, stdout, line_termination);

OK.  The addition (and disabling of the deduping) is quite trivial.
We do not even have to worry about line termination since the extra
pieces of info are prepended to the pathname.  Nice.

> @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			    N_("do a trivial merge only"), 't'),
>  		OPT_BOOL(0, "messages", &o.show_messages,
>  			 N_("also show informational/conflict messages")),
> +		OPT_BOOL_F('l', "exclude-modes-oids-stages",
> +			   &o.exclude_modes_oids_stages,
> +			   N_("list conflicted files without modes/oids/stages"),
> +			   PARSE_OPT_NONEG),

Why does "-l" give shorter output than without it?  "-l" strongly
hints a longer output than without, at least to me.  Just wondering
if this will not become a source of confusion to future scripting
users.

>  		OPT_END()
>  	};
>  

Re: [PATCH v2 12/13] merge-tree: add a --allow-unrelated-histories flag

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> @@ -430,7 +431,7 @@ static int real_merge(struct merge_tree_options *o,
>  	 * merge_incore_recursive in merge-ort.h
>  	 */
>  	common = get_merge_bases(parent1, parent2);
> -	if (!common)
> +	if (!common && !o->allow_unrelated_histories)
>  		die(_("refusing to merge unrelated histories"));
>  	for (j = common; j; j = j->next)
>  		commit_list_insert(j->item, &merge_bases);

Curious.  This step _adds_ an "--allow" option from the command
line, but we actually did not have to die() when seeing that there
is no common ancestor before this step.  The end result is OK either
way.

> @@ -494,6 +495,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			   &o.exclude_modes_oids_stages,
>  			   N_("list conflicted files without modes/oids/stages"),
>  			   PARSE_OPT_NONEG),
> +		OPT_BOOL_F(0, "allow-unrelated-histories",
> +			   &o.allow_unrelated_histories,
> +			   N_("allow merging unrelated histories"),
> +			   PARSE_OPT_NONEG),
>  		OPT_END()
>  	};

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren]

On Wed, Feb 2, 2022 at 1:22 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > @@ -392,7 +395,46 @@ struct merge_tree_options {
> >  static int real_merge(struct merge_tree_options *o,
> >                     const char *branch1, const char *branch2)
> >  {
> > -     die(_("real merges are not yet implemented"));
> > +     struct commit *parent1, *parent2;
> > +     struct commit_list *common;
> > +     struct commit_list *merge_bases = NULL;
> > +     struct commit_list *j;
> > +     struct merge_options opt;
> > +     struct merge_result result = { 0 };
> > +
> > +     parent1 = get_merge_parent(branch1);
> > +     if (!parent1)
> > +             help_unknown_ref(branch1, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     parent2 = get_merge_parent(branch2);
> > +     if (!parent2)
> > +             help_unknown_ref(branch2, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     init_merge_options(&opt, the_repository);
> > +
> > +     opt.show_rename_progress = 0;
> > +
> > +     opt.branch1 = branch1;
> > +     opt.branch2 = branch2;
> > +
> > +     /*
> > +      * Get the merge bases, in reverse order; see comment above
> > +      * merge_incore_recursive in merge-ort.h
> > +      */
> > +     common = get_merge_bases(parent1, parent2);
> > +     if (!common)
> > +             die(_("refusing to merge unrelated histories"));
>
> It appears to me that "merge-tree" in this mode, with the above
> code, cannot be used as a workhorse to implement server-side
> cherry-pick (or revert), which needs to allow the user to specify an
> arbitrary "common ancestor", instead of computing on its own.
>
> To replay the change made by commit A on top of commit X (i.e.
> "cherry-pick A on X"), we have to be able to say "compute the
> three-way merge between A and X, pretending as if A^ were their
> common ancestor".  The story is the same for revert---we compute
> three-way merge between A^ and X, pretending as if A were their
> common ancestor.
>
> The above interface into this function, sadly, does not seem to
> allow such a request, unless I am missing something.
>
> And if I am correct, it is a shame---after all, the point of the
> merge-trees command is to take three trees and run a three-way
> merge, and not being able to merge three "trees" and require
> "commits" makes this mode much less useful than its potential.

Yes, you are reading right.  I think the cherry-pick/rebase
replacement actually deserves a separate command from what merges
should use; replaying a sequence of commits just has a number of UI
differences and abilities that I think pull it in a different
direction.  I didn't want to wait and submit everything all at once
(this series is long enough), and I figured that providing the in-core
equivalent to `git merge` was a simpler first step worth submitting
first before later providing the in-core equivalents to `git
rebase/git cherry-pick`.

(Also, I'm a bit wary of providing a command meant to just do a single
three-way merge with a defined merge-base, because that seems to be a
path towards returning to a scripted rebase.  Having a way to do only
a single such special merge is fine but we should avoid encouraging
people to go down that route.)

Re: [PATCH v2 04/13] merge-tree: implement real merges

[Elijah Newren]

On Wed, Feb 2, 2022 at 1:30 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > This adds the ability to perform real merges rather than just trivial
> > merges (meaning handling three way content merges, recursive ancestor
> > consolidation, renames, proper directory/file conflict handling, and so
> > forth).  However, unlike `git merge`, the working tree and index are
> > left alone and no branch is updated.
> >
> > The only output is:
> >   - the toplevel resulting tree printed on stdout
> >   - exit status of 0 (clean), 1 (conflicts present), anything else
> >     (merge could not be performed; unknown if clean or conflicted)
> >
> > This output is meant to be used by some higher level script, perhaps in
> > a sequence of steps like this:
> >
> >    NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> >    test $? -eq 0 || die "There were conflicts..."
> >    NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> >    git update-ref $BRANCH1 $NEWCOMMIT
>
> It is unclear what NEWTREE has, if anything meaningful, when the
> command exited with non-zero status.  Let's read on.
>
> > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > index 58731c19422..569485815a0 100644
> > --- a/Documentation/git-merge-tree.txt
> > +++ b/Documentation/git-merge-tree.txt
> > @@ -3,26 +3,73 @@ git-merge-tree(1)
> >
> >  NAME
> >  ----
> > -git-merge-tree - Show three-way merge without touching index
> > +git-merge-tree - Perform merge without touching index or working tree
>
> OK.  It is interesting that both apply equally to either command
> mode ;-)
>
> > +Performs a merge, but does not make any new commits and does not read
> > +from or write to either the working tree or index.
> > +
> > +The second form is deprecated and supported only for backward
> > +compatibility.  It will likely be removed in the future, and will not
> > +be discussed further in this manual.
>
> This, especially the deletion of the original description on what
> trivial merge does, may be premature, especially if it is still
> "supported for backward compatibility".

I actually extended it, but Dscho suggested removing it entirely --
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2201251804250.2121@tvgsbejvaqbjf.bet/.
I can restore it; does that paragraph look good to you (you can see
the full thing even if it's split by Dscho's commentary).

> > +The first form will merge the two branches, doing a real merge.  A real
> > +merge is distinguished from a trivial merge in that it includes:
>
> I think this is worth keeping, simply because I do not think you
> should entirely omit description on the trivial one.
>
> But if we were to remove the description on the trivial one, then it
> is totally meaningless to label the following as "... distinguished
> from a trivial merge in that".  The list of things the real merge
> command mode does (below) is of course worth having.
>
> > +  * three way content merges of individual files
> > +  * rename detection
> > +  * proper directory/file conflict handling
> > +  * recursive ancestor consolidation (i.e. when there is more than one
> > +    merge base, creating a virtual merge base by merging the merge bases)
> > +  * etc.
> > +
> > +After the merge completes, it will create a new toplevel tree object.
> > +See `OUTPUT` below for details.
> > +
> > +OUTPUT
> > +------
> > +
> > +For either a successful or conflicted merge, the output from
> > +git-merge-tree is simply one line:
> > +
> > +     <OID of toplevel tree>
> > +
> > +The printed tree object corresponds to what would be checked out in
> > +the working tree at the end of `git merge`, and thus may have files
> > +with conflict markers in them.
>
> So we would leave cruft in the object store, but that is very much
> on purpose, and the expectation is that the user would commit-tree
> the tree object and reference it with a ref soon enough before
> garbage collection prunes them, just like how write-tree is meant to
> be used.  OK.
>
> > +EXIT STATUS
> > +-----------
> > +
> > +For a successful, non-conflicted merge, the exit status is 0.  When the
> > +merge has conflicts, the exit status is 1.  If the merge is not able to
> > +complete (or start) due to some kind of error, the exit status is
> > +something other than 0 or 1.
>
> And the output given to the standard output stream, when the command
> exits with status higher than 1, is...?  "unspecified" is of course
> an acceptable answer, of course, but I am wondering if it is worth
> spelling out in the doc.

Yeah, "unspecified" -- whatever random error the user triggered
(couldn't parse arguments, the refs you gave don't exist, the repo you
are running from isn't a repo, disk is full, etc.)

> > +USAGE NOTES
> > +-----------
> > +
> > +git-merge-tree was written to be low-level plumbing, similar to
> > +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
>
> A notable omission in the above list is 'write-tree'.

Ooh, yeah, I should add that one.

> > +be used as a part of a series of steps such as
> > +
> > +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> > +       test $? -eq 0 || die "There were conflicts..."
> > +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> > +       git update-ref $BRANCH1 $NEWCOMMIT
> > +
> > +However, it does not quite fit into the same category of low-level
> > +plumbing commands since the possibility of merge conflicts give it a
> > +much higher chance of the command not succeeding.
>
> I am not sure if that is a fair categorization.  It is a fine
> building block at the lowest level of the tool hierarchy.  The
> primary thing that differentiates plumbing from Porcelain is that
> the former is a better fit for scripting: doing one thing and one
> thing well with minimum and stable UI.  The complexity of that one
> thing it does has much less to do with the categorization, and the
> rate at which users may use the command in a failure-inducing
> situation has nothing to do with it.  The write-tree plumbing
> command will reliably fail with 100% chance when the index used as
> its input is unmerged.
>
> > @@ -392,7 +395,46 @@ struct merge_tree_options {
> >  static int real_merge(struct merge_tree_options *o,
> >                     const char *branch1, const char *branch2)
> >  {
> > -     die(_("real merges are not yet implemented"));
> > +     struct commit *parent1, *parent2;
> > +     struct commit_list *common;
> > +     struct commit_list *merge_bases = NULL;
> > +     struct commit_list *j;
> > +     struct merge_options opt;
> > +     struct merge_result result = { 0 };
> > +
> > +     parent1 = get_merge_parent(branch1);
> > +     if (!parent1)
> > +             help_unknown_ref(branch1, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     parent2 = get_merge_parent(branch2);
> > +     if (!parent2)
> > +             help_unknown_ref(branch2, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     init_merge_options(&opt, the_repository);
> > +
> > +     opt.show_rename_progress = 0;
> > +
> > +     opt.branch1 = branch1;
> > +     opt.branch2 = branch2;
> > +
> > +     /*
> > +      * Get the merge bases, in reverse order; see comment above
> > +      * merge_incore_recursive in merge-ort.h
> > +      */
> > +     common = get_merge_bases(parent1, parent2);
> > +     if (!common)
> > +             die(_("refusing to merge unrelated histories"));
> > +     for (j = common; j; j = j->next)
> > +             commit_list_insert(j->item, &merge_bases);
> > +
> > +     merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > +     if (result.clean < 0)
> > +             die(_("failure to merge"));
> > +     puts(oid_to_hex(&result.tree->object.oid));
> > +     merge_finalize(&opt, &result);
> > +     return !result.clean; /* result.clean < 0 handled above */
> >  }
>
> The implementation is rather straight-forward, if you know how to
> drive the merge_incore_recursive() helper ;-)

:-)

>
> > +test_expect_success setup '
> > +     test_write_lines 1 2 3 4 5 >numbers &&
> > +     echo hello >greeting &&
> > +     echo foo >whatever &&
> > +     git add numbers greeting whatever &&
> > +     test_tick &&
> > +     git commit -m initial &&
> > +
> > +     git branch side1 &&
> > +     git branch side2 &&
> > +
> > +     git checkout side1 &&
> > +     test_write_lines 1 2 3 4 5 6 >numbers &&
> > +     echo hi >greeting &&
> > +     echo bar >whatever &&
> > +     git add numbers greeting whatever &&
> > +     test_tick &&
> > +     git commit -m modify-stuff &&
> > +
> > +     git checkout side2 &&
> > +     test_write_lines 0 1 2 3 4 5 >numbers &&
> > +     echo yo >greeting &&
> > +     git rm whatever &&
> > +     mkdir whatever &&
> > +     >whatever/empty &&
> > +     git add numbers greeting whatever/empty &&
> > +     test_tick &&
> > +     git commit -m other-modifications
> > +'
> > +
> > +test_expect_success 'Content merge and a few conflicts' '
> > +     git checkout side1^0 &&
> > +     test_must_fail git merge side2 &&
> > +     expected_tree=$(cat .git/AUTO_MERGE) &&
> > +
> > +     # We will redo the merge, while we are still in a conflicted state!
> > +     test_when_finished "git reset --hard" &&
> > +
> > +     test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
> > +     actual_tree=$(head -n 1 RESULT) &&
> > +
> > +     # Due to differences of e.g. "HEAD" vs "side1", the results will not
> > +     # exactly match.  Dig into individual files.
> > +
> > +     # Numbers should have three-way merged cleanly
> > +     test_write_lines 0 1 2 3 4 5 6 >expect &&
> > +     git show ${actual_tree}:numbers >actual &&
> > +     test_cmp expect actual &&
> > +
> > +     # whatever and whatever~<branch> should have same HASHES
> > +     git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
> > +     git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
> > +     test_cmp expect actual &&
> > +
> > +     # greeting should have a merge conflict
> > +     git show ${expected_tree}:greeting >tmp &&
> > +     cat tmp | sed -e s/HEAD/side1/ >expect &&
> > +     git show ${actual_tree}:greeting >actual &&
> > +     test_cmp expect actual
> > +'
>
> It is somewhat sad that we need to reivent merge test cases over and
> over, instead of easily reuse an existing one by replacing
>
>         git checkout one &&
>         git merge two
>
> with
>
>         git checkout one &&
>         T=$(git merge-tree HEAD two) &&
>         C=$(git commit-tree $T -p HEAD -p two) &&
>         git reset --hard $C
>
> ;-)

Sorry...I'm afraid I'm not following.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Yes, you are reading right.  I think the cherry-pick/rebase
> replacement actually deserves a separate command from what merges
> should use; replaying a sequence of commits just has a number of UI
> differences and abilities that I think pull it in a different
> direction.

I completely disagree.  Each individual step in a sequence of
replaying commits in order (or in reverse order) should be
scriptable as a single merge-tree that takes "apply the change to go
from A^ to A on X".  Sequencing and placing UI around it is a job
for the script that drives merge-tree.

Re: [PATCH v2 08/13] merge-tree: support including merge messages in output

[Elijah Newren]

On Wed, Feb 2, 2022 at 1:30 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > +By default, for a successful merge, the output from git-merge-tree is
> > +simply one line:
> > +
> > +     <OID of toplevel tree>
> > +
> > +Whereas for a conflicted merge, the output is by default of the form:
> >
> >       <OID of toplevel tree>
> > +     <Informational messages>
>
> Sounds useful.  This made me wonder, as the only shuffling of the
> output destination in the past few steps were to send the output to
> some "FILE *", how you send the findings you make while coming up
> with the result _after_ the result.  It turns out that the ORT
> machinery already buffers these findings in a strbuf per path, so
> there is no trouble doing so ;-)

:-)

> It still makes me wonder how the "send rename warnings to the
> standard output stream, instead of the standard error stream" change
> interacts with this change, though.  That needs to be done way
> before you finish computing the result, and it does not seem to be
> buffered in-core, like per-path conflict information messages.

Actually, that's also stashed away in detect_regular_renames() from merge-ort.c:

    if (diff_opts.needed_rename_limit > renames->needed_limit)
        renames->needed_limit = diff_opts.needed_rename_limit;

and then the call to diff_warn_rename_limit() is deferred until after
all the informational messages are printed.  Also, this series
modifies diff_warn_rename_limit() later to allow it to send that
message to a different stream.

Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren]

On Wed, Feb 2, 2022 at 1:32 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
> >               merge_get_conflicted_files(&result, &conflicted_files);
> >               for (i = 0; i < conflicted_files.nr; i++) {
> >                       const char *name = conflicted_files.items[i].string;
> > -                     if (last && !strcmp(last, name))
> > +                     struct stage_info *c = conflicted_files.items[i].util;
> > +                     if (!o->exclude_modes_oids_stages)
> > +                             printf("%06o %s %d\t",
> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
> > +                     else if (last && !strcmp(last, name))
> >                               continue;
> >                       write_name_quoted_relative(
> >                               name, prefix, stdout, line_termination);
>
> OK.  The addition (and disabling of the deduping) is quite trivial.
> We do not even have to worry about line termination since the extra
> pieces of info are prepended to the pathname.  Nice.
>
> > @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >                           N_("do a trivial merge only"), 't'),
> >               OPT_BOOL(0, "messages", &o.show_messages,
> >                        N_("also show informational/conflict messages")),
> > +             OPT_BOOL_F('l', "exclude-modes-oids-stages",
> > +                        &o.exclude_modes_oids_stages,
> > +                        N_("list conflicted files without modes/oids/stages"),
> > +                        PARSE_OPT_NONEG),
>
> Why does "-l" give shorter output than without it?  "-l" strongly
> hints a longer output than without, at least to me.  Just wondering
> if this will not become a source of confusion to future scripting
> users.

Here's another example where I was struggling with naming.  Something
like ls-tree's `--name-only` would have been nice, but I was worried
it'd be confusing since it only affected the conflicted info section
and does not suppress the printing of the toplevel tree or the
informational messages sections.  And the name
--exclude-modes-oids-stages was long enough that I wanted a short flag
for it, and just used the first letter of the description ("list
conflicted files...").  I'm happy to change either the long or the
short name for this flag if anyone has suggestions.

Re: [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info

[Ævar Arnfjörð Bjarmason]

On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Much like `git merge` updates the index with information of the form
>     (mode, oid, stage, name)
> provide this output for conflicted files for merge-tree as well.
> Provide an --exclude-modes-oids-stages/-l option for users to exclude

This.

> +--exclude-oids-and-modes::

No longer matches this.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren]

On Wed, Feb 2, 2022 at 2:01 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > Yes, you are reading right.  I think the cherry-pick/rebase
> > replacement actually deserves a separate command from what merges
> > should use; replaying a sequence of commits just has a number of UI
> > differences and abilities that I think pull it in a different
> > direction.
>
> I completely disagree.  Each individual step in a sequence of
> replaying commits in order (or in reverse order) should be
> scriptable as a single merge-tree that takes "apply the change to go
> from A^ to A on X".  Sequencing and placing UI around it is a job
> for the script that drives merge-tree.

Adding such an ability to merge-tree would be trivial -- it basically
involves just two things: (1) accepting one extra argument, and (2)
calling merge_incore_nonrecursive() instead of
merge_incore_recursive().

However, I think forking a subprocess for every merge of a series of
commits is a completely unreasonable overhead, so even if we provide
such an option to merge-tree, I still want a separate plumbing-ish
tool that does non-worktree/non-index replaying of commits which is
not written as a driver of merge-tree.  That other tool should just
call merge_incore_nonrecursive() directly.  And such a tool, since it
should handle an arbitrary number of commits, should certainly be able
to handle just one commit.  From that angle, it feels like adding
another mode to merge-tree would just be a partial duplication of the
other tool.

However, if the other tool doesn't obviate the need for this
additional mode (perhaps it ends up being forced to be too
porcelain-ish insteading of plumbing-ish?), or folks really just want
another merge-tree mode, I'm happy to add one along with the tool I
submit later.  Does that sound reasonable to you, or is there
something you're still objecting to that I've missed?

Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info

[Ævar Arnfjörð Bjarmason]

On Wed, Feb 02 2022, Elijah Newren wrote:

> On Wed, Feb 2, 2022 at 1:32 PM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>> > @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
>> >               merge_get_conflicted_files(&result, &conflicted_files);
>> >               for (i = 0; i < conflicted_files.nr; i++) {
>> >                       const char *name = conflicted_files.items[i].string;
>> > -                     if (last && !strcmp(last, name))
>> > +                     struct stage_info *c = conflicted_files.items[i].util;
>> > +                     if (!o->exclude_modes_oids_stages)
>> > +                             printf("%06o %s %d\t",
>> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
>> > +                     else if (last && !strcmp(last, name))
>> >                               continue;
>> >                       write_name_quoted_relative(
>> >                               name, prefix, stdout, line_termination);
>>
>> OK.  The addition (and disabling of the deduping) is quite trivial.
>> We do not even have to worry about line termination since the extra
>> pieces of info are prepended to the pathname.  Nice.
>>
>> > @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>> >                           N_("do a trivial merge only"), 't'),
>> >               OPT_BOOL(0, "messages", &o.show_messages,
>> >                        N_("also show informational/conflict messages")),
>> > +             OPT_BOOL_F('l', "exclude-modes-oids-stages",
>> > +                        &o.exclude_modes_oids_stages,
>> > +                        N_("list conflicted files without modes/oids/stages"),
>> > +                        PARSE_OPT_NONEG),
>>
>> Why does "-l" give shorter output than without it?  "-l" strongly
>> hints a longer output than without, at least to me.  Just wondering
>> if this will not become a source of confusion to future scripting
>> users.
>
> Here's another example where I was struggling with naming.  Something
> like ls-tree's `--name-only` would have been nice, but I was worried
> it'd be confusing since it only affected the conflicted info section
> and does not suppress the printing of the toplevel tree or the
> informational messages sections.  And the name
> --exclude-modes-oids-stages was long enough that I wanted a short flag
> for it, and just used the first letter of the description ("list
> conflicted files...").  I'm happy to change either the long or the
> short name for this flag if anyone has suggestions.

There's always sidestepping it by replacing it with a --format :)

Anyway, I'd mentioned that in an earlier review in
<220124.864k5tigto.gmgdl@evledraar.gmail.com>. FWIW here's an experiment
to do that that I polished up (mostly copied from the ls-tree WIP code
I'd written already).

I don't know if it will ever be useful, or if you think it's
worthwhile/simpler, but in either case I think in doing this I spotted
the following issues or otherwise noted inconsistencies in the pre-image:

   The docs say that "<stage> <path>" is SP-separated, but it's
   actually TAB-separated, the rest is SP-separated.

 * That you de-dupe --exclude-modes-oids-stages is a bit of a hidden feature,
   but argubly initiative. Should it by optional? In any case my formatting
   experiment makes it optional, since it then needs to be generalized to de-dupe
   after we've formatted.

 * Perhaps we should support --abbrev as ls-tree does? The below diff shows
   it's easy enough.

 * The dance you have with sed-ing out the hash in the tests could be made much
   easier with "sed 1d <out >actual" and --no-messages for some existing tests.

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 6a2ed475106..e906d1dc9bf 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -44,10 +44,9 @@ OPTIONS
 	newline.  Also begin the messages section with a NUL character
 	instead of a newline.  See OUTPUT below for more information.
 
---exclude-oids-and-modes::
-	Instead of writing a list of (mode, oid, stage, path) tuples
-	to output for conflicted files, just provide a list of
-	filenames with conflicts.
+--conflict-format::
+	Override the default "%(objectmode) %(objectname)
+	%(stage)%x09%(path)" format.
 
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
@@ -89,13 +88,13 @@ Conflicted file info
 
 This is a sequence of lines with the format
 
-	<mode> <object> <stage> <filename>
+	%(objectmode) %(objectname) %(stage)%x09%(path)
 
 The filename will be quoted as explained for the configuration
-variable `core.quotePath` (see linkgit:git-config[1]).  However, if
-the `--exclude-oids-and-modes` option is passed, the mode, object, and
-stage will be omitted.  If `-z` is passed, the "lines" are terminated
-by a NUL character instead of a newline character.
+variable `core.quotePath` (see linkgit:git-config[1]).
+
+If `-z` is passed, the "lines" are terminated by a NUL character
+instead of a newline character.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 58c0ddc5a32..14fed95a8ce 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -395,9 +395,64 @@ struct merge_tree_options {
 	int mode;
 	int allow_unrelated_histories;
 	int show_messages;
-	int exclude_modes_oids_stages;
+	const char *conflict_format;
+	int unique_conflicts;
+	int abbrev;
 };
 
+struct expand_conflict_data {
+	const char *prefix;
+	struct string_list_item *item;
+	struct strbuf *scratch;
+	int abbrev;
+	struct strbuf *sb_tmp;
+};
+static size_t expand_conflict_format(struct strbuf *sb,
+				     const char *start,
+				     void *context)
+{
+	struct expand_conflict_data *data = context;
+	struct string_list_item *item = data->item;
+	struct stage_info *info = item->util;
+	const char *end;
+	const char *p;
+	size_t len;
+
+	len = strbuf_expand_literal_cb(sb, start, NULL);
+	if (len)
+		return len;
+
+	if (*start != '(')
+		die(_("bad format as of '%s'"), start);
+	end = strchr(start + 1, ')');
+	if (!end)
+		die(_("format element '%s' does not end in ')'"), start);
+	len = end - start + 1;
+
+	if (skip_prefix(start, "(objectmode)", &p)) {
+		strbuf_addf(sb, "%06o", info->mode);
+	} else if (skip_prefix(start, "(objectname)", &p)) {
+		strbuf_addstr(sb, find_unique_abbrev(&info->oid, data->abbrev));
+	} else if (skip_prefix(start, "(stage)", &p)) {
+		strbuf_addf(sb, "%d", info->stage);
+	} else if (skip_prefix(start, "(path)", &p)) {
+		const char *name = item->string;
+
+		if (data->prefix)
+			name = relative_path(name, data->prefix, data->scratch);
+		strbuf_addstr(sb, name);
+
+		strbuf_reset(data->sb_tmp);
+		/* The relative_path() function resets "scratch" */
+
+	} else {
+		unsigned int errlen = (unsigned long)len;
+		die(_("bad format specifier %%%.*s"), errlen, start);
+	}
+
+	return len;
+}
+
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2,
 		      const char *prefix)
@@ -446,23 +501,43 @@ static int real_merge(struct merge_tree_options *o,
 	puts(oid_to_hex(&result.tree->object.oid));
 	if (!result.clean) {
 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
-		const char *last = NULL;
-		int i;
+		struct string_list_item *item;
+		char *last = NULL;
+		struct strbuf sb = STRBUF_INIT;
+		struct strbuf tmp = STRBUF_INIT;
 
 		merge_get_conflicted_files(&result, &conflicted_files);
-		for (i = 0; i < conflicted_files.nr; i++) {
-			const char *name = conflicted_files.items[i].string;
-			struct stage_info *c = conflicted_files.items[i].util;
-			if (!o->exclude_modes_oids_stages)
-				printf("%06o %s %d\t",
-				       c->mode, oid_to_hex(&c->oid), c->stage);
-			else if (last && !strcmp(last, name))
+		for_each_string_list_item(item, &conflicted_files) {
+			struct expand_conflict_data ctx = {
+				.prefix = prefix,
+				.item = item,
+				.abbrev = o->abbrev,
+				.scratch = &sb,
+				.sb_tmp = &tmp,
+			};
+
+			strbuf_expand(&sb, o->conflict_format, expand_conflict_format, &ctx);
+			strbuf_addch(&sb, line_termination);
+
+			if (o->unique_conflicts && last && !strcmp(last, sb.buf)) {
+				free(last);
+				last = strbuf_detach(&sb, NULL);
 				continue;
-			write_name_quoted_relative(
-				name, prefix, stdout, line_termination);
-			last = name;
+			}
+
+			fwrite(sb.buf, sb.len, 1, stdout);
+
+			if (o->unique_conflicts) {
+				free(last);
+				last = strbuf_detach(&sb, NULL);
+			} else {
+				strbuf_reset(&sb);
+			}
 		}
 		string_list_clear(&conflicted_files, 1);
+		strbuf_release(&sb);
+		strbuf_release(&tmp);
+		free(last);
 	}
 	if (o->show_messages) {
 		putchar(line_termination);
@@ -474,7 +549,11 @@ static int real_merge(struct merge_tree_options *o,
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { .show_messages = -1 };
+	struct merge_tree_options o = {
+		.show_messages = -1,
+		.conflict_format = "%(objectmode) %(objectname) %(stage)%x09%(path)",
+		.unique_conflicts = 1,
+	};
 	int expected_remaining_argc;
 	int original_argc;
 
@@ -493,14 +572,15 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			 N_("also show informational/conflict messages")),
 		OPT_SET_INT('z', NULL, &line_termination,
 			    N_("separate paths with the NUL character"), '\0'),
-		OPT_BOOL_F('l', "exclude-modes-oids-stages",
-			   &o.exclude_modes_oids_stages,
-			   N_("list conflicted files without modes/oids/stages"),
-			   PARSE_OPT_NONEG),
+		OPT_STRING(0, "conflict-format", &o.conflict_format, N_("format"),
+			   N_("specify a custom format to use for conflicted files")),
+		OPT_BOOL(0, "unique-conflicts", &o.unique_conflicts,
+			 N_("omit duplicate --conflict-format lines")),
 		OPT_BOOL_F(0, "allow-unrelated-histories",
 			   &o.allow_unrelated_histories,
 			   N_("allow merging unrelated histories"),
 			   PARSE_OPT_NONEG),
+		OPT__ABBREV(&o.abbrev),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 4de089d976d..e6354b2d284 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -93,7 +93,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -115,8 +115,35 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'merge-tree --unique-conflicts is the default' '
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out &&
+	sed 1d <out >actual &&
+	cat >expect <<-\EOF &&
+	greeting
+	whatever~side1
+	EOF
+	test_cmp expect actual &&
+
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out2 &&
+	sed 1d <out2 >actual2 &&
+	test_cmp actual actual2
+'
+
+test_expect_success 'merge-tree --no-unique-conflicts' '
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-unique-conflicts --no-messages side1 side2 >out &&
+	sed 1d <out >actual &&
+	cat >expect <<-\EOF &&
+	greeting
+	greeting
+	greeting
+	whatever~side1
+	whatever~side1
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --conflict-format="%(path)" side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Ævar Arnfjörð Bjarmason]

On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> This modifies the new display_update_messages() function to allow
> printing to somewhere other than stdout.  It also consolidates the
> location of the diff_warn_rename_limit() message with the rest of the
> CONFLICT and other update messages to all go to the same stream.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-ort.c | 9 +++++----
>  merge-ort.h | 3 ++-
>  2 files changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/merge-ort.c b/merge-ort.c
> index 82d2faf5bf9..d28d1721d14 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
>  }
>  
>  void merge_display_update_messages(struct merge_options *opt,
> -				   struct merge_result *result)
> +				   struct merge_result *result,
> +				   FILE *stream)
>  {
>  	struct merge_options_internal *opti = result->priv;
>  	struct hashmap_iter iter;
> @@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
>  	for (i = 0; i < olist.nr; ++i) {
>  		struct strbuf *sb = olist.items[i].util;
>  
> -		printf("%s", sb->buf);
> +		strbuf_write(sb, stream);
>  	}
>  	string_list_clear(&olist, 0);
>  
>  	/* Also include needed rename limit adjustment now */
>  	diff_warn_rename_limit("merge.renamelimit",
> -			       opti->renames.needed_limit, 0, stderr);
> +			       opti->renames.needed_limit, 0, stream);

At the tip of this series I tried to s/stream/stderr/g this, and
t4301-merge-tree-write-tree.sh passes, doesn't this warning_fp() special
behavior need a test somewhere?

I assumed that warning_fp() would be using vreportf() in usage.c, but
it's not, it's just falling back to the equivalent of fprintf(out, ...),
no? I don't really see why 05/15 and parts of 06/15 & this are needed
over a much simpler simple helper macro like the below. applied on top
of this series.

I would get it if the point was to actually use the full usage.c
machinery, but we're just either calling warning(), or printing a
formatted string to a file FILE *. There's no need to go through usage.c
for that, and adding an API to it that behaves like this new
warning_fp() is really confusing.

I.e. an API in usage.c that allowed warning to a given FD would be
trying to replace the "2" in the write_in_full() call in vreportf(), I
would think.

diff --git a/diff.c b/diff.c
index 28368110147..4cf67e93dea 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
+#define warning_fp(out, fmt, ...) do { \
+	if (out == stderr) \
+		warning(fmt, __VA_ARGS__); \
+	else \
+		fprintf(out, fmt, __VA_ARGS__); \
+} while (0)
+
 void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
 			    FILE *out)
 {
 	fflush(stdout);
 	if (degraded_cc)
-		warning_fp(out, _(degrade_cc_to_c_warning));
+		warning_fp(out, _(degrade_cc_to_c_warning), NULL);
 	else if (needed)
-		warning_fp(out, _(rename_limit_warning));
+		warning_fp(out, _(rename_limit_warning), NULL);
 	else
 		return;
 
diff --git a/git-compat-util.h b/git-compat-util.h
index 64ba60e5c71..d70ce142861 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
-void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index 0bfd2c603c0..c7d233b0de9 100644
--- a/usage.c
+++ b/usage.c
@@ -253,20 +253,6 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
-void warning_fp(FILE *out, const char *warn, ...)
-{
-	va_list params;
-
-	va_start(params, warn);
-	if (out == stderr)
-		warn_routine(warn, params);
-	else {
-		vfprintf(out, warn, params);
-		fputc('\n', out);
-	}
-	va_end(params);
-}
-
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Ævar Arnfjörð Bjarmason]

On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.

Maybe that ship has sailed, but just my 0.02: I thought this whole thing
was much less confusing with your initial merge-tree-ort proposal at
https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
I.e. the end-state of merge-tree.c is that you end up reading largely
unrelated code (various static functions only used by one side or
another).

But maybe that's all water under the bridge etc, however...

>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_CMDMODE(0, "write-tree", &o.mode,
> +			    N_("do a real merge instead of a trivial merge"),
> +			    'w'),
> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
> +			    N_("do a trivial merge only"), 't'),
> +		OPT_END()
> +	};
> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +	if (o.mode) {
> +		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.mode = (argc == 2 ? 'w' : 't');
> +	}

Do we really need to make this interface more special-casey by
auto-guessing based on argc what argument you want? I.e. instead of
usage like:

	N_("git merge-tree [--write-tree] <branch1> <branch2>"),
	N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),

Wouldn't it be simpler to just have the equivalent of:

	# old
        git merge-tree ...
        # new
        git merge-tree --new-thing ...

And not have to look at ... to figure out if we're dispatching to the
new or old thing.

Re: [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren]

On Wed, Feb 2, 2022 at 5:07 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Much like `git merge` updates the index with information of the form
> >     (mode, oid, stage, name)
> > provide this output for conflicted files for merge-tree as well.
> > Provide an --exclude-modes-oids-stages/-l option for users to exclude
>
> This.
>
> > +--exclude-oids-and-modes::
>
> No longer matches this.

Thanks; good catch.

Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren]

On Wed, Feb 2, 2022 at 5:22 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren wrote:
>
> > On Wed, Feb 2, 2022 at 1:32 PM Junio C Hamano <gitster@pobox.com> wrote:
> >>
> >> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
> >>
> >> > @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
> >> >               merge_get_conflicted_files(&result, &conflicted_files);
> >> >               for (i = 0; i < conflicted_files.nr; i++) {
> >> >                       const char *name = conflicted_files.items[i].string;
> >> > -                     if (last && !strcmp(last, name))
> >> > +                     struct stage_info *c = conflicted_files.items[i].util;
> >> > +                     if (!o->exclude_modes_oids_stages)
> >> > +                             printf("%06o %s %d\t",
> >> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
> >> > +                     else if (last && !strcmp(last, name))
> >> >                               continue;
> >> >                       write_name_quoted_relative(
> >> >                               name, prefix, stdout, line_termination);
> >>
> >> OK.  The addition (and disabling of the deduping) is quite trivial.
> >> We do not even have to worry about line termination since the extra
> >> pieces of info are prepended to the pathname.  Nice.
> >>
> >> > @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >> >                           N_("do a trivial merge only"), 't'),
> >> >               OPT_BOOL(0, "messages", &o.show_messages,
> >> >                        N_("also show informational/conflict messages")),
> >> > +             OPT_BOOL_F('l', "exclude-modes-oids-stages",
> >> > +                        &o.exclude_modes_oids_stages,
> >> > +                        N_("list conflicted files without modes/oids/stages"),
> >> > +                        PARSE_OPT_NONEG),
> >>
> >> Why does "-l" give shorter output than without it?  "-l" strongly
> >> hints a longer output than without, at least to me.  Just wondering
> >> if this will not become a source of confusion to future scripting
> >> users.
> >
> > Here's another example where I was struggling with naming.  Something
> > like ls-tree's `--name-only` would have been nice, but I was worried
> > it'd be confusing since it only affected the conflicted info section
> > and does not suppress the printing of the toplevel tree or the
> > informational messages sections.  And the name
> > --exclude-modes-oids-stages was long enough that I wanted a short flag
> > for it, and just used the first letter of the description ("list
> > conflicted files...").  I'm happy to change either the long or the
> > short name for this flag if anyone has suggestions.
>
> There's always sidestepping it by replacing it with a --format :)

Another solution that occurred to me, and I was _really_ close to
doing it for v3, was to just flat drop this patch entirely and not
include any such option.  But...

  * "Which files had conflicts?" seems like such an obvious question
  * I've used `git ls-files -u | awk {print\$4} | uniq` a lot in the
past after `git merge` (Or `git rebase`) to get this info (yeah, it
turns out `git diff --name-only --diff-filter=U` is 4 fewer
characters)
  * "display the list of files where conflicts were present in the web
UI" was listed as an early usecase[1]

[1] https://lore.kernel.org/git/YYlqpuzv+bmZaFzz@nand.local/

So it seemed like making that question easy to answer was worthwhile.

> Anyway, I'd mentioned that in an earlier review in
> <220124.864k5tigto.gmgdl@evledraar.gmail.com>. FWIW here's an experiment
> to do that that I polished up (mostly copied from the ls-tree WIP code
> I'd written already).
>
> I don't know if it will ever be useful, or if you think it's
> worthwhile/simpler, but in either case I think in doing this I spotted
> the following issues or otherwise noted inconsistencies in the pre-image:
>
>    The docs say that "<stage> <path>" is SP-separated, but it's
>    actually TAB-separated, the rest is SP-separated.

Yeah, good catch.  However, it doesn't actually say they are
SP-separated; it's ambiguous about the spacing.  Which probably isn't
a good thing, but it was kind of copied from the ls-files manual:

"""
       git ls-files just outputs the filenames unless --stage is specified in
       which case it outputs:

           [<tag> ]<mode> <object> <stage> <file>
"""

(which also uses a tab between <stage> and <file> and a space
otherwise, but the output above may lead you to believe otherwise.)

>  * That you de-dupe --exclude-modes-oids-stages is a bit of a hidden feature,
>    but argubly initiative. Should it by optional? In any case my formatting
>    experiment makes it optional, since it then needs to be generalized to de-dupe
>    after we've formatted.

I think without de-duping the flag isn't helpful enough to bother
implementing.  Requiring two flags also seems painful, given the
common case scenario.

I hope I'm not coming across as dismissive.  I think eventually adding
a --format and --dedupe (the combination of which might be implied by
whatever flag is used now) might be useful additions.  Maybe --abbrev
too...eventually.  But I'm worried that it's distracting from focusing
on usecases.  In particular, I'm worried it leads to "well, script
writers technically can get what they want because we provided
everything" rather than focusing on making the most common things easy
to get, and then extending the command for flexibility as needed
later.

I'd really rather that early versions _just_ focus on actual usecases
as far as UI is concerned (and thus I was really happy to see Dscho
and Taylor concentrate on that side; I think Christian might have been
talking about that angle some but it was hard to differentiate from
the "merge-tree on steroids" spitballing).  While I want to be careful
to avoid preventing UI flexibility, I think building it in from the
beginning tends to lead to a design that is less usable.  (e.g. the
possible loss of de-duping that would naturally have arisen from
looking at things from the other angle.)  It's just a bias I have.

>  * Perhaps we should support --abbrev as ls-tree does? The below diff shows
>    it's easy enough.

This one is less problematic to me, but I'd still rather that the UI
side of things focused on the usecases for early versions.

>  * The dance you have with sed-ing out the hash in the tests could be made much
>    easier with "sed 1d <out >actual" and --no-messages for some existing tests.

Ignoring the first line is semantically different than verifying it
looks like a hash.  It also only works on the first line, and hashes
appear in multiple places, so you'd need a variety of different sed
commands for different parts of the output, which doesn't seem any
easier at all to me; I think using the same replacement everywhere is
simpler.  But perhaps I should turn it into a shell function that I
use in each case.

> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 6a2ed475106..e906d1dc9bf 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -44,10 +44,9 @@ OPTIONS
>         newline.  Also begin the messages section with a NUL character
>         instead of a newline.  See OUTPUT below for more information.
>
> ---exclude-oids-and-modes::
> -       Instead of writing a list of (mode, oid, stage, path) tuples
> -       to output for conflicted files, just provide a list of
> -       filenames with conflicts.
> +--conflict-format::
> +       Override the default "%(objectmode) %(objectname)
> +       %(stage)%x09%(path)" format.
>
>  --[no-]messages::
>         Write any informational messages such as "Auto-merging <path>"
> @@ -89,13 +88,13 @@ Conflicted file info
>
>  This is a sequence of lines with the format
>
> -       <mode> <object> <stage> <filename>
> +       %(objectmode) %(objectname) %(stage)%x09%(path)
>
>  The filename will be quoted as explained for the configuration
> -variable `core.quotePath` (see linkgit:git-config[1]).  However, if
> -the `--exclude-oids-and-modes` option is passed, the mode, object, and
> -stage will be omitted.  If `-z` is passed, the "lines" are terminated
> -by a NUL character instead of a newline character.
> +variable `core.quotePath` (see linkgit:git-config[1]).
> +
> +If `-z` is passed, the "lines" are terminated by a NUL character
> +instead of a newline character.
>
>  Informational messages
>  ~~~~~~~~~~~~~~~~~~~~~~
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 58c0ddc5a32..14fed95a8ce 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -395,9 +395,64 @@ struct merge_tree_options {
>         int mode;
>         int allow_unrelated_histories;
>         int show_messages;
> -       int exclude_modes_oids_stages;
> +       const char *conflict_format;
> +       int unique_conflicts;
> +       int abbrev;
>  };
>
> +struct expand_conflict_data {
> +       const char *prefix;
> +       struct string_list_item *item;
> +       struct strbuf *scratch;
> +       int abbrev;
> +       struct strbuf *sb_tmp;
> +};
> +static size_t expand_conflict_format(struct strbuf *sb,
> +                                    const char *start,
> +                                    void *context)
> +{
> +       struct expand_conflict_data *data = context;
> +       struct string_list_item *item = data->item;
> +       struct stage_info *info = item->util;
> +       const char *end;
> +       const char *p;
> +       size_t len;
> +
> +       len = strbuf_expand_literal_cb(sb, start, NULL);
> +       if (len)
> +               return len;
> +
> +       if (*start != '(')
> +               die(_("bad format as of '%s'"), start);
> +       end = strchr(start + 1, ')');
> +       if (!end)
> +               die(_("format element '%s' does not end in ')'"), start);
> +       len = end - start + 1;
> +
> +       if (skip_prefix(start, "(objectmode)", &p)) {
> +               strbuf_addf(sb, "%06o", info->mode);
> +       } else if (skip_prefix(start, "(objectname)", &p)) {
> +               strbuf_addstr(sb, find_unique_abbrev(&info->oid, data->abbrev));
> +       } else if (skip_prefix(start, "(stage)", &p)) {
> +               strbuf_addf(sb, "%d", info->stage);
> +       } else if (skip_prefix(start, "(path)", &p)) {
> +               const char *name = item->string;
> +
> +               if (data->prefix)
> +                       name = relative_path(name, data->prefix, data->scratch);
> +               strbuf_addstr(sb, name);
> +
> +               strbuf_reset(data->sb_tmp);
> +               /* The relative_path() function resets "scratch" */
> +
> +       } else {
> +               unsigned int errlen = (unsigned long)len;
> +               die(_("bad format specifier %%%.*s"), errlen, start);
> +       }
> +
> +       return len;
> +}
> +
>  static int real_merge(struct merge_tree_options *o,
>                       const char *branch1, const char *branch2,
>                       const char *prefix)
> @@ -446,23 +501,43 @@ static int real_merge(struct merge_tree_options *o,
>         puts(oid_to_hex(&result.tree->object.oid));
>         if (!result.clean) {
>                 struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> -               const char *last = NULL;
> -               int i;
> +               struct string_list_item *item;
> +               char *last = NULL;
> +               struct strbuf sb = STRBUF_INIT;
> +               struct strbuf tmp = STRBUF_INIT;
>
>                 merge_get_conflicted_files(&result, &conflicted_files);
> -               for (i = 0; i < conflicted_files.nr; i++) {
> -                       const char *name = conflicted_files.items[i].string;
> -                       struct stage_info *c = conflicted_files.items[i].util;
> -                       if (!o->exclude_modes_oids_stages)
> -                               printf("%06o %s %d\t",
> -                                      c->mode, oid_to_hex(&c->oid), c->stage);
> -                       else if (last && !strcmp(last, name))
> +               for_each_string_list_item(item, &conflicted_files) {
> +                       struct expand_conflict_data ctx = {
> +                               .prefix = prefix,
> +                               .item = item,
> +                               .abbrev = o->abbrev,
> +                               .scratch = &sb,
> +                               .sb_tmp = &tmp,
> +                       };
> +
> +                       strbuf_expand(&sb, o->conflict_format, expand_conflict_format, &ctx);
> +                       strbuf_addch(&sb, line_termination);
> +
> +                       if (o->unique_conflicts && last && !strcmp(last, sb.buf)) {
> +                               free(last);
> +                               last = strbuf_detach(&sb, NULL);
>                                 continue;
> -                       write_name_quoted_relative(
> -                               name, prefix, stdout, line_termination);
> -                       last = name;
> +                       }
> +
> +                       fwrite(sb.buf, sb.len, 1, stdout);
> +
> +                       if (o->unique_conflicts) {
> +                               free(last);
> +                               last = strbuf_detach(&sb, NULL);
> +                       } else {
> +                               strbuf_reset(&sb);
> +                       }
>                 }
>                 string_list_clear(&conflicted_files, 1);
> +               strbuf_release(&sb);
> +               strbuf_release(&tmp);
> +               free(last);
>         }
>         if (o->show_messages) {
>                 putchar(line_termination);
> @@ -474,7 +549,11 @@ static int real_merge(struct merge_tree_options *o,
>
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -       struct merge_tree_options o = { .show_messages = -1 };
> +       struct merge_tree_options o = {
> +               .show_messages = -1,
> +               .conflict_format = "%(objectmode) %(objectname) %(stage)%x09%(path)",
> +               .unique_conflicts = 1,
> +       };
>         int expected_remaining_argc;
>         int original_argc;
>
> @@ -493,14 +572,15 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>                          N_("also show informational/conflict messages")),
>                 OPT_SET_INT('z', NULL, &line_termination,
>                             N_("separate paths with the NUL character"), '\0'),
> -               OPT_BOOL_F('l', "exclude-modes-oids-stages",
> -                          &o.exclude_modes_oids_stages,
> -                          N_("list conflicted files without modes/oids/stages"),
> -                          PARSE_OPT_NONEG),
> +               OPT_STRING(0, "conflict-format", &o.conflict_format, N_("format"),
> +                          N_("specify a custom format to use for conflicted files")),
> +               OPT_BOOL(0, "unique-conflicts", &o.unique_conflicts,
> +                        N_("omit duplicate --conflict-format lines")),

The latter of which you didn't include in the manual?  Also,
unique_conflicts seems like something that is trivial to understand
from the coding perspective, but probably require quite a bit more
explanation from the manual.  For example, if objectname is included
in the format, unique-conflicts is essentially a no-op.  And that's
the default...so, you'd probably have to spend time in the manual
explaining under what circumstances it's useful.  I'm also not sure if
a user who wanted (mode, path) would want unique_conflicts to default
to 1; it may be something only meaningful in the particular case of
"just give me conflicted filenames".

>                 OPT_BOOL_F(0, "allow-unrelated-histories",
>                            &o.allow_unrelated_histories,
>                            N_("allow merging unrelated histories"),
>                            PARSE_OPT_NONEG),
> +               OPT__ABBREV(&o.abbrev),
>                 OPT_END()
>         };
>
> diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
> index 4de089d976d..e6354b2d284 100755
> --- a/t/t4301-merge-tree-write-tree.sh
> +++ b/t/t4301-merge-tree-write-tree.sh
> @@ -93,7 +93,7 @@ test_expect_success 'Barf on too many arguments' '
>  '
>
>  test_expect_success 'test conflict notices and such' '
> -       test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" side1 side2 >out &&
>         sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
>
>         # Expected results:
> @@ -115,8 +115,35 @@ test_expect_success 'test conflict notices and such' '
>         test_cmp expect actual
>  '
>
> +test_expect_success 'merge-tree --unique-conflicts is the default' '
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out &&
> +       sed 1d <out >actual &&
> +       cat >expect <<-\EOF &&
> +       greeting
> +       whatever~side1
> +       EOF
> +       test_cmp expect actual &&
> +
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out2 &&
> +       sed 1d <out2 >actual2 &&
> +       test_cmp actual actual2
> +'
> +
> +test_expect_success 'merge-tree --no-unique-conflicts' '
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-unique-conflicts --no-messages side1 side2 >out &&
> +       sed 1d <out >actual &&
> +       cat >expect <<-\EOF &&
> +       greeting
> +       greeting
> +       greeting
> +       whatever~side1
> +       whatever~side1
> +       EOF
> +       test_cmp expect actual
> +'
> +
>  test_expect_success 'Just the conflicted files without the messages' '
> -       test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
> +       test_expect_code 1 git merge-tree --write-tree --no-messages --conflict-format="%(path)" side1 side2 >out &&
>         sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
>
>         test_write_lines HASH greeting whatever~side1 >expect &&

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren]

On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Let merge-tree accept a `--write-tree` parameter for choosing real
> > merges instead of trivial merges, and accept an optional
> > `--trivial-merge` option to get the traditional behavior.  Note that
> > these accept different numbers of arguments, though, so these names
> > need not actually be used.
>
> Maybe that ship has sailed, but just my 0.02: I thought this whole thing
> was much less confusing with your initial merge-tree-ort proposal at
> https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
> I.e. the end-state of merge-tree.c is that you end up reading largely
> unrelated code (various static functions only used by one side or
> another).

Christian's merge-tree-ort proposal?

> But maybe that's all water under the bridge etc, however...
>
> >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >  {
> > -     if (argc != 4)
> > -             usage(merge_tree_usage);
> > -     return trivial_merge(argc, argv);
> > +     struct merge_tree_options o = { 0 };
> > +     int expected_remaining_argc;
> > +
> > +     const char * const merge_tree_usage[] = {
> > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> > +             NULL
> > +     };
> > +     struct option mt_options[] = {
> > +             OPT_CMDMODE(0, "write-tree", &o.mode,
> > +                         N_("do a real merge instead of a trivial merge"),
> > +                         'w'),
> > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
> > +                         N_("do a trivial merge only"), 't'),
> > +             OPT_END()
> > +     };
> > +
> > +     /* Parse arguments */
> > +     argc = parse_options(argc, argv, prefix, mt_options,
> > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> > +     if (o.mode) {
> > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> > +             if (argc != expected_remaining_argc)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +     } else {
> > +             if (argc < 2 || argc > 3)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +             o.mode = (argc == 2 ? 'w' : 't');
> > +     }
>
> Do we really need to make this interface more special-casey by
> auto-guessing based on argc what argument you want? I.e. instead of
> usage like:
>
>         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>
> Wouldn't it be simpler to just have the equivalent of:
>
>         # old
>         git merge-tree ...
>         # new
>         git merge-tree --new-thing ...
>
> And not have to look at ... to figure out if we're dispatching to the
> new or old thing.

You seem to be focusing on code simplicity?  Sure, that'd be simpler
code, it'd just be a less useful feature.

I think passing --write-tree all the time would be an annoyance.  I
don't see why anyone would ever use the other mode.  However, for as
long as both exist in the manual, it makes the manual easier to
explain to users, and example testcases more self-documenting by
having the flag there.  That's the sole purpose of the flag.

I'm never going to actually use it when I invoke it from the command
line.  And I suspect most would leave it off.

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Elijah Newren]

On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > This modifies the new display_update_messages() function to allow
> > printing to somewhere other than stdout.  It also consolidates the
> > location of the diff_warn_rename_limit() message with the rest of the
> > CONFLICT and other update messages to all go to the same stream.
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  merge-ort.c | 9 +++++----
> >  merge-ort.h | 3 ++-
> >  2 files changed, 7 insertions(+), 5 deletions(-)
> >
> > diff --git a/merge-ort.c b/merge-ort.c
> > index 82d2faf5bf9..d28d1721d14 100644
> > --- a/merge-ort.c
> > +++ b/merge-ort.c
> > @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
> >  }
> >
> >  void merge_display_update_messages(struct merge_options *opt,
> > -                                struct merge_result *result)
> > +                                struct merge_result *result,
> > +                                FILE *stream)
> >  {
> >       struct merge_options_internal *opti = result->priv;
> >       struct hashmap_iter iter;
> > @@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
> >       for (i = 0; i < olist.nr; ++i) {
> >               struct strbuf *sb = olist.items[i].util;
> >
> > -             printf("%s", sb->buf);
> > +             strbuf_write(sb, stream);
> >       }
> >       string_list_clear(&olist, 0);
> >
> >       /* Also include needed rename limit adjustment now */
> >       diff_warn_rename_limit("merge.renamelimit",
> > -                            opti->renames.needed_limit, 0, stderr);
> > +                            opti->renames.needed_limit, 0, stream);
>
> At the tip of this series I tried to s/stream/stderr/g this, and
> t4301-merge-tree-write-tree.sh passes, doesn't this warning_fp() special
> behavior need a test somewhere?

That's a fair point, but...this gets back to my cover letter comments
about patches 5, 6, and 8.  They implement a code feature that seems
useful in general...but which Dscho and Christian didn't like using in
this particular command; they just wanted all output on stdout.

So, it's hard to add a test, because there's no code anywhere that
exercises it in this series anymore.  I originally wanted this feature
in my remerge-diff series, but the idea of conflict headers made me
punt it to this series.  I wanted it for this series, but Dscho and
Christian didn't.  I could have punted again, but decided the
underlying want kept coming up and decided to not excise it --
especially since Dscho was helping improve it.  And Junio commented
that he liked the idea[1].

[1] https://lore.kernel.org/git/xmqqh79hx8g1.fsf@gitster.g/

But yeah, it does leave it feeling slightly odd that we implemented a
feature that nothing is currently using.  Maybe these 3 should be
split off into their own series?  Still wouldn't have a test yet,
though.

> I assumed that warning_fp() would be using vreportf() in usage.c, but
> it's not, it's just falling back to the equivalent of fprintf(out, ...),
> no? I don't really see why 05/15 and parts of 06/15 & this are needed
> over a much simpler simple helper macro like the below. applied on top
> of this series.

That macro is simple?  I thought I basically understood Dscho's code,
but looking at what you did with diff_warn_rename_limit(), I think I'm
lost.

> I would get it if the point was to actually use the full usage.c
> machinery, but we're just either calling warning(), or printing a
> formatted string to a file FILE *. There's no need to go through usage.c
> for that, and adding an API to it that behaves like this new
> warning_fp() is really confusing.

Because the formatted string being printed to the file won't have the
same "warning: " prefix that is normally added to stuff in usage?

That's a fair point; that does have a bit of a consistency problem.
And I'd rather the messages were consistent regardless of where they
are printed.

> I.e. an API in usage.c that allowed warning to a given FD would be
> trying to replace the "2" in the write_in_full() call in vreportf(), I
> would think.

Hmm, makes sense.

> diff --git a/diff.c b/diff.c
> index 28368110147..4cf67e93dea 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
>  N_("you may want to set your %s variable to at least "
>     "%d and retry the command.");
>
> +#define warning_fp(out, fmt, ...) do { \
> +       if (out == stderr) \
> +               warning(fmt, __VA_ARGS__); \
> +       else \
> +               fprintf(out, fmt, __VA_ARGS__); \
> +} while (0)
> +
>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
>                             FILE *out)
>  {
>         fflush(stdout);
>         if (degraded_cc)
> -               warning_fp(out, _(degrade_cc_to_c_warning));
> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
>         else if (needed)
> -               warning_fp(out, _(rename_limit_warning));
> +               warning_fp(out, _(rename_limit_warning), NULL);

Why do the only callers have a NULL parameter here?  Is this one of
those va_list/va_args things I never bothered to properly learn?

>         else
>                 return;
>
> diff --git a/git-compat-util.h b/git-compat-util.h
> index 64ba60e5c71..d70ce142861 100644
> --- a/git-compat-util.h
> +++ b/git-compat-util.h
> @@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
>  int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
>  void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
>  void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
> -void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
>
>  #ifndef NO_OPENSSL
>  #ifdef APPLE_COMMON_CRYPTO
> diff --git a/usage.c b/usage.c
> index 0bfd2c603c0..c7d233b0de9 100644
> --- a/usage.c
> +++ b/usage.c
> @@ -253,20 +253,6 @@ void warning(const char *warn, ...)
>         va_end(params);
>  }
>
> -void warning_fp(FILE *out, const char *warn, ...)
> -{
> -       va_list params;
> -
> -       va_start(params, warn);
> -       if (out == stderr)
> -               warn_routine(warn, params);
> -       else {
> -               vfprintf(out, warn, params);
> -               fputc('\n', out);
> -       }
> -       va_end(params);
> -}
> -
>  /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
>  int BUG_exit_code;

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren]

On Thu, Feb 3, 2022 at 1:04 AM Elijah Newren <newren@gmail.com> wrote:
>
> On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >
> > On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
> >
> > > From: Elijah Newren <newren@gmail.com>
> > >
> > > Let merge-tree accept a `--write-tree` parameter for choosing real
> > > merges instead of trivial merges, and accept an optional
> > > `--trivial-merge` option to get the traditional behavior.  Note that
> > > these accept different numbers of arguments, though, so these names
> > > need not actually be used.
> >
> > Maybe that ship has sailed, but just my 0.02: I thought this whole thing
> > was much less confusing with your initial merge-tree-ort proposal at
> > https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
> > I.e. the end-state of merge-tree.c is that you end up reading largely
> > unrelated code (various static functions only used by one side or
> > another).
>
> Christian's merge-tree-ort proposal?
>
> > But maybe that's all water under the bridge etc, however...
> >
> > >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> > >  {
> > > -     if (argc != 4)
> > > -             usage(merge_tree_usage);
> > > -     return trivial_merge(argc, argv);
> > > +     struct merge_tree_options o = { 0 };
> > > +     int expected_remaining_argc;
> > > +
> > > +     const char * const merge_tree_usage[] = {
> > > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> > > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> > > +             NULL
> > > +     };
> > > +     struct option mt_options[] = {
> > > +             OPT_CMDMODE(0, "write-tree", &o.mode,
> > > +                         N_("do a real merge instead of a trivial merge"),
> > > +                         'w'),
> > > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
> > > +                         N_("do a trivial merge only"), 't'),
> > > +             OPT_END()
> > > +     };
> > > +
> > > +     /* Parse arguments */
> > > +     argc = parse_options(argc, argv, prefix, mt_options,
> > > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> > > +     if (o.mode) {
> > > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> > > +             if (argc != expected_remaining_argc)
> > > +                     usage_with_options(merge_tree_usage, mt_options);
> > > +     } else {
> > > +             if (argc < 2 || argc > 3)
> > > +                     usage_with_options(merge_tree_usage, mt_options);
> > > +             o.mode = (argc == 2 ? 'w' : 't');
> > > +     }
> >
> > Do we really need to make this interface more special-casey by
> > auto-guessing based on argc what argument you want? I.e. instead of
> > usage like:
> >
> >         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> >         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> >
> > Wouldn't it be simpler to just have the equivalent of:
> >
> >         # old
> >         git merge-tree ...
> >         # new
> >         git merge-tree --new-thing ...
> >
> > And not have to look at ... to figure out if we're dispatching to the
> > new or old thing.
>
> You seem to be focusing on code simplicity?  Sure, that'd be simpler
> code, it'd just be a less useful feature.
>
> I think passing --write-tree all the time would be an annoyance.  I
> don't see why anyone would ever use the other mode.  However, for as
> long as both exist in the manual, it makes the manual easier to
> explain to users, and example testcases more self-documenting by
> having the flag there.  That's the sole purpose of the flag.
>
> I'm never going to actually use it when I invoke it from the command
> line.  And I suspect most would leave it off.

...also, even if we did require the `--write-tree` flag, we'd still
have to look at argc.  Since the option parsing handles both modes,
someone could leave off --write-tree, but include a bunch of other
options that only make sense with --write-tree.  Individually checking
the setting of every extra flag along with write_tree is a royal pain
and I don't want to repeat that for each new option added.  Simply
checking argc allows you to provide an error message if the user does
that.

(And I think it's sad that in Git we often forgot to warn and notify
users of options that are only functional with certain other
arguments; it makes it harder for users to figure out, and has in the
past even made it harder for other developers to figure out what was
meant and how things are to be used.  I think I've seen multiple Git
devs be confused over ls-files --directory and --no-empty-directory
options, assuming they'd do something sensible for tracked files, when
in fact those arguments are simply ignored because they are only
modifiers for how untracked files are treated.)

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 03 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 1:04 AM Elijah Newren <newren@gmail.com> wrote:
>>
>> On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> >
>> > On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>> >
>> > > From: Elijah Newren <newren@gmail.com>
>> > >
>> > > Let merge-tree accept a `--write-tree` parameter for choosing real
>> > > merges instead of trivial merges, and accept an optional
>> > > `--trivial-merge` option to get the traditional behavior.  Note that
>> > > these accept different numbers of arguments, though, so these names
>> > > need not actually be used.
>> >
>> > Maybe that ship has sailed, but just my 0.02: I thought this whole thing
>> > was much less confusing with your initial merge-tree-ort proposal at
>> > https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
>> > I.e. the end-state of merge-tree.c is that you end up reading largely
>> > unrelated code (various static functions only used by one side or
>> > another).
>>
>> Christian's merge-tree-ort proposal?
>>
>> > But maybe that's all water under the bridge etc, however...
>> >
>> > >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>> > >  {
>> > > -     if (argc != 4)
>> > > -             usage(merge_tree_usage);
>> > > -     return trivial_merge(argc, argv);
>> > > +     struct merge_tree_options o = { 0 };
>> > > +     int expected_remaining_argc;
>> > > +
>> > > +     const char * const merge_tree_usage[] = {
>> > > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> > > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> > > +             NULL
>> > > +     };
>> > > +     struct option mt_options[] = {
>> > > +             OPT_CMDMODE(0, "write-tree", &o.mode,
>> > > +                         N_("do a real merge instead of a trivial merge"),
>> > > +                         'w'),
>> > > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
>> > > +                         N_("do a trivial merge only"), 't'),
>> > > +             OPT_END()
>> > > +     };
>> > > +
>> > > +     /* Parse arguments */
>> > > +     argc = parse_options(argc, argv, prefix, mt_options,
>> > > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
>> > > +     if (o.mode) {
>> > > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
>> > > +             if (argc != expected_remaining_argc)
>> > > +                     usage_with_options(merge_tree_usage, mt_options);
>> > > +     } else {
>> > > +             if (argc < 2 || argc > 3)
>> > > +                     usage_with_options(merge_tree_usage, mt_options);
>> > > +             o.mode = (argc == 2 ? 'w' : 't');
>> > > +     }
>> >
>> > Do we really need to make this interface more special-casey by
>> > auto-guessing based on argc what argument you want? I.e. instead of
>> > usage like:
>> >
>> >         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> >         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> >
>> > Wouldn't it be simpler to just have the equivalent of:
>> >
>> >         # old
>> >         git merge-tree ...
>> >         # new
>> >         git merge-tree --new-thing ...
>> >
>> > And not have to look at ... to figure out if we're dispatching to the
>> > new or old thing.
>>
>> You seem to be focusing on code simplicity?  Sure, that'd be simpler
>> code, it'd just be a less useful feature.
>>
>> I think passing --write-tree all the time would be an annoyance.  I
>> don't see why anyone would ever use the other mode.  However, for as
>> long as both exist in the manual, it makes the manual easier to
>> explain to users, and example testcases more self-documenting by
>> having the flag there.  That's the sole purpose of the flag.
>>
>> I'm never going to actually use it when I invoke it from the command
>> line.  And I suspect most would leave it off.
>
> ...also, even if we did require the `--write-tree` flag, we'd still
> have to look at argc.  Since the option parsing handles both modes,
> someone could leave off --write-tree, but include a bunch of other
> options that only make sense with --write-tree.  Individually checking
> the setting of every extra flag along with write_tree is a royal pain
> and I don't want to repeat that for each new option added.  Simply
> checking argc allows you to provide an error message if the user does
> that.
>
> (And I think it's sad that in Git we often forgot to warn and notify
> users of options that are only functional with certain other
> arguments; it makes it harder for users to figure out, and has in the
> past even made it harder for other developers to figure out what was
> meant and how things are to be used.  I think I've seen multiple Git
> devs be confused over ls-files --directory and --no-empty-directory
> options, assuming they'd do something sensible for tracked files, when
> in fact those arguments are simply ignored because they are only
> modifiers for how untracked files are treated.)

There's a much simpler way to do what you're trying to do here which is
to only parse --write-tree, and as soon as you have that pass off two
one function or the other, and have those functions call
parse_options().

This is how builtin/{bundle,stash,commit-graph}.c etc. solve the same
problem. It avoids having to the sort of check you did in 09/15:

	+	if (o.mode == 't' && original_argc < argc)
	+		die(_("--trivial-merge is incompatible with all other options"));

The builtin/submodule--helper.c then does it with a first argument that
--looks-like-an-option, but is really the same sort of sub-command
selector.

Which, at least for me brings this back to liking your merge-tree-ort
(or merge-tree-ng, merge-tree-new or whatever) better. I.e. both for the
code & manual we effectively have two completely different commands here
anyway. Splitting them up would make both the code simpler, and also
what we have to explain to users.

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 03 2022, Elijah Newren wrote:

> On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>>
>> > From: Elijah Newren <newren@gmail.com>
>> >
>> > This modifies the new display_update_messages() function to allow
>> > printing to somewhere other than stdout.  It also consolidates the
>> > location of the diff_warn_rename_limit() message with the rest of the
>> > CONFLICT and other update messages to all go to the same stream.
>> >
>> > Signed-off-by: Elijah Newren <newren@gmail.com>
>> > ---
>> >  merge-ort.c | 9 +++++----
>> >  merge-ort.h | 3 ++-
>> >  2 files changed, 7 insertions(+), 5 deletions(-)
>> >
>> > diff --git a/merge-ort.c b/merge-ort.c
>> > index 82d2faf5bf9..d28d1721d14 100644
>> > --- a/merge-ort.c
>> > +++ b/merge-ort.c
>> > @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
>> >  }
>> >
>> >  void merge_display_update_messages(struct merge_options *opt,
>> > -                                struct merge_result *result)
>> > +                                struct merge_result *result,
>> > +                                FILE *stream)
>> >  {
>> >       struct merge_options_internal *opti = result->priv;
>> >       struct hashmap_iter iter;
>> > @@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
>> >       for (i = 0; i < olist.nr; ++i) {
>> >               struct strbuf *sb = olist.items[i].util;
>> >
>> > -             printf("%s", sb->buf);
>> > +             strbuf_write(sb, stream);
>> >       }
>> >       string_list_clear(&olist, 0);
>> >
>> >       /* Also include needed rename limit adjustment now */
>> >       diff_warn_rename_limit("merge.renamelimit",
>> > -                            opti->renames.needed_limit, 0, stderr);
>> > +                            opti->renames.needed_limit, 0, stream);
>>
>> At the tip of this series I tried to s/stream/stderr/g this, and
>> t4301-merge-tree-write-tree.sh passes, doesn't this warning_fp() special
>> behavior need a test somewhere?
>
> That's a fair point, but...this gets back to my cover letter comments
> about patches 5, 6, and 8.  They implement a code feature that seems
> useful in general...but which Dscho and Christian didn't like using in
> this particular command; they just wanted all output on stdout.
>
> So, it's hard to add a test, because there's no code anywhere that
> exercises it in this series anymore.  I originally wanted this feature
> in my remerge-diff series, but the idea of conflict headers made me
> punt it to this series.  I wanted it for this series, but Dscho and
> Christian didn't.  I could have punted again, but decided the
> underlying want kept coming up and decided to not excise it --
> especially since Dscho was helping improve it.  And Junio commented
> that he liked the idea[1].
>
> [1] https://lore.kernel.org/git/xmqqh79hx8g1.fsf@gitster.g/
>
> But yeah, it does leave it feeling slightly odd that we implemented a
> feature that nothing is currently using.  Maybe these 3 should be
> split off into their own series?  Still wouldn't have a test yet,
> though.
>
>> I assumed that warning_fp() would be using vreportf() in usage.c, but
>> it's not, it's just falling back to the equivalent of fprintf(out, ...),
>> no? I don't really see why 05/15 and parts of 06/15 & this are needed
>> over a much simpler simple helper macro like the below. applied on top
>> of this series.
>
> That macro is simple?  I thought I basically understood Dscho's code,
> but looking at what you did with diff_warn_rename_limit(), I think I'm
> lost.

I guess that's a matter of taste, yeah it's a bit of macro soup if
you're not familiar with it. FWIW (sans bug I noted below) it's the
macro soup we already use for other functions in usage.c.

>> I would get it if the point was to actually use the full usage.c
>> machinery, but we're just either calling warning(), or printing a
>> formatted string to a file FILE *. There's no need to go through usage.c
>> for that, and adding an API to it that behaves like this new
>> warning_fp() is really confusing.
>
> Because the formatted string being printed to the file won't have the
> same "warning: " prefix that is normally added to stuff in usage?

But the pre-image doesn't add that either. We're just calling
vfprintf(), not our own vreportf().

> That's a fair point; that does have a bit of a consistency problem.
> And I'd rather the messages were consistent regardless of where they
> are printed.

I think that makes sense, that's why I added die_message() recently. If
you meant to print a "warning: " prefix I think it would also be fine in
this case to just do it inline. See prior art at:

    git grep '"(fatal|error|warning):' -- '*.c'

>> I.e. an API in usage.c that allowed warning to a given FD would be
>> trying to replace the "2" in the write_in_full() call in vreportf(), I
>> would think.
>
> Hmm, makes sense.

The reason I'm barking up this particular tree is that I've got some
upcoming patches for usage.c (the C99-only macro series):
https://lore.kernel.org/git/RFC-cover-00.21-00000000000-20211115T220831Z-avarab@gmail.com/

It would need to deal with anything in the API. In this case there's not
much to deal with, since it's really not at all using the rest of
usage.c, it's just a "or to stderr".

>> diff --git a/diff.c b/diff.c
>> index 28368110147..4cf67e93dea 100644
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
>>  N_("you may want to set your %s variable to at least "
>>     "%d and retry the command.");
>>
>> +#define warning_fp(out, fmt, ...) do { \
>> +       if (out == stderr) \
>> +               warning(fmt, __VA_ARGS__); \
>> +       else \
>> +               fprintf(out, fmt, __VA_ARGS__); \
>> +} while (0)
>> +
>>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
>>                             FILE *out)
>>  {
>>         fflush(stdout);
>>         if (degraded_cc)
>> -               warning_fp(out, _(degrade_cc_to_c_warning));
>> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
>>         else if (needed)
>> -               warning_fp(out, _(rename_limit_warning));
>> +               warning_fp(out, _(rename_limit_warning), NULL);
>
> Why do the only callers have a NULL parameter here?  Is this one of
> those va_list/va_args things I never bothered to properly learn?

That's wrong (I blame tiredness last night),an actual working version is
produced below. Clang accepted my broken code, but gcc rightly yells
about it:

diff --git a/diff.c b/diff.c
index 28368110147..a2bc2595533 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,6 +6377,13 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
+#define warning_fp(out, ...) do { \
+	if (out == stderr) \
+		warning(__VA_ARGS__); \
+	else \
+		fprintf(out, __VA_ARGS__); \
+} while (0)
+
 void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
 			    FILE *out)
 {
diff --git a/git-compat-util.h b/git-compat-util.h
index 64ba60e5c71..d70ce142861 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
-void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index 0bfd2c603c0..c7d233b0de9 100644
--- a/usage.c
+++ b/usage.c
@@ -253,20 +253,6 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
-void warning_fp(FILE *out, const char *warn, ...)
-{
-	va_list params;
-
-	va_start(params, warn);
-	if (out == stderr)
-		warn_routine(warn, params);
-	else {
-		vfprintf(out, warn, params);
-		fputc('\n', out);
-	}
-	va_end(params);
-}
-
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 
I do think you'd probably prefer the non-macro version, which is pretty
much just going back to this:
https://lore.kernel.org/git/6fb4f4580a581b2e43bc4b8deaa3d2d2bf4a8756.1643479633.git.gitgitgadget@gmail.com/

diff --git a/diff.c b/diff.c
index 28368110147..21c9561f546 100644
--- a/diff.c
+++ b/diff.c
@@ -6380,17 +6380,28 @@ N_("you may want to set your %s variable to at least "
 void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
 			    FILE *out)
 {
+	const char *msg;
+
 	fflush(stdout);
 	if (degraded_cc)
-		warning_fp(out, _(degrade_cc_to_c_warning));
+		msg = _(degrade_cc_to_c_warning);
 	else if (needed)
-		warning_fp(out, _(rename_limit_warning));
+		msg = _(rename_limit_warning);
 	else
 		return;
 
+	if (out == stderr)
+		warning("%s", msg);
+	else
+		fprintf(stderr, "%s", msg);
 
-	if (0 < needed)
-		warning_fp(out, _(rename_limit_advice), varname, needed);
+	if (0 >= needed)
+		return;
+
+	if (out == stderr)
+		warning(_(rename_limit_advice), varname, needed);
+	else
+		fprintf(stderr, _(rename_limit_advice), varname, needed);
 }
 
 static void create_filepairs_for_header_only_notifications(struct diff_options *o)
diff --git a/git-compat-util.h b/git-compat-util.h
index 64ba60e5c71..d70ce142861 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
-void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index 0bfd2c603c0..c7d233b0de9 100644
--- a/usage.c
+++ b/usage.c
@@ -253,20 +253,6 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
-void warning_fp(FILE *out, const char *warn, ...)
-{
-	va_list params;
-
-	va_start(params, warn);
-	if (out == stderr)
-		warn_routine(warn, params);
-	else {
-		vfprintf(out, warn, params);
-		fputc('\n', out);
-	}
-	va_end(params);
-}
-
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 
Note that both your pre-image, my macro version and Johannes's
linked-to-above are technically buggy in that they treat a
non-formatting format as a formatting format. I.e. we should use
warning("%s", msg) in that case, not warning(msg).

See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
usage, 2021-07-13) for a similar bug/fix.

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 03 2022, Elijah Newren wrote:

> On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>>
>> > From: Elijah Newren <newren@gmail.com>
>> >
>> > Let merge-tree accept a `--write-tree` parameter for choosing real
>> > merges instead of trivial merges, and accept an optional
>> > `--trivial-merge` option to get the traditional behavior.  Note that
>> > these accept different numbers of arguments, though, so these names
>> > need not actually be used.
>>
>> Maybe that ship has sailed, but just my 0.02: I thought this whole thing
>> was much less confusing with your initial merge-tree-ort proposal at
>> https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
>> I.e. the end-state of merge-tree.c is that you end up reading largely
>> unrelated code (various static functions only used by one side or
>> another).
>
> Christian's merge-tree-ort proposal?

Yes. I'm clearly getting the chronology wrong here. Sorry.

>> But maybe that's all water under the bridge etc, however...
>>
>> >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>> >  {
>> > -     if (argc != 4)
>> > -             usage(merge_tree_usage);
>> > -     return trivial_merge(argc, argv);
>> > +     struct merge_tree_options o = { 0 };
>> > +     int expected_remaining_argc;
>> > +
>> > +     const char * const merge_tree_usage[] = {
>> > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> > +             NULL
>> > +     };
>> > +     struct option mt_options[] = {
>> > +             OPT_CMDMODE(0, "write-tree", &o.mode,
>> > +                         N_("do a real merge instead of a trivial merge"),
>> > +                         'w'),
>> > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
>> > +                         N_("do a trivial merge only"), 't'),
>> > +             OPT_END()
>> > +     };
>> > +
>> > +     /* Parse arguments */
>> > +     argc = parse_options(argc, argv, prefix, mt_options,
>> > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
>> > +     if (o.mode) {
>> > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
>> > +             if (argc != expected_remaining_argc)
>> > +                     usage_with_options(merge_tree_usage, mt_options);
>> > +     } else {
>> > +             if (argc < 2 || argc > 3)
>> > +                     usage_with_options(merge_tree_usage, mt_options);
>> > +             o.mode = (argc == 2 ? 'w' : 't');
>> > +     }
>>
>> Do we really need to make this interface more special-casey by
>> auto-guessing based on argc what argument you want? I.e. instead of
>> usage like:
>>
>>         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>>         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>>
>> Wouldn't it be simpler to just have the equivalent of:
>>
>>         # old
>>         git merge-tree ...
>>         # new
>>         git merge-tree --new-thing ...
>>
>> And not have to look at ... to figure out if we're dispatching to the
>> new or old thing.
>
> You seem to be focusing on code simplicity?  Sure, that'd be simpler
> code, it'd just be a less useful feature.
>
> I think passing --write-tree all the time would be an annoyance.  I
> don't see why anyone would ever use the other mode.  However, for as
> long as both exist in the manual, it makes the manual easier to
> explain to users, and example testcases more self-documenting by
> having the flag there.  That's the sole purpose of the flag.
>
> I'm never going to actually use it when I invoke it from the command
> line.  And I suspect most would leave it off.

I think I mostly covered this in
https://lore.kernel.org/git/220203.86wnic5lba.gmgdl@evledraar.gmail.com/
in the side-thread.

I'm not opposed to them being in the same command, it just seems like
more work for those reasons in every way.

E.g. the current manual page doesn't really describe the format the
--trivial-merge emits, so we could do with a section on that, but then
it would say "only for this option, not the other" etc.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Johannes Altmanninger]

On Wed, Feb 02, 2022 at 04:18:39PM -0800, Elijah Newren wrote:
> On Wed, Feb 2, 2022 at 2:01 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Elijah Newren <newren@gmail.com> writes:
> >
> > > Yes, you are reading right.  I think the cherry-pick/rebase
> > > replacement actually deserves a separate command from what merges
> > > should use; replaying a sequence of commits just has a number of UI
> > > differences and abilities that I think pull it in a different
> > > direction.
> >
> > I completely disagree.  Each individual step in a sequence of
> > replaying commits in order (or in reverse order) should be
> > scriptable as a single merge-tree that takes "apply the change to go
> > from A^ to A on X".  Sequencing and placing UI around it is a job
> > for the script that drives merge-tree.
> 
> Adding such an ability to merge-tree would be trivial -- it basically
> involves just two things: (1) accepting one extra argument, and (2)
> calling merge_incore_nonrecursive() instead of
> merge_incore_recursive().
> 
> However, I think forking a subprocess for every merge of a series of
> commits is a completely unreasonable overhead, so even if we provide
> such an option to merge-tree, I still want a separate plumbing-ish
> tool that does non-worktree/non-index replaying of commits which is
> not written as a driver of merge-tree.  That other tool should just
> call merge_incore_nonrecursive() directly.  And such a tool, since it
> should handle an arbitrary number of commits, should certainly be able
> to handle just one commit.  From that angle, it feels like adding
> another mode to merge-tree would just be a partial duplication of the
> other tool.

I wonder how the UI of a tool that does non-worktree/non-index cherry-picks
will look like.  I'd expect it to produce the same output as merge-tree,
except cherry-pick should probably output a commit OID, not a tree.

Maybe we want a unified command that produces commits from any sequence of
merge/cherry-pick/revert/reword steps. The obvious UI would use something
like the rebase-todo list as input.  For example:

	$ echo '
	pick commit1
	reword commit2	# edit commit message in $GIT_EDITOR
	merge commit3 -m "log message"
	' | git create-commit commit0
	<OID of final commit>

we start from commit0 and apply steps one-by-one. Obviously, one unsolved
problem is how to pass parameters like commit messages if no editor should
be invoked (my sketch uses -m).
If any of the steps fails when merging merge, then we get the tree with
conflicts

	$ echo '
	pick commit1
	pick commit2
	pick commit-that-does-not-apply
	' | git create-commit commit0
	<OID of commit after step 2>
	<OID of toplevel tree after failed merge>
	<Conflicted file info>
	<Informational messages>

Replaying a series of commits might look like this:

	$ echo 'pick commit1 ^commit0' | git create-commit new-base

I'm concluding that this is a difficult UI problem, and having a merge-tree
command that accepts a "common ancestor" parameter could make it easier
to experiment.  Of course that depends on who is experimenting.

> 
> However, if the other tool doesn't obviate the need for this
> additional mode (perhaps it ends up being forced to be too
> porcelain-ish insteading of plumbing-ish?), or folks really just want
> another merge-tree mode, I'm happy to add one along with the tool I
> submit later.  Does that sound reasonable to you, or is there
> something you're still objecting to that I've missed?

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Elijah Newren]

On Thu, Feb 3, 2022 at 2:26 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
> > On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
[...]
> >> I would get it if the point was to actually use the full usage.c
> >> machinery, but we're just either calling warning(), or printing a
> >> formatted string to a file FILE *. There's no need to go through usage.c
> >> for that, and adding an API to it that behaves like this new
> >> warning_fp() is really confusing.
> >
> > Because the formatted string being printed to the file won't have the
> > same "warning: " prefix that is normally added to stuff in usage?
>
> But the pre-image doesn't add that either. We're just calling
> vfprintf(), not our own vreportf().

Right, I'm saying that I thought you were reporting the original patch
as buggy because it doesn't produce the same message when given a
different stream; it'll omit the "warning: " prefix.  And I was
agreeing that it was buggy for those reasons.

Or was there a different reason you didn't like that function being in usage.c?

> > That's a fair point; that does have a bit of a consistency problem.
> > And I'd rather the messages were consistent regardless of where they
> > are printed.
>
> I think that makes sense, that's why I added die_message() recently. If
> you meant to print a "warning: " prefix I think it would also be fine in
> this case to just do it inline. See prior art at:
>
>     git grep '"(fatal|error|warning):' -- '*.c'

So, making diff_warn_rename_limit() stop using warning(), and just
always directly writing out and including "warning:" in its message?

I'm wondering if that might cause problems if there are any existing
callers of diff_warn_rename_limit() that might also be using
set_warn_routine() (e.g. perhaps apply.c?).  Of course, those callers
probably couldn't handle anything other than the default stream.
Hmm...

> >> diff --git a/diff.c b/diff.c
> >> index 28368110147..4cf67e93dea 100644
> >> --- a/diff.c
> >> +++ b/diff.c
> >> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
> >>  N_("you may want to set your %s variable to at least "
> >>     "%d and retry the command.");
> >>
> >> +#define warning_fp(out, fmt, ...) do { \
> >> +       if (out == stderr) \
> >> +               warning(fmt, __VA_ARGS__); \
> >> +       else \
> >> +               fprintf(out, fmt, __VA_ARGS__); \
> >> +} while (0)
> >> +
> >>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> >>                             FILE *out)
> >>  {
> >>         fflush(stdout);
> >>         if (degraded_cc)
> >> -               warning_fp(out, _(degrade_cc_to_c_warning));
> >> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
> >>         else if (needed)
> >> -               warning_fp(out, _(rename_limit_warning));
> >> +               warning_fp(out, _(rename_limit_warning), NULL);
> >
> > Why do the only callers have a NULL parameter here?  Is this one of
> > those va_list/va_args things I never bothered to properly learn?
>
> That's wrong (I blame tiredness last night),an actual working version is
> produced below. Clang accepted my broken code, but gcc rightly yells
> about it:

Well, seeing the new code makes me feel better as it makes more sense
to me now.  ;-)

> Note that both your pre-image, my macro version and Johannes's
> linked-to-above are technically buggy in that they treat a
> non-formatting format as a formatting format. I.e. we should use
> warning("%s", msg) in that case, not warning(msg).
>
> See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
> usage, 2021-07-13) for a similar bug/fix.

Good point.


Man, what a can of worms this all is.  Maybe I really should just drop
patches 5, 6, and 8 for now...

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 03 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 2:26 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Thu, Feb 03 2022, Elijah Newren wrote:
>>
>> > On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> [...]
>> >> I would get it if the point was to actually use the full usage.c
>> >> machinery, but we're just either calling warning(), or printing a
>> >> formatted string to a file FILE *. There's no need to go through usage.c
>> >> for that, and adding an API to it that behaves like this new
>> >> warning_fp() is really confusing.
>> >
>> > Because the formatted string being printed to the file won't have the
>> > same "warning: " prefix that is normally added to stuff in usage?
>>
>> But the pre-image doesn't add that either. We're just calling
>> vfprintf(), not our own vreportf().
>
> Right, I'm saying that I thought you were reporting the original patch
> as buggy because it doesn't produce the same message when given a
> different stream; it'll omit the "warning: " prefix.  And I was
> agreeing that it was buggy for those reasons.
>
> Or was there a different reason you didn't like that function being in usage.c?

Maybe it was accidentally a bug report :) But no, I was just observing
that it was odd that it was in usage.c when it seemingly had almost
nothing to do with what that API accomplishes.

But maybe the underlying issue is that the "warning: " part is missing
here. But I didn't mean to report that/missed it.

>> > That's a fair point; that does have a bit of a consistency problem.
>> > And I'd rather the messages were consistent regardless of where they
>> > are printed.
>>
>> I think that makes sense, that's why I added die_message() recently. If
>> you meant to print a "warning: " prefix I think it would also be fine in
>> this case to just do it inline. See prior art at:
>>
>>     git grep '"(fatal|error|warning):' -- '*.c'
>
> So, making diff_warn_rename_limit() stop using warning(), and just
> always directly writing out and including "warning:" in its message?
>
> I'm wondering if that might cause problems if there are any existing
> callers of diff_warn_rename_limit() that might also be using
> set_warn_routine() (e.g. perhaps apply.c?).  Of course, those callers
> probably couldn't handle anything other than the default stream.
> Hmm...

Using set_warn_routine() is the "right" way to do it currently, and with
or without a "warning: " prefix the current API use is "wrong" if the
purpose is to have it behave nicely with the pluggable usage.c API.

But of course that may not be the goal at all, i.e. I think here we've
probably stopped caring about usage.c's formatting, logging
etc. entirely, and are just emitting a string.

Just like serve.c emits "E <msg>" or whatever (and not with error()).

>> >> diff --git a/diff.c b/diff.c
>> >> index 28368110147..4cf67e93dea 100644
>> >> --- a/diff.c
>> >> +++ b/diff.c
>> >> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
>> >>  N_("you may want to set your %s variable to at least "
>> >>     "%d and retry the command.");
>> >>
>> >> +#define warning_fp(out, fmt, ...) do { \
>> >> +       if (out == stderr) \
>> >> +               warning(fmt, __VA_ARGS__); \
>> >> +       else \
>> >> +               fprintf(out, fmt, __VA_ARGS__); \
>> >> +} while (0)
>> >> +
>> >>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
>> >>                             FILE *out)
>> >>  {
>> >>         fflush(stdout);
>> >>         if (degraded_cc)
>> >> -               warning_fp(out, _(degrade_cc_to_c_warning));
>> >> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
>> >>         else if (needed)
>> >> -               warning_fp(out, _(rename_limit_warning));
>> >> +               warning_fp(out, _(rename_limit_warning), NULL);
>> >
>> > Why do the only callers have a NULL parameter here?  Is this one of
>> > those va_list/va_args things I never bothered to properly learn?
>>
>> That's wrong (I blame tiredness last night),an actual working version is
>> produced below. Clang accepted my broken code, but gcc rightly yells
>> about it:
>
> Well, seeing the new code makes me feel better as it makes more sense
> to me now.  ;-)
>
>> Note that both your pre-image, my macro version and Johannes's
>> linked-to-above are technically buggy in that they treat a
>> non-formatting format as a formatting format. I.e. we should use
>> warning("%s", msg) in that case, not warning(msg).
>>
>> See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
>> usage, 2021-07-13) for a similar bug/fix.
>
> Good point.
>
> Man, what a can of worms this all is.  Maybe I really should just drop
> patches 5, 6, and 8 for now...

Yeah, I really think it's worth it to just sprinkle a tiny bit of
if/else (or a macro) here and print to stderr inline or not. We can make
some use of some usage.c when there's good reason to do so, but this bit
just seems like a needless digression.

I hope all of this has helped somewhat ...

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren]

On Thu, Feb 3, 2022 at 1:52 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
[...]
> > ...also, even if we did require the `--write-tree` flag, we'd still
> > have to look at argc.  Since the option parsing handles both modes,
> > someone could leave off --write-tree, but include a bunch of other
> > options that only make sense with --write-tree.  Individually checking
> > the setting of every extra flag along with write_tree is a royal pain
> > and I don't want to repeat that for each new option added.  Simply
> > checking argc allows you to provide an error message if the user does
> > that.
> >
> > (And I think it's sad that in Git we often forgot to warn and notify
> > users of options that are only functional with certain other
> > arguments; it makes it harder for users to figure out, and has in the
> > past even made it harder for other developers to figure out what was
> > meant and how things are to be used.  I think I've seen multiple Git
> > devs be confused over ls-files --directory and --no-empty-directory
> > options, assuming they'd do something sensible for tracked files, when
> > in fact those arguments are simply ignored because they are only
> > modifiers for how untracked files are treated.)
>
> There's a much simpler way to do what you're trying to do here which is
> to only parse --write-tree, and as soon as you have that pass off two
> one function or the other, and have those functions call
> parse_options().

But that makes --write-tree a mandatory argument when trying to use
that mode, right?  If so, that is not a simpler way to do what I'm
trying to do at all; it breaks my intended usage.

--write-tree is a documentation-only construct that users should never
have to pass.

Also, what happens if we remove the --trivial-merge flag and its whole
mode after a sufficient deprecation period?  Would the --write-tree
parameter remain required in your model to select the only existing
mode, simply due to us having gone through a transition period?

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren]

Hi,

On Thu, Feb 3, 2022 at 2:42 AM Johannes Altmanninger <aclopte@gmail.com> wrote:
>
> On Wed, Feb 02, 2022 at 04:18:39PM -0800, Elijah Newren wrote:
> > On Wed, Feb 2, 2022 at 2:01 PM Junio C Hamano <gitster@pobox.com> wrote:
> > >
> > > Elijah Newren <newren@gmail.com> writes:
> > >
> > > > Yes, you are reading right.  I think the cherry-pick/rebase
> > > > replacement actually deserves a separate command from what merges
> > > > should use; replaying a sequence of commits just has a number of UI
> > > > differences and abilities that I think pull it in a different
> > > > direction.
> > >
> > > I completely disagree.  Each individual step in a sequence of
> > > replaying commits in order (or in reverse order) should be
> > > scriptable as a single merge-tree that takes "apply the change to go
> > > from A^ to A on X".  Sequencing and placing UI around it is a job
> > > for the script that drives merge-tree.
> >
> > Adding such an ability to merge-tree would be trivial -- it basically
> > involves just two things: (1) accepting one extra argument, and (2)
> > calling merge_incore_nonrecursive() instead of
> > merge_incore_recursive().
> >
> > However, I think forking a subprocess for every merge of a series of
> > commits is a completely unreasonable overhead, so even if we provide
> > such an option to merge-tree, I still want a separate plumbing-ish
> > tool that does non-worktree/non-index replaying of commits which is
> > not written as a driver of merge-tree.  That other tool should just
> > call merge_incore_nonrecursive() directly.  And such a tool, since it
> > should handle an arbitrary number of commits, should certainly be able
> > to handle just one commit.  From that angle, it feels like adding
> > another mode to merge-tree would just be a partial duplication of the
> > other tool.
>
> I wonder how the UI of a tool that does non-worktree/non-index cherry-picks
> will look like.  I'd expect it to produce the same output as merge-tree,
> except cherry-pick should probably output a commit OID, not a tree.
>
> Maybe we want a unified command that produces commits from any sequence of
> merge/cherry-pick/revert/reword steps. The obvious UI would use something
> like the rebase-todo list as input.  For example:
>
>         $ echo '
>         pick commit1
>         reword commit2  # edit commit message in $GIT_EDITOR
>         merge commit3 -m "log message"
>         ' | git create-commit commit0
>         <OID of final commit>
>
> we start from commit0 and apply steps one-by-one. Obviously, one unsolved
> problem is how to pass parameters like commit messages if no editor should
> be invoked (my sketch uses -m).
> If any of the steps fails when merging merge, then we get the tree with
> conflicts
>
>         $ echo '
>         pick commit1
>         pick commit2
>         pick commit-that-does-not-apply
>         ' | git create-commit commit0
>         <OID of commit after step 2>
>         <OID of toplevel tree after failed merge>
>         <Conflicted file info>
>         <Informational messages>
>
> Replaying a series of commits might look like this:
>
>         $ echo 'pick commit1 ^commit0' | git create-commit new-base
>
> I'm concluding that this is a difficult UI problem

I agree.  I've got a lot of thoughts on it, and some work in progress
towards it (https://github.com/newren/git/tree/replay -- _very_ hacky,
not even close to alpha quality, lots of fixup commits, todo comments,
random brain dump files added to the tree, based on a previous round
of this patch series, not updated for weeks, etc., etc.)

> and having a merge-tree
> command that accepts a "common ancestor" parameter could make it easier
> to experiment.  Of course that depends on who is experimenting.

I think that would result in experiments and eventually full-blown
scripts designed around forking subprocesses for every merge, and
pushes us back into the world of having a scripted-rebase again.  Yes,
I know people can transliterate shell back to C; it seems to always be
done as a half-way measure with the forking just being done from C or
have other UI-warts guided by the shell design.  In fact, *that* was
the primary reason for me not providing a merge-tree option based on
merge_incore_nonrecursive(), despite how trivial it'd be to provide
it.  If someone wanted a merge_incore_nonrecursive() mode for
merge-tree for reasons other than attempting to build a
rebase/cherry-pick replacement based on it, then I'd be much happier
to provide it.

If someone wants to experiment with what a plumbing-ish
rebase/cherry-pick would look like, the _right_ way to do it would be
making using of merge_incore_nonrecursive() directly.  If they want
example code, I already provided some a year and a half ago and got it
merged into git.git in the form of t/helper/test-fast-rebase.c.  My
"replay" branch is based on that code, but (a) moves it from t/helper
to a real builtin, (b) removes the hardcoded very strict input, (c)
removes the line of code doing the index & working tree updates, and
(d) modifies the output to be a more plumbing-ish style.

We'll certainly have discussions on what that should look like.  But a
plumbing-ish replacement for merge was much simpler, and made sense to
do first.  I would prefer to concentrate on getting that hammered down
first.  Then I'll start discussions on a plumbing-ish
rebase/cherry-pick.  And if that doesn't fulfill all the needs that
folks think they want out of merge-tree, then we can add a
merge_incore_nonrecursive()-based mode to merge-tree.  It's all
coming, but having fought transliterations-of-scripts in
merge-recursive.c, sequencer.c, stash.c, rebase.c, etc. for years I
really, really don't want any more of that.  Let's end that insanity.

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Elijah Newren]

On Thu, Feb 3, 2022 at 8:24 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
> > On Thu, Feb 3, 2022 at 2:26 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >>
> >> On Thu, Feb 03 2022, Elijah Newren wrote:
> >>
> >> > On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> > [...]
> >> >> I would get it if the point was to actually use the full usage.c
> >> >> machinery, but we're just either calling warning(), or printing a
> >> >> formatted string to a file FILE *. There's no need to go through usage.c
> >> >> for that, and adding an API to it that behaves like this new
> >> >> warning_fp() is really confusing.
> >> >
> >> > Because the formatted string being printed to the file won't have the
> >> > same "warning: " prefix that is normally added to stuff in usage?
> >>
> >> But the pre-image doesn't add that either. We're just calling
> >> vfprintf(), not our own vreportf().
> >
> > Right, I'm saying that I thought you were reporting the original patch
> > as buggy because it doesn't produce the same message when given a
> > different stream; it'll omit the "warning: " prefix.  And I was
> > agreeing that it was buggy for those reasons.
> >
> > Or was there a different reason you didn't like that function being in usage.c?
>
> Maybe it was accidentally a bug report :) But no, I was just observing
> that it was odd that it was in usage.c when it seemingly had almost
> nothing to do with what that API accomplishes.
>
> But maybe the underlying issue is that the "warning: " part is missing
> here. But I didn't mean to report that/missed it.
>
> >> > That's a fair point; that does have a bit of a consistency problem.
> >> > And I'd rather the messages were consistent regardless of where they
> >> > are printed.
> >>
> >> I think that makes sense, that's why I added die_message() recently. If
> >> you meant to print a "warning: " prefix I think it would also be fine in
> >> this case to just do it inline. See prior art at:
> >>
> >>     git grep '"(fatal|error|warning):' -- '*.c'
> >
> > So, making diff_warn_rename_limit() stop using warning(), and just
> > always directly writing out and including "warning:" in its message?
> >
> > I'm wondering if that might cause problems if there are any existing
> > callers of diff_warn_rename_limit() that might also be using
> > set_warn_routine() (e.g. perhaps apply.c?).  Of course, those callers
> > probably couldn't handle anything other than the default stream.
> > Hmm...
>
> Using set_warn_routine() is the "right" way to do it currently, and with
> or without a "warning: " prefix the current API use is "wrong" if the
> purpose is to have it behave nicely with the pluggable usage.c API.
>
> But of course that may not be the goal at all, i.e. I think here we've
> probably stopped caring about usage.c's formatting, logging
> etc. entirely, and are just emitting a string.
>
> Just like serve.c emits "E <msg>" or whatever (and not with error()).
>
> >> >> diff --git a/diff.c b/diff.c
> >> >> index 28368110147..4cf67e93dea 100644
> >> >> --- a/diff.c
> >> >> +++ b/diff.c
> >> >> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
> >> >>  N_("you may want to set your %s variable to at least "
> >> >>     "%d and retry the command.");
> >> >>
> >> >> +#define warning_fp(out, fmt, ...) do { \
> >> >> +       if (out == stderr) \
> >> >> +               warning(fmt, __VA_ARGS__); \
> >> >> +       else \
> >> >> +               fprintf(out, fmt, __VA_ARGS__); \
> >> >> +} while (0)
> >> >> +
> >> >>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> >> >>                             FILE *out)
> >> >>  {
> >> >>         fflush(stdout);
> >> >>         if (degraded_cc)
> >> >> -               warning_fp(out, _(degrade_cc_to_c_warning));
> >> >> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
> >> >>         else if (needed)
> >> >> -               warning_fp(out, _(rename_limit_warning));
> >> >> +               warning_fp(out, _(rename_limit_warning), NULL);
> >> >
> >> > Why do the only callers have a NULL parameter here?  Is this one of
> >> > those va_list/va_args things I never bothered to properly learn?
> >>
> >> That's wrong (I blame tiredness last night),an actual working version is
> >> produced below. Clang accepted my broken code, but gcc rightly yells
> >> about it:
> >
> > Well, seeing the new code makes me feel better as it makes more sense
> > to me now.  ;-)
> >
> >> Note that both your pre-image, my macro version and Johannes's
> >> linked-to-above are technically buggy in that they treat a
> >> non-formatting format as a formatting format. I.e. we should use
> >> warning("%s", msg) in that case, not warning(msg).
> >>
> >> See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
> >> usage, 2021-07-13) for a similar bug/fix.
> >
> > Good point.
> >
> > Man, what a can of worms this all is.  Maybe I really should just drop
> > patches 5, 6, and 8 for now...
>
> Yeah, I really think it's worth it to just sprinkle a tiny bit of
> if/else (or a macro) here and print to stderr inline or not. We can make
> some use of some usage.c when there's good reason to do so, but this bit
> just seems like a needless digression.
>
> I hope all of this has helped somewhat ...

Absolutely; thanks for reviewing!  These parts may just end up in me
dropping some patches for now (since they're not actually being used
anyway), but I think it's all good feedback.

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 03 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 1:52 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Thu, Feb 03 2022, Elijah Newren wrote:
>>
> [...]
>> > ...also, even if we did require the `--write-tree` flag, we'd still
>> > have to look at argc.  Since the option parsing handles both modes,
>> > someone could leave off --write-tree, but include a bunch of other
>> > options that only make sense with --write-tree.  Individually checking
>> > the setting of every extra flag along with write_tree is a royal pain
>> > and I don't want to repeat that for each new option added.  Simply
>> > checking argc allows you to provide an error message if the user does
>> > that.
>> >
>> > (And I think it's sad that in Git we often forgot to warn and notify
>> > users of options that are only functional with certain other
>> > arguments; it makes it harder for users to figure out, and has in the
>> > past even made it harder for other developers to figure out what was
>> > meant and how things are to be used.  I think I've seen multiple Git
>> > devs be confused over ls-files --directory and --no-empty-directory
>> > options, assuming they'd do something sensible for tracked files, when
>> > in fact those arguments are simply ignored because they are only
>> > modifiers for how untracked files are treated.)
>>
>> There's a much simpler way to do what you're trying to do here which is
>> to only parse --write-tree, and as soon as you have that pass off two
>> one function or the other, and have those functions call
>> parse_options().
>
> But that makes --write-tree a mandatory argument when trying to use
> that mode, right?  If so, that is not a simpler way to do what I'm
> trying to do at all; it breaks my intended usage.
>
> --write-tree is a documentation-only construct that users should never
> have to pass.
>
> Also, what happens if we remove the --trivial-merge flag and its whole
> mode after a sufficient deprecation period?  Would the --write-tree
> parameter remain required in your model to select the only existing
> mode, simply due to us having gone through a transition period?

You can have your cake and eat it too by running parse_optionss() N
number of times. Although perhaps in this case the end result isn't
worth it.

I was hoping this could be a simpler case of a subcommand dispatch, and
perhaps it can still be generalized to that.

If the "trivial" mode never takes options and always 3 argv elements, we
could just run parse_options() for it with no options, after checking
that we have 3 arguments, and none start with '-'.

But the below is a generalization of this I tried out just now, it
passes all your tests, and means that whenever you add new options you
don't need to keep saying "no, not with the trivial mode" for each one.

Basically we run parse_options() once with the full set of options, and
save away argc/argv (note the lack of strvec_clear() there, that's a
TODO memory leak).

Then we've got a o.mode, which along with argc is the *only* thing we
pay attention to at that point.

Then we dispatch to the "trivial" or "write" functions, which do
parse_options() again, this time with only their options.

It means that e.g. this now works as expected:
    
    ./git merge-tree --trivial-merge -z origin/{master,next,seen}
    error: unknown switch `z'
    usage: git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>
    
        --trivial-merge       do a trivial merge only

I.e. we error out, with your verison we'll just ignore the -z.

Your "--trivial-merge is incompatible with all other options" doesn't
work as you expect, and is buggy whether you want to go this route or
not, as the added tests show.

Basically it does nothing at all. Because if you add --foo we'll die
before we get there, parse_options() will die for us.

But if you do --trivial-merge -z your argc/argv will be trimmed, because
-z is a known option. So your check is doing nothing. Your tests also
pass with this removal of the only option compatibily check on top:
	
	diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
	index 58c0ddc5a32..08f18d43334 100644
	--- a/builtin/merge-tree.c
	+++ b/builtin/merge-tree.c
	@@ -476,7 +476,6 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
	 {
	 	struct merge_tree_options o = { .show_messages = -1 };
	 	int expected_remaining_argc;
	-	int original_argc;
	 
	 	const char * const merge_tree_usage[] = {
	 		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
	@@ -505,7 +504,6 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
	 	};
	 
	 	/* Parse arguments */
	-	original_argc = argc;
	 	argc = parse_options(argc, argv, prefix, mt_options,
	 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
	 	if (o.mode) {
	@@ -517,8 +515,6 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
	 			usage_with_options(merge_tree_usage, mt_options);
	 		o.mode = (argc == 2 ? 'w' : 't');
	 	}
	-	if (o.mode == 't' && original_argc < argc)
	-		die(_("--trivial-merge is incompatible with all other options"));
	 
	 	/* Do the relevant type of merge */
	 	if (o.mode == 'w')

So, here it is in all its glory :) A bit nasty for sure, but IMO
preferrable to an ever expanding list of "X isn't compatible with A".

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 58c0ddc5a32..1d47912816d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -12,6 +12,7 @@
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 #include "quote.h"
+#include "strvec.h"
 
 static int line_termination = '\n';
 
@@ -371,13 +372,66 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(const char *base,
-			 const char *branch1,
-			 const char *branch2)
+struct merge_tree_options {
+	int mode;
+	int allow_unrelated_histories;
+	int show_messages;
+	int exclude_modes_oids_stages;
+};
+
+#define BUILTIN_MERGE_TREE_USAGE_WRITE \
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>")
+#define BUILTIN_MERGE_TREE_USAGE_TRIVIAL \
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>")
+
+#define BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL \
+		OPT_CMDMODE(0, "trivial-merge", &o.mode, \
+			    N_("do a trivial merge only"), 't')
+
+#define BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE \
+		OPT_CMDMODE(0, "write-tree", &o.mode, \
+			    N_("do a real merge instead of a trivial merge"), \
+			    'w')
+
+#define BUILTIN_MERGE_TREE_OPT_WRITE \
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE, \
+		OPT_BOOL(0, "messages", &o.show_messages, \
+			 N_("also show informational/conflict messages")), \
+		OPT_SET_INT('z', NULL, &line_termination, \
+			    N_("separate paths with the NUL character"), '\0'), \
+		OPT_BOOL_F('l', "exclude-modes-oids-stages", \
+			   &o.exclude_modes_oids_stages, \
+			   N_("list conflicted files without modes/oids/stages"), \
+			   PARSE_OPT_NONEG), \
+		OPT_BOOL_F(0, "allow-unrelated-histories", \
+			   &o.allow_unrelated_histories, \
+			   N_("allow merging unrelated histories"), \
+			   PARSE_OPT_NONEG)
+
+static int trivial_merge(int argc, const char **argv, const char *prefix)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
+	struct merge_tree_options o = { 0 };
+	const char * const usage[] = {
+		BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
+		NULL,
+	};
+	struct option options[] = {
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
+		OPT_END()
+	};
+	const char *base, *branch1, *branch2;
+
+	argc = parse_options(argc, argv, prefix, options, usage,
+			     PARSE_OPT_STOP_AT_NON_OPTION);
+	if (argc != 3)
+		BUG("should have ensured remaining argc == 3 already! Got %d", argc);
+
+	base = argv[0];
+	branch1 = argv[1];
+	branch2 = argv[2];;
 
 	buf1 = get_tree_descriptor(r, t+0, base);
 	buf2 = get_tree_descriptor(r, t+1, branch1);
@@ -391,24 +445,34 @@ static int trivial_merge(const char *base,
 	return 0;
 }
 
-struct merge_tree_options {
-	int mode;
-	int allow_unrelated_histories;
-	int show_messages;
-	int exclude_modes_oids_stages;
-};
-
-static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2,
-		      const char *prefix)
+static int real_merge(int argc, const char **argv, const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
 	struct commit_list *merge_bases = NULL;
 	struct commit_list *j;
+	struct merge_tree_options o = { .show_messages = 1 };
 	struct merge_options opt;
 	struct merge_result result = { 0 };
 
+	const char * const usage[] = {
+		BUILTIN_MERGE_TREE_USAGE_WRITE,
+		NULL,
+	};
+	struct option options[] = {
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
+		BUILTIN_MERGE_TREE_OPT_WRITE,
+		OPT_END()
+	};
+	const char *branch1, *branch2;
+
+	argc = parse_options(argc, argv, prefix, options, usage,
+			     PARSE_OPT_STOP_AT_NON_OPTION);
+	if (argc != 2)
+		BUG("should have ensured remaining argc == 2 already! Got %d", argc);
+	branch1 = argv[0];
+	branch2 = argv[1];
+
 	parent1 = get_merge_parent(branch1);
 	if (!parent1)
 		help_unknown_ref(branch1, "merge-tree",
@@ -431,7 +495,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common && !o->allow_unrelated_histories)
+	if (!common && !o.allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -440,8 +504,8 @@ static int real_merge(struct merge_tree_options *o,
 	if (result.clean < 0)
 		die(_("failure to merge"));
 
-	if (o->show_messages == -1)
-		o->show_messages = !result.clean;
+	if (o.show_messages == -1)
+		o.show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
 	if (!result.clean) {
@@ -453,7 +517,7 @@ static int real_merge(struct merge_tree_options *o,
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
 			struct stage_info *c = conflicted_files.items[i].util;
-			if (!o->exclude_modes_oids_stages)
+			if (!o.exclude_modes_oids_stages)
 				printf("%06o %s %d\t",
 				       c->mode, oid_to_hex(&c->oid), c->stage);
 			else if (last && !strcmp(last, name))
@@ -464,7 +528,7 @@ static int real_merge(struct merge_tree_options *o,
 		}
 		string_list_clear(&conflicted_files, 1);
 	}
-	if (o->show_messages) {
+	if (o.show_messages) {
 		putchar(line_termination);
 		merge_display_update_messages(&opt, &result, stdout);
 	}
@@ -474,40 +538,32 @@ static int real_merge(struct merge_tree_options *o,
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { .show_messages = -1 };
+	struct merge_tree_options o;
 	int expected_remaining_argc;
-	int original_argc;
-
+	int original_argc = argc;
+	struct strvec original_args = STRVEC_INIT;
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
-		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		BUILTIN_MERGE_TREE_USAGE_WRITE,
+		BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
 		NULL
 	};
 	struct option mt_options[] = {
-		OPT_CMDMODE(0, "write-tree", &o.mode,
-			    N_("do a real merge instead of a trivial merge"),
-			    'w'),
-		OPT_CMDMODE(0, "trivial-merge", &o.mode,
-			    N_("do a trivial merge only"), 't'),
-		OPT_BOOL(0, "messages", &o.show_messages,
-			 N_("also show informational/conflict messages")),
-		OPT_SET_INT('z', NULL, &line_termination,
-			    N_("separate paths with the NUL character"), '\0'),
-		OPT_BOOL_F('l', "exclude-modes-oids-stages",
-			   &o.exclude_modes_oids_stages,
-			   N_("list conflicted files without modes/oids/stages"),
-			   PARSE_OPT_NONEG),
-		OPT_BOOL_F(0, "allow-unrelated-histories",
-			   &o.allow_unrelated_histories,
-			   N_("allow merging unrelated histories"),
-			   PARSE_OPT_NONEG),
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
+		BUILTIN_MERGE_TREE_OPT_WRITE,
 		OPT_END()
 	};
 
+	/* We only care about deciding "o.mode" here */
+	o.mode = 0;
+	/*
+	 * We need our original argv, and
+	 * PARSE_OPT_KEEP_{ARGV0,UNKNOWN} would do the wrong thing
+	 */
+	strvec_pushv(&original_args, argv);
 	/* Parse arguments */
-	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
-			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+			     merge_tree_usage, 0);
 	if (o.mode) {
 		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
 		if (argc != expected_remaining_argc)
@@ -517,12 +573,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			usage_with_options(merge_tree_usage, mt_options);
 		o.mode = (argc == 2 ? 'w' : 't');
 	}
-	if (o.mode == 't' && original_argc < argc)
-		die(_("--trivial-merge is incompatible with all other options"));
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
-		return real_merge(&o, argv[0], argv[1], prefix);
+		return real_merge(original_argc, original_args.v, prefix);
 	else
-		return trivial_merge(argv[0], argv[1], argv[2]);
+		return trivial_merge(original_argc, original_args.v, prefix);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 4de089d976d..749bdb6862d 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -92,6 +92,18 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+for opt in $(git merge-tree --git-completion-helper-all)
+do
+	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
+	then
+		continue
+	fi
+
+	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
+		test_expect_code 129 git merge-tree --trivial-merge $opt side1 side2 side3
+	'
+done
+
 test_expect_success 'test conflict notices and such' '
 	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren]

On Thu, Feb 3, 2022 at 9:38 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
[...]
> > But that makes --write-tree a mandatory argument when trying to use
> > that mode, right?  If so, that is not a simpler way to do what I'm
> > trying to do at all; it breaks my intended usage.
> >
> > --write-tree is a documentation-only construct that users should never
> > have to pass.
> >
> > Also, what happens if we remove the --trivial-merge flag and its whole
> > mode after a sufficient deprecation period?  Would the --write-tree
> > parameter remain required in your model to select the only existing
> > mode, simply due to us having gone through a transition period?
>
> You can have your cake and eat it too by running parse_optionss() N
> number of times. Although perhaps in this case the end result isn't
> worth it.
>
> I was hoping this could be a simpler case of a subcommand dispatch, and
> perhaps it can still be generalized to that.
>
> If the "trivial" mode never takes options and always 3 argv elements, we
> could just run parse_options() for it with no options, after checking
> that we have 3 arguments, and none start with '-'.
>
> But the below is a generalization of this I tried out just now, it
> passes all your tests, and means that whenever you add new options you
> don't need to keep saying "no, not with the trivial mode" for each one.
>
> Basically we run parse_options() once with the full set of options, and
> save away argc/argv (note the lack of strvec_clear() there, that's a
> TODO memory leak).
>
> Then we've got a o.mode, which along with argc is the *only* thing we
> pay attention to at that point.
>
> Then we dispatch to the "trivial" or "write" functions, which do
> parse_options() again, this time with only their options.
>
> It means that e.g. this now works as expected:
>
>     ./git merge-tree --trivial-merge -z origin/{master,next,seen}
>     error: unknown switch `z'
>     usage: git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>
>
>         --trivial-merge       do a trivial merge only
>
> I.e. we error out, with your verison we'll just ignore the -z.
>
> Your "--trivial-merge is incompatible with all other options" doesn't
> work as you expect, and is buggy whether you want to go this route or
> not, as the added tests show.
>
> Basically it does nothing at all. Because if you add --foo we'll die
> before we get there, parse_options() will die for us.
>
> But if you do --trivial-merge -z your argc/argv will be trimmed, because
> -z is a known option. So your check is doing nothing.

Gah, good catch.  How did I get the check reversed??  I know I tested
it at one point.  Anyway, this fixes it:

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 58c0ddc5a3..fb25e3d10d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -505,19 +505,22 @@ int cmd_merge_tree(int argc, const char **argv, const cha
r *prefix)
        };

        /* Parse arguments */
-       original_argc = argc;
+       original_argc = argc - 1; /* ignoring program name, i.e. argv[0] */
        argc = parse_options(argc, argv, prefix, mt_options,
                             merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
        if (o.mode) {
                expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
                if (argc != expected_remaining_argc)
                        usage_with_options(merge_tree_usage, mt_options);
+               if (o.mode == 't')
+                       /* Removal of `--trivial-merge` is expected */
+                       original_argc_options--;
        } else {
                if (argc < 2 || argc > 3)
                        usage_with_options(merge_tree_usage, mt_options);
                o.mode = (argc == 2 ? 'w' : 't');
        }
-       if (o.mode == 't' && original_argc < argc)
+       if (o.mode == 't' && argc < original_argc)
                die(_("--trivial-merge is incompatible with all other
options"));

        /* Do the relevant type of merge */


and results in this error being shown when any other option is listed
with --trivial-merge.

> So, here it is in all its glory :) A bit nasty for sure, but IMO
> preferrable to an ever expanding list of "X isn't compatible with A".

Agreed...on both counts.  :-)

> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 58c0ddc5a32..1d47912816d 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -12,6 +12,7 @@
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>  #include "quote.h"
> +#include "strvec.h"
>
>  static int line_termination = '\n';
>
> @@ -371,13 +372,66 @@ static void *get_tree_descriptor(struct repository *r,
>         return buf;
>  }
>
> -static int trivial_merge(const char *base,
> -                        const char *branch1,
> -                        const char *branch2)
> +struct merge_tree_options {
> +       int mode;
> +       int allow_unrelated_histories;
> +       int show_messages;
> +       int exclude_modes_oids_stages;
> +};
> +
> +#define BUILTIN_MERGE_TREE_USAGE_WRITE \
> +               N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>")
> +#define BUILTIN_MERGE_TREE_USAGE_TRIVIAL \
> +               N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>")
> +
> +#define BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL \
> +               OPT_CMDMODE(0, "trivial-merge", &o.mode, \
> +                           N_("do a trivial merge only"), 't')
> +
> +#define BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE \
> +               OPT_CMDMODE(0, "write-tree", &o.mode, \
> +                           N_("do a real merge instead of a trivial merge"), \
> +                           'w')
> +
> +#define BUILTIN_MERGE_TREE_OPT_WRITE \
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE, \
> +               OPT_BOOL(0, "messages", &o.show_messages, \
> +                        N_("also show informational/conflict messages")), \
> +               OPT_SET_INT('z', NULL, &line_termination, \
> +                           N_("separate paths with the NUL character"), '\0'), \
> +               OPT_BOOL_F('l', "exclude-modes-oids-stages", \
> +                          &o.exclude_modes_oids_stages, \
> +                          N_("list conflicted files without modes/oids/stages"), \
> +                          PARSE_OPT_NONEG), \
> +               OPT_BOOL_F(0, "allow-unrelated-histories", \
> +                          &o.allow_unrelated_histories, \
> +                          N_("allow merging unrelated histories"), \
> +                          PARSE_OPT_NONEG)
> +
> +static int trivial_merge(int argc, const char **argv, const char *prefix)
>  {
>         struct repository *r = the_repository;
>         struct tree_desc t[3];
>         void *buf1, *buf2, *buf3;
> +       struct merge_tree_options o = { 0 };
> +       const char * const usage[] = {
> +               BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
> +               NULL,
> +       };
> +       struct option options[] = {
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
> +               OPT_END()
> +       };
> +       const char *base, *branch1, *branch2;
> +
> +       argc = parse_options(argc, argv, prefix, options, usage,
> +                            PARSE_OPT_STOP_AT_NON_OPTION);
> +       if (argc != 3)
> +               BUG("should have ensured remaining argc == 3 already! Got %d", argc);
> +
> +       base = argv[0];
> +       branch1 = argv[1];
> +       branch2 = argv[2];;
>
>         buf1 = get_tree_descriptor(r, t+0, base);
>         buf2 = get_tree_descriptor(r, t+1, branch1);
> @@ -391,24 +445,34 @@ static int trivial_merge(const char *base,
>         return 0;
>  }
>
> -struct merge_tree_options {
> -       int mode;
> -       int allow_unrelated_histories;
> -       int show_messages;
> -       int exclude_modes_oids_stages;
> -};
> -
> -static int real_merge(struct merge_tree_options *o,
> -                     const char *branch1, const char *branch2,
> -                     const char *prefix)
> +static int real_merge(int argc, const char **argv, const char *prefix)
>  {
>         struct commit *parent1, *parent2;
>         struct commit_list *common;
>         struct commit_list *merge_bases = NULL;
>         struct commit_list *j;
> +       struct merge_tree_options o = { .show_messages = 1 };
>         struct merge_options opt;
>         struct merge_result result = { 0 };
>
> +       const char * const usage[] = {
> +               BUILTIN_MERGE_TREE_USAGE_WRITE,
> +               NULL,
> +       };
> +       struct option options[] = {
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
> +               BUILTIN_MERGE_TREE_OPT_WRITE,
> +               OPT_END()
> +       };
> +       const char *branch1, *branch2;
> +
> +       argc = parse_options(argc, argv, prefix, options, usage,
> +                            PARSE_OPT_STOP_AT_NON_OPTION);
> +       if (argc != 2)
> +               BUG("should have ensured remaining argc == 2 already! Got %d", argc);
> +       branch1 = argv[0];
> +       branch2 = argv[1];
> +
>         parent1 = get_merge_parent(branch1);
>         if (!parent1)
>                 help_unknown_ref(branch1, "merge-tree",
> @@ -431,7 +495,7 @@ static int real_merge(struct merge_tree_options *o,
>          * merge_incore_recursive in merge-ort.h
>          */
>         common = get_merge_bases(parent1, parent2);
> -       if (!common && !o->allow_unrelated_histories)
> +       if (!common && !o.allow_unrelated_histories)
>                 die(_("refusing to merge unrelated histories"));
>         for (j = common; j; j = j->next)
>                 commit_list_insert(j->item, &merge_bases);
> @@ -440,8 +504,8 @@ static int real_merge(struct merge_tree_options *o,
>         if (result.clean < 0)
>                 die(_("failure to merge"));
>
> -       if (o->show_messages == -1)
> -               o->show_messages = !result.clean;
> +       if (o.show_messages == -1)
> +               o.show_messages = !result.clean;
>
>         puts(oid_to_hex(&result.tree->object.oid));
>         if (!result.clean) {
> @@ -453,7 +517,7 @@ static int real_merge(struct merge_tree_options *o,
>                 for (i = 0; i < conflicted_files.nr; i++) {
>                         const char *name = conflicted_files.items[i].string;
>                         struct stage_info *c = conflicted_files.items[i].util;
> -                       if (!o->exclude_modes_oids_stages)
> +                       if (!o.exclude_modes_oids_stages)
>                                 printf("%06o %s %d\t",
>                                        c->mode, oid_to_hex(&c->oid), c->stage);
>                         else if (last && !strcmp(last, name))
> @@ -464,7 +528,7 @@ static int real_merge(struct merge_tree_options *o,
>                 }
>                 string_list_clear(&conflicted_files, 1);
>         }
> -       if (o->show_messages) {
> +       if (o.show_messages) {
>                 putchar(line_termination);
>                 merge_display_update_messages(&opt, &result, stdout);
>         }
> @@ -474,40 +538,32 @@ static int real_merge(struct merge_tree_options *o,
>
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -       struct merge_tree_options o = { .show_messages = -1 };
> +       struct merge_tree_options o;
>         int expected_remaining_argc;
> -       int original_argc;
> -
> +       int original_argc = argc;
> +       struct strvec original_args = STRVEC_INIT;
>         const char * const merge_tree_usage[] = {
> -               N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
> -               N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +               BUILTIN_MERGE_TREE_USAGE_WRITE,
> +               BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
>                 NULL
>         };
>         struct option mt_options[] = {
> -               OPT_CMDMODE(0, "write-tree", &o.mode,
> -                           N_("do a real merge instead of a trivial merge"),
> -                           'w'),
> -               OPT_CMDMODE(0, "trivial-merge", &o.mode,
> -                           N_("do a trivial merge only"), 't'),
> -               OPT_BOOL(0, "messages", &o.show_messages,
> -                        N_("also show informational/conflict messages")),
> -               OPT_SET_INT('z', NULL, &line_termination,
> -                           N_("separate paths with the NUL character"), '\0'),
> -               OPT_BOOL_F('l', "exclude-modes-oids-stages",
> -                          &o.exclude_modes_oids_stages,
> -                          N_("list conflicted files without modes/oids/stages"),
> -                          PARSE_OPT_NONEG),
> -               OPT_BOOL_F(0, "allow-unrelated-histories",
> -                          &o.allow_unrelated_histories,
> -                          N_("allow merging unrelated histories"),
> -                          PARSE_OPT_NONEG),
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
> +               BUILTIN_MERGE_TREE_OPT_WRITE,
>                 OPT_END()
>         };
>
> +       /* We only care about deciding "o.mode" here */
> +       o.mode = 0;
> +       /*
> +        * We need our original argv, and
> +        * PARSE_OPT_KEEP_{ARGV0,UNKNOWN} would do the wrong thing
> +        */
> +       strvec_pushv(&original_args, argv);
>         /* Parse arguments */
> -       original_argc = argc;
>         argc = parse_options(argc, argv, prefix, mt_options,
> -                            merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +                            merge_tree_usage, 0);
>         if (o.mode) {
>                 expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
>                 if (argc != expected_remaining_argc)
> @@ -517,12 +573,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>                         usage_with_options(merge_tree_usage, mt_options);
>                 o.mode = (argc == 2 ? 'w' : 't');
>         }
> -       if (o.mode == 't' && original_argc < argc)
> -               die(_("--trivial-merge is incompatible with all other options"));
>
>         /* Do the relevant type of merge */
>         if (o.mode == 'w')
> -               return real_merge(&o, argv[0], argv[1], prefix);
> +               return real_merge(original_argc, original_args.v, prefix);
>         else
> -               return trivial_merge(argv[0], argv[1], argv[2]);
> +               return trivial_merge(original_argc, original_args.v, prefix);
>  }


Yeah, that's waaay more complex than my code which basically just
needs 6 lines to check the incompatibility; the basic patch for that
part (going back to before this patch we're commenting on) was just:

 +    original_argc = argc - 1;  /* ignore program name, i.e. argv[0] */
...
 +        if (o.mode == 't')
 +            /* Removal of `--trivial-merge` is expected */
 +            original_argc_options--;
...
 +    if (o.mode == 't' && argc < original_argc)
 +        die(_("--trivial-merge is incompatible with all other options"));


> diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
> index 4de089d976d..749bdb6862d 100755
> --- a/t/t4301-merge-tree-write-tree.sh
> +++ b/t/t4301-merge-tree-write-tree.sh
> @@ -92,6 +92,18 @@ test_expect_success 'Barf on too many arguments' '
>         grep "^usage: git merge-tree" expect
>  '
>
> +for opt in $(git merge-tree --git-completion-helper-all)
> +do
> +       if test $opt = "--trivial-merge" || test $opt = "--write-tree"
> +       then
> +               continue
> +       fi
> +
> +       test_expect_success "usage: --trivial-merge is incompatible with $opt" '
> +               test_expect_code 129 git merge-tree --trivial-merge $opt side1 side2 side3
> +       '
> +done

Yep, I was clearly missing an important testcase.  Thanks for
providing one and for pointing out the error in my patch!  Very cool.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Junio C Hamano]

Johannes Altmanninger <aclopte@gmail.com> writes:

> I wonder how the UI of a tool that does non-worktree/non-index cherry-picks
> will look like.  I'd expect it to produce the same output as merge-tree,
> except cherry-pick should probably output a commit OID, not a tree.

A tool that creates a "merge", "cherry-pick" and "revert" should all
output a resulting commit object name, not a tree object.

And "merge-tree" that creates a three-way merge to apply a change to
go from tree A to tree B on top of tree X should take three tree-ish
object names and produce a resulting one tree object name.

Using the "merge-tree" that merges two trees using the third tree as
a common ancestor, these three higher-level tools can perform
"merge", "cherry-pick", and "revert".  They would all take two
commit objects and produce one commit object.



   

Re: [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).

Makes sense.  Ideally things like this should be discoverable by
inspecting the tree object shown as the result of the (conflicted)
merge, but since the design of the output is to show only a single
tree, there is nowhere to store such an extra piece of information
per path (grepping for markers in blobs of course does not count).

I guess an alternative to show four trees when conflicted instead of
one (i.e. the primary tree may either contain only the cleanly
merged paths _or_ also blobs with conflict markers for conflicted
paths; the three other trees record three stages that would be in
the index, if we were performing the same merge using the index),
but a machine-parseable list of paths is fine.

> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;

OK.  The iteration used here makes casual readers wonder why the
helper doesn't make paths unique, but the string list item holds
in its util pointer a pointer to a structure with <stage, mode, oid>
tuple, so it is natural to make the consumer, who wants uniquified
list, responsible for deduping, like this loop.

> +		}
> +		string_list_clear(&conflicted_files, 1);

And the stage-info structure associated with these paths are
deallocated with this call.  Good.

> +	}

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Josh Steadmon]

On 2022.02.02 07:34, Elijah Newren via GitGitGadget wrote:
> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 58731c19422..569485815a0 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -3,26 +3,73 @@ git-merge-tree(1)
>  
>  NAME
>  ----
> -git-merge-tree - Show three-way merge without touching index
> +git-merge-tree - Perform merge without touching index or working tree
>  
>  
>  SYNOPSIS
>  --------
>  [verse]
> -'git merge-tree' <base-tree> <branch1> <branch2>
> +'git merge-tree' [--write-tree] <branch1> <branch2>
> +'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
>  
>  DESCRIPTION
>  -----------
> -Reads three tree-ish, and output trivial merge results and
> -conflicting stages to the standard output.  This is similar to
> -what three-way 'git read-tree -m' does, but instead of storing the
> -results in the index, the command outputs the entries to the
> -standard output.
> -
> -This is meant to be used by higher level scripts to compute
> -merge results outside of the index, and stuff the results back into the
> -index.  For this reason, the output from the command omits
> -entries that match the <branch1> tree.
> +
> +Performs a merge, but does not make any new commits and does not read
> +from or write to either the working tree or index.
> +
> +The second form is deprecated and supported only for backward
> +compatibility.  It will likely be removed in the future, and will not
> +be discussed further in this manual.
> +
> +The first form will merge the two branches, doing a real merge.  A real
> +merge is distinguished from a trivial merge in that it includes:
> +
> +  * three way content merges of individual files
> +  * rename detection
> +  * proper directory/file conflict handling
> +  * recursive ancestor consolidation (i.e. when there is more than one
> +    merge base, creating a virtual merge base by merging the merge bases)
> +  * etc.
> +
> +After the merge completes, it will create a new toplevel tree object.
> +See `OUTPUT` below for details.
> +
> +OUTPUT
> +------
> +
> +For either a successful or conflicted merge, the output from
> +git-merge-tree is simply one line:
> +
> +	<OID of toplevel tree>
> +
> +The printed tree object corresponds to what would be checked out in
> +the working tree at the end of `git merge`, and thus may have files
> +with conflict markers in them.
> +
> +EXIT STATUS
> +-----------
> +
> +For a successful, non-conflicted merge, the exit status is 0.  When the
> +merge has conflicts, the exit status is 1.  If the merge is not able to
> +complete (or start) due to some kind of error, the exit status is
> +something other than 0 or 1.
> +
> +USAGE NOTES
> +-----------
> +
> +git-merge-tree was written to be low-level plumbing, similar to
> +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
> +be used as a part of a series of steps such as
> +
> +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> +       test $? -eq 0 || die "There were conflicts..."
> +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> +       git update-ref $BRANCH1 $NEWCOMMIT
> +
> +However, it does not quite fit into the same category of low-level
> +plumbing commands since the possibility of merge conflicts give it a
> +much higher chance of the command not succeeding.

I found this final paragraph confusing. It seems to be hinting at some
conclusion it expects readers to make, but I haven't been able to figure
out what. Could this be made more explicit, or perhaps dropped
altogether?

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren]

On Thu, Feb 3, 2022 at 8:48 PM Josh Steadmon <steadmon@google.com> wrote:
>
> On 2022.02.02 07:34, Elijah Newren via GitGitGadget wrote:
[...]
> > +USAGE NOTES
> > +-----------
> > +
> > +git-merge-tree was written to be low-level plumbing, similar to
> > +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
> > +be used as a part of a series of steps such as
> > +
> > +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> > +       test $? -eq 0 || die "There were conflicts..."
> > +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> > +       git update-ref $BRANCH1 $NEWCOMMIT
> > +
> > +However, it does not quite fit into the same category of low-level
> > +plumbing commands since the possibility of merge conflicts give it a
> > +much higher chance of the command not succeeding.
>
> I found this final paragraph confusing. It seems to be hinting at some
> conclusion it expects readers to make, but I haven't been able to figure
> out what. Could this be made more explicit, or perhaps dropped
> altogether?

Yep, I'll drop it.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Sat, 29 Jan 2022, Elijah Newren wrote:

> On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
> >
> > Just a heckling from the peanut gallery...
> >
> > Am 29.01.22 um 07:08 schrieb Elijah Newren:
> > > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> > > <Johannes.Schindelin@gmx.de> wrote:
> > >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> > >> missing from the second conflict, in the output we would see stages 1, 2,
> > >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> > >> different conflicts.
> > >
> > > I don't understand why you're fixating on the stage here.  Why would
> > > you want to group all the stage 2s together, count them up, and then
> > > determine there are N conflicting files because there are N stage 2's?
> >
> > Looks like you are misunderstanding Dscho's point: When you have two
> > conflicts, the first with stages 1 and 2, the second with stages 2 and
> > 3, then the 2s occur lumped together when the 4 lines are printed in a
> > row, and that is the cue to the parser where the new conflict begins.
> > Dscho did not mean that all N 2s of should be listed together.
>
> Ah, so...I didn't understand his misunderstanding?  Using stages as a
> cue to the parser where the new conflict begins is broken; you should
> instead check for when the filename listed on a line does not match
> the filename on the previous line.

But that would break down in case of rename/rename conflicts, right?

> In particular, if one conflict has stages 1 and 2, and the next conflict
> has only stage 3, then looking at stages only might cause you to
> accidentally lump unrelated conflicts together.

Precisely. That's why I would love to have a way to deviate from the
output of `ls-files -u`'s format, and have a reliable way to indicate
stages that belong to the same merge conflict.

Thanks,
Dscho

Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts

[Johannes Schindelin]

Hi Elijah,

On Fri, 28 Jan 2022, Elijah Newren wrote:

> On Fri, Jan 28, 2022 at 8:57 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Hi Elijah,
> >
> > On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
> >
> > > From: Elijah Newren <newren@gmail.com>
> > >
> [...]
> > > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > > index fd7a867de60..041a4ac2785 100644
> > > --- a/Documentation/git-merge-tree.txt
> > > +++ b/Documentation/git-merge-tree.txt
> > > @@ -58,6 +58,7 @@ simply one line:
> > >  Whereas for a conflicted merge, the output is by default of the form:
> > >
> > >       <OID of toplevel tree>
> > > +     <Conflicted file list>
> > >       <Informational messages>
> >
> > To distinguish between the list of conflicted files and the informational
> > messages, I think it would be good to insert an empty line, as a
> > separator, like.
>
> Yes, I agree; that's why I did so.  :-)

My concern was that I did not see this empty line reflected in the quoted
diff. I would have expected an empty line between the `<Conflicted [...]>`
and the `<Informational [...]>` line.

Thanks,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Fri, Feb 4, 2022 at 3:10 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 29 Jan 2022, Elijah Newren wrote:
>
> > On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
> > >
> > > Just a heckling from the peanut gallery...
> > >
> > > Am 29.01.22 um 07:08 schrieb Elijah Newren:
> > > > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> > > > <Johannes.Schindelin@gmx.de> wrote:
> > > >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> > > >> missing from the second conflict, in the output we would see stages 1, 2,
> > > >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> > > >> different conflicts.
> > > >
> > > > I don't understand why you're fixating on the stage here.  Why would
> > > > you want to group all the stage 2s together, count them up, and then
> > > > determine there are N conflicting files because there are N stage 2's?
> > >
> > > Looks like you are misunderstanding Dscho's point: When you have two
> > > conflicts, the first with stages 1 and 2, the second with stages 2 and
> > > 3, then the 2s occur lumped together when the 4 lines are printed in a
> > > row, and that is the cue to the parser where the new conflict begins.
> > > Dscho did not mean that all N 2s of should be listed together.
> >
> > Ah, so...I didn't understand his misunderstanding?  Using stages as a
> > cue to the parser where the new conflict begins is broken; you should
> > instead check for when the filename listed on a line does not match
> > the filename on the previous line.
>
> But that would break down in case of rename/rename conflicts, right?
>
> > In particular, if one conflict has stages 1 and 2, and the next conflict
> > has only stage 3, then looking at stages only might cause you to
> > accidentally lump unrelated conflicts together.
>
> Precisely. That's why I would love to have a way to deviate from the
> output of `ls-files -u`'s format, and have a reliable way to indicate
> stages that belong to the same merge conflict.

Ah, attempting to somehow identify and present logical separate
conflicts?  That could be awesome, but I'm not sure it's technically
possible.  It certainly isn't with today's merge-ort.

Let me ask some questions first...

If I understand you correctly then in the event of a rename/rename,
i.e. foo->bar & foo->baz, then you want foo's, bar's, & baz's stages
all listed together.  Right?  And in some way that you can identify
them as related?

If we do so, how do we mark the beginning and the end of what you call
"the same merge conflict"?  If you say it's always 3 stages (with the
possibility of all-zero modes/oids), then what about the rename/rename
case above modified so that the side that did foo->baz also added a
different 'bar'?  That'd be 4 non-zero modes/oids, all of them
relevant.  Or what if the side that did foo->bar also renamed
something else to 'baz', giving us even more non-zero stages for these
three paths?  Perhaps you consider these different conflicts and want
them listed separately -- if so, where does one conflict begin and
another start and which stages are parts of which conflict?

If you are attempting to somehow present the stuff that "belongs to
the same merge conflict" are you also trying to identify what kind of
merge conflict it is?  If so, do you want each type of merge conflict
listed?  For example, let's switch from the example above of logically
disjoint paths coming together to result in more than 3 stages, and
instead pick an example with a single logical path with less than
three stages.  And now let's say that path has multiple conflicts
associated with it; let's use an example with 3: rename/delete +
modify/delete + directory/file (one side renames foo->bar while
modifying the contents, the other side deletes foo and adds the
directory 'bar/').  In this case, there is a target file 'bar' that
has two non-zero modes/oids in the ls-files-u output.  If all three
types of conflicts need to be listed, does each need to be listed with
the two non-zero modes/oids (and perhaps one zero mode/oid), resulting
in six listings for 'bar'?  Or would the duplication be confusing
enough that we instead decide to list some merge conflicts with no
stages associated with them?

Thinking about both sets of questions in the last two paragraphs from
a higher level -- should we focus on and group the higher order stages
by the individual conflicts that happen, or should we group them by
the paths that they happen to (which is what `ls-files -u` happens to
do), or should we not bother grouping them and instead duplicate the
higher order stages for each logical conflict it is part of?

As an alternative to duplicating higher order stages, do we sometimes
decide to "lump" separate conflicts together and treat them as one
conflict?  If so, what are the rules on how we decide to lump
conflicts and when not to?  Is there a bright line boundary?  And can
it be done without sucking in arbitrarily more stages for a single
conflict?


Some testcases that might be useful while considering the above
questions: take a look at the "rad", "rrdd", and "mod6" tests of
t6422.  How many "same merge conflicts" are there for each of those,
and what's the boundary between them?  And can you give the answer in
the form of rules that generically handle all cases, rather than just
answering these three specific cases?


I've thought about this problem long and hard before (in part because
of some conversations I had with Edward Thompson about libgit2 and
merging at Git Merge 2020).  It wasn't at all clear to me that libgit2
had considered anything beyond simple rename cases.  The only rules I
ever figured out that made sense to me was "group the stages by target
filename rather than by logical conflict" (so we get `ls -files -u`
populated) and print a meant-for-human message for each logical
conflict (found in the <Informational Messages> section for
merge-tree), and make NO attempt to connect stages by conflict type.

I'm sure that's not what you wanted to hear, and maybe doesn't even
play nicely with your design.  But short of ignoring the edge and
corner cases, I don't see how to solve that problem.  If you do just
want to ignore edge and corner cases, then just ignore the
rename/rename case you brought up in the first place and just use
`ls-files -u`-type output as-is within your design.  If you don't want
to ignore edge cases and want something that works with a specific
design that somehow groups conflicted file stages by conflict type,
then we're going to have to dig into all these questions above and do
some big replumbing within merge-ort.

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Emily Shaffer]

On Wed, Feb 02, 2022 at 07:34:29AM +0000, Elijah Newren via GitGitGadget wrote:
> 
> 
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.
> 
> Note that real merges differ from trivial merges in that they handle:
>   - three way content merges
>   - recursive ancestor consolidation
>   - renames
>   - proper directory/file conflict handling
>   - etc.
> Basically all the stuff you'd expect from `git merge`, just without
> updating the index and working tree.  The initial shell added here does
> nothing more than die with "real merges are not yet implemented", but
> that will be fixed in subsequent commits.
> 
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/merge-tree.c | 61 +++++++++++++++++++++++++++++++++++++-------
>  git.c                |  2 +-
>  2 files changed, 53 insertions(+), 10 deletions(-)
> 
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 914ec960b7e..e98ec8a9f1d 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -3,13 +3,12 @@
>  #include "tree-walk.h"
>  #include "xdiff-interface.h"
>  #include "object-store.h"
> +#include "parse-options.h"
>  #include "repository.h"
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>  
> -static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
> -
>  struct merge_list {
>  	struct merge_list *next;
>  	struct merge_list *link;	/* other stages for this object */
> @@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
>  	return buf;
>  }
>  
> -static int trivial_merge(int argc, const char **argv)
> +static int trivial_merge(const char *base,
> +			 const char *branch1,
> +			 const char *branch2)
>  {
>  	struct repository *r = the_repository;
>  	struct tree_desc t[3];
>  	void *buf1, *buf2, *buf3;
>  
> -	buf1 = get_tree_descriptor(r, t+0, argv[1]);
> -	buf2 = get_tree_descriptor(r, t+1, argv[2]);
> -	buf3 = get_tree_descriptor(r, t+2, argv[3]);
> +	buf1 = get_tree_descriptor(r, t+0, base);
> +	buf2 = get_tree_descriptor(r, t+1, branch1);
> +	buf3 = get_tree_descriptor(r, t+2, branch2);
>  	trivial_merge_trees(t, "");
>  	free(buf1);
>  	free(buf2);
> @@ -384,9 +385,51 @@ static int trivial_merge(int argc, const char **argv)
>  	return 0;
>  }
>  
> +struct merge_tree_options {
> +	int mode;
> +};
> +
> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_CMDMODE(0, "write-tree", &o.mode,
> +			    N_("do a real merge instead of a trivial merge"),
> +			    'w'),
> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
> +			    N_("do a trivial merge only"), 't'),
> +		OPT_END()
> +	};

In the review club last week I had mentioned I thought OPT_CMDMODE
worked well with enums. I found some a reasonably nice example in
builtin/replace.c:cmd_replace(), although I have some Opinions about the
enum declaration placement there. Regardless, I think using an enum
instead of a single character would make this more readable - otherwise
I need to remember what 'w' means when I'm reasoning about how many args
to expect below.


> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +	if (o.mode) {
> +		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.mode = (argc == 2 ? 'w' : 't');
> +	}
> +
> +	/* Do the relevant type of merge */
> +	if (o.mode == 'w')
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }

Sorry for the slow reply.

 - Emily

Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function

[Junio C Hamano]

Emily Shaffer <emilyshaffer@google.com> writes:

>> +	const char * const merge_tree_usage[] = {
>> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> +		NULL
>> +	};
>> +	struct option mt_options[] = {
>> +		OPT_CMDMODE(0, "write-tree", &o.mode,
>> +			    N_("do a real merge instead of a trivial merge"),
>> +			    'w'),
>> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
>> +			    N_("do a trivial merge only"), 't'),
>> +		OPT_END()
>> +	};
>
> In the review club last week I had mentioned I thought OPT_CMDMODE
> worked well with enums. I found some a reasonably nice example in
> builtin/replace.c:cmd_replace(), although I have some Opinions about the
> enum declaration placement there. Regardless, I think using an enum
> instead of a single character would make this more readable - otherwise
> I need to remember what 'w' means when I'm reasoning about how many args
> to expect below.

I am reasonably sure whoever did the above use of OPT_CMDMODE()
feature mimicked an existing one that use it to make options with a
single-letter shorthand mutually exclusive.  If the options are with
short-hand, you wouldn't be complaining that you do not know what
'w' stands for.  When using it with options without short-hand, like
this case, I'd agree it would make it easier to read to use symbolic
constants of some kind.

[PATCH v4 00/12] In-core git merge-tree ("Server side merges")

[Elijah Newren via GitGitGadget]

Note: Depends on en/remerge-diff (to avoid a small textual conflict)

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs. It could
be used to do merges when in a bare repository (thus potentially making it
of interest to Git hosting sites, i.e. "Server side merges"), or for doing a
merge of branches that aren't checked out.

It does not handle similar functionality for cherry-picks, rebases, or
reverts; that is also of interest, but is being deferred for a future
series.

== Quick Overview ==

 * Patches 1-2: preparatory cleanups
 * Patches 3-4: implement basic real merges
 * Patches 5-6: include informational messages ("CONFLICT" messages and
   such) in output
 * Patches 7-10: add ability to include ls-files -u style of info in the
   output
 * Patch 11: support --allow-unrelated-histories
 * Patch 12: augment the manual with potential usage mistakes

== Updates Log ==

Many thanks to the many reviewers who provided good feedback on the most
recent round -- Junio, Ævar, Josh, Emily, and perhaps some others I've
forgotten from review club.

Updates since v3 (or v5, if you include the rounds at
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):

 * Dropped previous patches 5, 6, and 8 of the old series; they weren't
   being used and opened a can of worms[1]
 * [Patch 3] Restructured argument checking, including using an enum
 * [Patch 4] Restored the extended paragraph about the deprecated form of
   git-merge-tree, mentioned write-tree in plumbing commands, and a few
   other small fixups to the documentation
 * [Patch 4] Also provide an example of a clean merge rather than just a
   conflicted one
 * [Patch 6] Fix the incompatible arguments check and add some tests for it
 * [Patch 6] Introduce an anonymize_hash() shell function to make tests
   easier to read (less repeated sed)
 * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
   option for now
 * [Patch 10] When -z passed, the tree in the first section should have a
   trailing NUL rather than trailing newline [1]
   https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/

Stuff NOT included that reviewers brought up in earlier rounds:

 * Very generic (mode, oid, stage, filename) printing formatting[2]
 * Always printing 3 stages for each filename with conflicts[3]
 * Attempting to group conflict stages by logical conflict rather than by
   affected target filepath[4]
 * Providing similar functionality for doing cherry-picks/rebases/reverts,
   i.e. a scheme for three-way merges with a specified merge-base[5]. That's
   being deferred to a future series. [2]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [3]
   https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/
   [4]
   https://lore.kernel.org/git/CABPp-BGCL0onSmpgKuO1k2spYCkx=v27ed9TSSxFib=OdDcLbw@mail.gmail.com/
   [5]
   https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/

Updates since v2:

 * Improved patches from Dscho for the diff_warn_rename_limit() handling
 * Add a -z option for NUL-terminated conflict info lines (so that filenames
   do not have to be quoted)

Updates since v1 (or v3 depending on how you count; thanks to René, Ævar,
Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

Updates since original submission v2 (thanks to Christian, Dscho, Ramsay,
and René for suggestions and comments):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since original submission v1 (thanks to Johannes Altmanninger and
Fabian for suggestions):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

 Documentation/git-merge-tree.txt | 204 ++++++++++++++++++++++++--
 builtin/merge-tree.c             | 189 ++++++++++++++++++++++--
 git.c                            |   2 +-
 merge-ort.c                      | 109 +++++++++-----
 merge-ort.h                      |  29 ++++
 t/t4301-merge-tree-write-tree.sh | 238 +++++++++++++++++++++++++++++++
 6 files changed, 709 insertions(+), 62 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v3:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  63f42df21ae !  3:  60253745f5c merge-tree: add option parsing and initial shell for real merge function
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
       	return 0;
       }
       
     ++enum mode {
     ++	MODE_UNKNOWN,
     ++	MODE_TRIVIAL,
     ++	MODE_REAL,
     ++};
     ++
      +struct merge_tree_options {
      +	int mode;
      +};
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
      +	struct option mt_options[] = {
      +		OPT_CMDMODE(0, "write-tree", &o.mode,
      +			    N_("do a real merge instead of a trivial merge"),
     -+			    'w'),
     ++			    MODE_REAL),
      +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     -+			    N_("do a trivial merge only"), 't'),
     ++			    N_("do a trivial merge only"), MODE_TRIVIAL),
      +		OPT_END()
      +	};
      +
      +	/* Parse arguments */
      +	argc = parse_options(argc, argv, prefix, mt_options,
      +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     -+	if (o.mode) {
     -+		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
     -+		if (argc != expected_remaining_argc)
     ++	switch (o.mode) {
     ++	default:
     ++		BUG("unexpected command mode %d", o.mode);
     ++	case MODE_UNKNOWN:
     ++		switch (argc) {
     ++		default:
      +			usage_with_options(merge_tree_usage, mt_options);
     -+	} else {
     -+		if (argc < 2 || argc > 3)
     -+			usage_with_options(merge_tree_usage, mt_options);
     -+		o.mode = (argc == 2 ? 'w' : 't');
     ++		case 2:
     ++			o.mode = MODE_REAL;
     ++			break;
     ++		case 3:
     ++			o.mode = MODE_TRIVIAL;
     ++			break;
     ++		}
     ++		expected_remaining_argc = argc;
     ++		break;
     ++	case MODE_REAL:
     ++		expected_remaining_argc = 2;
     ++		break;
     ++	case MODE_TRIVIAL:
     ++		expected_remaining_argc = 3;
     ++		break;
      +	}
      +
     ++	if (argc != expected_remaining_argc)
     ++		usage_with_options(merge_tree_usage, mt_options);
     ++
      +	/* Do the relevant type of merge */
     -+	if (o.mode == 'w')
     ++	if (o.mode == MODE_REAL)
      +		return real_merge(&o, argv[0], argv[1]);
      +	else
      +		return trivial_merge(argv[0], argv[1], argv[2]);
  4:  02c29f920d0 !  4:  d7b51da94e6 merge-tree: implement real merges
     @@ Commit message
          conflict/warning messages normally output during a merge, or have quick
          access to a list of files with conflicts.  That is not available in this
          preliminary implementation, but subsequent commits will add that
     -    ability.
     +    ability (meaning that NEWTREE would be a lot more than a tree in the
     +    case of conflicts).
      
          This also marks the traditional trivial merge of merge-tree as
          deprecated.  The trivial merge not only had limited applicability, the
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +Performs a merge, but does not make any new commits and does not read
      +from or write to either the working tree or index.
      +
     -+The second form is deprecated and supported only for backward
     -+compatibility.  It will likely be removed in the future, and will not
     -+be discussed further in this manual.
     -+
      +The first form will merge the two branches, doing a real merge.  A real
      +merge is distinguished from a trivial merge in that it includes:
      +
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +    merge base, creating a virtual merge base by merging the merge bases)
      +  * etc.
      +
     -+After the merge completes, it will create a new toplevel tree object.
     -+See `OUTPUT` below for details.
     ++After the merge completes, the first form will create a new toplevel
     ++tree object.  See `OUTPUT` below for details.
     ++
     ++The second form is deprecated; it is kept for backward compatibility
     ++reasons but may be deleted in the future.  Other than the optional
     ++`--trivial-merge`, it accepts no options.  It can only do a trivial
     ++merge.  It reads three tree-ish, and outputs trivial merge results and
     ++conflicting stages to the standard output in a semi-diff format.
     ++Since this was designed for higher level scripts to consume and merge
     ++the results back into the index, it omits entries that match
     ++<branch1>.  The result of this second form is is similar to what
     ++three-way 'git read-tree -m' does, but instead of storing the results
     ++in the index, the command outputs the entries to the standard output.
     ++This form not only has limited applicability, the output format is
     ++also difficult to work with, and it will generally be less performant
     ++than the first form even on successful merges (especially if working
     ++in large repositories).  The remainder of this manual will only
     ++discuss the first form.
      +
      +OUTPUT
      +------
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +For a successful, non-conflicted merge, the exit status is 0.  When the
      +merge has conflicts, the exit status is 1.  If the merge is not able to
      +complete (or start) due to some kind of error, the exit status is
     -+something other than 0 or 1.
     ++something other than 0 or 1 (and the output is unspecified).
      +
      +USAGE NOTES
      +-----------
      +
      +git-merge-tree was written to be low-level plumbing, similar to
     -+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
     -+be used as a part of a series of steps such as
     ++hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
     ++Thus, it could be used as a part of a series of steps such as
      +
      +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
      +       test $? -eq 0 || die "There were conflicts..."
      +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
      +       git update-ref $BRANCH1 $NEWCOMMIT
     -+
     -+However, it does not quite fit into the same category of low-level
     -+plumbing commands since the possibility of merge conflicts give it a
     -+much higher chance of the command not succeeding.
       
       GIT
       ---
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +
      +	git branch side1 &&
      +	git branch side2 &&
     ++	git branch side3 &&
      +
      +	git checkout side1 &&
      +	test_write_lines 1 2 3 4 5 6 >numbers &&
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +	>whatever/empty &&
      +	git add numbers greeting whatever/empty &&
      +	test_tick &&
     -+	git commit -m other-modifications
     ++	git commit -m other-modifications &&
     ++
     ++	git checkout side3 &&
     ++	git mv numbers sequence &&
     ++	test_tick &&
     ++	git commit -m rename-numbers
     ++'
     ++
     ++test_expect_success 'Clean merge' '
     ++	git merge-tree --write-tree side1 side3 >RESULT &&
     ++	q_to_tab <<-EOF >expect &&
     ++	100644 blob $(git rev-parse side1:greeting)Qgreeting
     ++	100644 blob $(git rev-parse side1:numbers)Qsequence
     ++	100644 blob $(git rev-parse side1:whatever)Qwhatever
     ++	EOF
     ++
     ++	git ls-tree $(cat RESULT) >actual &&
     ++	test_cmp expect actual
      +'
      +
      +test_expect_success 'Content merge and a few conflicts' '
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +'
      +
      +test_expect_success 'Barf on too many arguments' '
     -+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
     ++	test_expect_code 129 git merge-tree --write-tree side1 side2 invalid 2>expect &&
      +
      +	grep "^usage: git merge-tree" expect
      +'
  5:  290b42846b5 <  -:  ----------- Introduce a variant of the `warning()` function that takes a `FILE *`
  6:  2083fbe9b2e <  -:  ----------- diff: allow diff_warn_rename_limit to write somewhere besides stderr
  7:  1be858e6aa6 !  5:  58a5594aeb6 merge-ort: split out a separate display_update_messages() function
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
      +
      +	/* Also include needed rename limit adjustment now */
      +	diff_warn_rename_limit("merge.renamelimit",
     -+			       opti->renames.needed_limit, 0, stderr);
     ++			       opti->renames.needed_limit, 0);
      +
      +	trace2_region_leave("merge", "display messages", opt->repo);
      +}
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
       			    struct tree *head,
       			    struct merge_result *result,
      @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
     + 		fclose(fp);
       		trace2_region_leave("merge", "write_auto_merge", opt->repo);
       	}
     - 
     +-
      -	if (display_update_msgs) {
      -		struct merge_options_internal *opti = result->priv;
      -		struct hashmap_iter iter;
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
      -
      -		/* Also include needed rename limit adjustment now */
      -		diff_warn_rename_limit("merge.renamelimit",
     --				       opti->renames.needed_limit, 0, stderr);
     +-				       opti->renames.needed_limit, 0);
      -
      -		trace2_region_leave("merge", "display messages", opt->repo);
      -	}
  8:  04c3bdc44d2 <  -:  ----------- merge-ort: allow update messages to be written to different file stream
  9:  c8ed002408d !  6:  fa55cb4d644 merge-tree: support including merge messages in output
     @@ Documentation/git-merge-tree.txt: git-merge-tree - Perform merge without touchin
       'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
       DESCRIPTION
     -@@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in that it includes:
     - After the merge completes, it will create a new toplevel tree object.
     - See `OUTPUT` below for details.
     +@@ Documentation/git-merge-tree.txt: than the first form even on successful merges (especially if working
     + in large repositories).  The remainder of this manual will only
     + discuss the first form.
       
      +OPTIONS
      +-------
     @@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in
      +	<Informational messages>
      +
      +These are discussed individually below.
     -+
     + 
     +-The printed tree object corresponds to what would be checked out in
     +-the working tree at the end of `git merge`, and thus may have files
     +-with conflict markers in them.
      +OID of toplevel tree
      +~~~~~~~~~~~~~~~~~~~~
      +
     @@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in
      +
      +This always starts with a blank line to separate it from the previous
      +section, and then has free-form messages about the merge, such as:
     - 
     --The printed tree object corresponds to what would be checked out in
     --the working tree at the end of `git merge`, and thus may have files
     --with conflict markers in them.
     ++
      +  * "Auto-merging <file>"
      +  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
      +  * "Failed to merge submodule <submodule> (<reason>)"
     @@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in
       
       EXIT STATUS
       -----------
     -@@ Documentation/git-merge-tree.txt: be used as a part of a series of steps such as
     - 
     - However, it does not quite fit into the same category of low-level
     - plumbing commands since the possibility of merge conflicts give it a
     --much higher chance of the command not succeeding.
     -+much higher chance of the command not succeeding (and NEWTREE containing
     -+a bunch of stuff other than just a toplevel tree).
     +@@ Documentation/git-merge-tree.txt: Thus, it could be used as a part of a series of steps such as
     +        NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
     +        git update-ref $BRANCH1 $NEWCOMMIT
       
     ++Note that when the exit status is non-zero, NEWTREE in this sequence
     ++will contain a lot more output than just a tree.
     ++
       GIT
       ---
     + Part of the linkgit:git[1] suite
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     +@@ builtin/merge-tree.c: enum mode {
       
       struct merge_tree_options {
       	int mode;
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	puts(oid_to_hex(&result.tree->object.oid));
      +	if (o->show_messages) {
      +		printf("\n");
     -+		merge_display_update_messages(&opt, &result, stdout);
     ++		merge_display_update_messages(&opt, &result);
      +	}
       	merge_finalize(&opt, &result);
       	return !result.clean; /* result.clean < 0 handled above */
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		NULL
       	};
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			    'w'),
     + 			    MODE_REAL),
       		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     - 			    N_("do a trivial merge only"), 't'),
     + 			    N_("do a trivial merge only"), MODE_TRIVIAL),
      +		OPT_BOOL(0, "messages", &o.show_messages,
      +			 N_("also show informational/conflict messages")),
       		OPT_END()
       	};
       
       	/* Parse arguments */
     -+	original_argc = argc;
     ++	original_argc = argc - 1; /* ignoring argv[0] */
       	argc = parse_options(argc, argv, prefix, mt_options,
       			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     - 	if (o.mode) {
     + 	switch (o.mode) {
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			usage_with_options(merge_tree_usage, mt_options);
     - 		o.mode = (argc == 2 ? 'w' : 't');
     + 		break;
     + 	case MODE_TRIVIAL:
     + 		expected_remaining_argc = 3;
     ++		/* Removal of `--trivial-merge` is expected */
     ++		original_argc--;
     + 		break;
       	}
     -+	if (o.mode == 't' && original_argc < argc)
     ++	if (o.mode == MODE_TRIVIAL && argc < original_argc)
      +		die(_("--trivial-merge is incompatible with all other options"));
       
     - 	/* Do the relevant type of merge */
     - 	if (o.mode == 'w')
     + 	if (argc != expected_remaining_argc)
     + 		usage_with_options(merge_tree_usage, mt_options);
      
       ## t/t4301-merge-tree-write-tree.sh ##
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
       	grep "^usage: git merge-tree" expect
       '
       
     ++anonymize_hash() {
     ++	sed -e "s/[0-9a-f]\{40,\}/HASH/g" "$@"
     ++}
     ++
      +test_expect_success 'test conflict notices and such' '
      +	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	# Expected results:
      +	#   "greeting" should merge with conflicts
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many argument
      +
      +	test_cmp expect actual
      +'
     ++
     ++for opt in $(git merge-tree --git-completion-helper-all)
     ++do
     ++	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
     ++	then
     ++		continue
     ++	fi
     ++
     ++	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
     ++		test_expect_code 128 git merge-tree --trivial-merge $opt side1 side2 side3
     ++	'
     ++done
      +
       test_done
 10:  1c2a3f5ef63 !  7:  f3ad7add515 merge-ort: provide a merge_get_conflicted_files() helper function
     @@ merge-ort.h
       
       struct commit;
       struct tree;
     -@@ merge-ort.h: void merge_display_update_messages(struct merge_options *opt,
     - 				   struct merge_result *result,
     - 				   FILE *stream);
     +@@ merge-ort.h: void merge_switch_to_result(struct merge_options *opt,
     + void merge_display_update_messages(struct merge_options *opt,
     + 				   struct merge_result *result);
       
      +struct stage_info {
      +	struct object_id oid;
 11:  9c2389eef0e !  8:  6058190d1b1 merge-tree: provide a list of which files have conflicts
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
      +	}
       	if (o->show_messages) {
       		printf("\n");
     - 		merge_display_update_messages(&opt, &result, stdout);
     + 		merge_display_update_messages(&opt, &result);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
       
       	/* Do the relevant type of merge */
     - 	if (o.mode == 'w')
     + 	if (o.mode == MODE_REAL)
      -		return real_merge(&o, argv[0], argv[1]);
      +		return real_merge(&o, argv[0], argv[1], prefix);
       	else
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and
       
       	Auto-merging greeting
       	CONFLICT (content): Merge conflict in greeting
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
     - 	test_cmp expect actual
     - '
     +@@ t/t4301-merge-tree-write-tree.sh: do
     + 	'
     + done
       
      +test_expect_success 'Just the conflicted files without the messages' '
      +	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	test_write_lines HASH greeting whatever~side1 >expect &&
      +
 12:  2188a8ca1e7 !  9:  435f66ea699 merge-tree: provide easy access to `ls-files -u` style info
     @@ Commit message
          Much like `git merge` updates the index with information of the form
              (mode, oid, stage, name)
          provide this output for conflicted files for merge-tree as well.
     -    Provide an --exclude-modes-oids-stages/-l option for users to exclude
     -    the mode, oid, and stage and only get the list of conflicted filenames.
     +    Provide a --name-only option for users to exclude the mode, oid, and
     +    stage and only get the list of conflicted filenames.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
     +@@ Documentation/git-merge-tree.txt: discuss the first form.
       OPTIONS
       -------
       
     -+--exclude-oids-and-modes::
     -+	Instead of writing a list of (mode, oid, stage, path) tuples
     -+	to output for conflicted files, just provide a list of
     -+	filenames with conflicts.
     ++--name-only::
     ++	In the Conflicted file info section, instead of writing a list
     ++	of (mode, oid, stage, path) tuples to output for conflicted
     ++	files, just provide a list of filenames with conflicts (and
     ++	do not list filenames multiple times if they have multiple
     ++	conflicting stages).
      +
       --[no-]messages::
       	Write any informational messages such as "Auto-merging <path>"
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
      +
      +The filename will be quoted as explained for the configuration
      +variable `core.quotePath` (see linkgit:git-config[1]).  However, if
     -+the `--exclude-oids-and-modes` option is passed, the mode, object, and
     -+stage will be omitted.
     ++the `--name-only` option is passed, the mode, object, and stage will
     ++be omitted.
       
       Informational messages
       ~~~~~~~~~~~~~~~~~~~~~~
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
       
         * "Auto-merging <file>"
         * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
     -@@ Documentation/git-merge-tree.txt: plumbing commands since the possibility of merge conflicts give it a
     - much higher chance of the command not succeeding (and NEWTREE containing
     - a bunch of stuff other than just a toplevel tree).
     +@@ Documentation/git-merge-tree.txt: Thus, it could be used as a part of a series of steps such as
     + Note that when the exit status is non-zero, NEWTREE in this sequence
     + will contain a lot more output than just a tree.
       
      +git-merge-tree was written to provide users with the same information
      +that they'd have access to if using `git merge`:
     @@ Documentation/git-merge-tree.txt: plumbing commands since the possibility of mer
       Part of the linkgit:git[1] suite
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     +@@ builtin/merge-tree.c: enum mode {
       struct merge_tree_options {
       	int mode;
       	int show_messages;
     -+	int exclude_modes_oids_stages;
     ++	int name_only;
       };
       
       static int real_merge(struct merge_tree_options *o,
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			const char *name = conflicted_files.items[i].string;
      -			if (last && !strcmp(last, name))
      +			struct stage_info *c = conflicted_files.items[i].util;
     -+			if (!o->exclude_modes_oids_stages)
     ++			if (!o->name_only)
      +				printf("%06o %s %d\t",
      +				       c->mode, oid_to_hex(&c->oid), c->stage);
      +			else if (last && !strcmp(last, name))
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			write_name_quoted_relative(
       				name, prefix, stdout, line_termination);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			    N_("do a trivial merge only"), 't'),
     + 			    N_("do a trivial merge only"), MODE_TRIVIAL),
       		OPT_BOOL(0, "messages", &o.show_messages,
       			 N_("also show informational/conflict messages")),
     -+		OPT_BOOL_F('l', "exclude-modes-oids-stages",
     -+			   &o.exclude_modes_oids_stages,
     -+			   N_("list conflicted files without modes/oids/stages"),
     ++		OPT_BOOL_F(0, "name-only",
     ++			   &o.name_only,
     ++			   N_("list filenames without modes/oids/stages"),
      +			   PARSE_OPT_NONEG),
       		OPT_END()
       	};
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Content merge and a few c
       	test_when_finished "git reset --hard" &&
       
       	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
     - '
     +@@ t/t4301-merge-tree-write-tree.sh: anonymize_hash() {
     + }
       
       test_expect_success 'test conflict notices and such' '
      -	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
     - 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	test_expect_code 1 git merge-tree --write-tree --name-only side1 side2 >out &&
     + 	anonymize_hash out >actual &&
       
       	# Expected results:
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
     - '
     +@@ t/t4301-merge-tree-write-tree.sh: do
     + done
       
       test_expect_success 'Just the conflicted files without the messages' '
      -	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
     - 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	test_expect_code 1 git merge-tree --write-tree --no-messages --name-only side1 side2 >out &&
     + 	anonymize_hash out >actual &&
       
       	test_write_lines HASH greeting whatever~side1 >expect &&
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Just the conflicted files without the messages' '
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Just the conflicted files
       
      +test_expect_success 'Check conflicted oids and modes without messages' '
      +	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	# Compare the basic output format
      +	q_to_tab >expect <<-\EOF &&
 13:  52339b396fa ! 10:  5f253e298b3 merge-tree: allow `ls-files -u` style info to be NUL terminated
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
     +@@ Documentation/git-merge-tree.txt: discuss the first form.
       OPTIONS
       -------
       
     @@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
      +	newline.  Also begin the messages section with a NUL character
      +	instead of a newline.  See OUTPUT below for more information.
      +
     - --exclude-oids-and-modes::
     - 	Instead of writing a list of (mode, oid, stage, path) tuples
     - 	to output for conflicted files, just provide a list of
     + --name-only::
     + 	In the Conflicted file info section, instead of writing a list
     + 	of (mode, oid, stage, path) tuples to output for conflicted
      @@ Documentation/git-merge-tree.txt: OID of toplevel tree
       
       This is a tree object that represents what would be checked out in the
       working tree at the end of `git merge`.  If there were conflicts, then
      -files within this tree may have embedded conflict markers.
      +files within this tree may have embedded conflict markers.  This section
     -+is always followed by a newline.
     ++is always followed by a newline (or NUL if `-z` is passed).
       
       Conflicted file info
       ~~~~~~~~~~~~~~~~~~~~
      @@ Documentation/git-merge-tree.txt: This is a sequence of lines with the format
       The filename will be quoted as explained for the configuration
       variable `core.quotePath` (see linkgit:git-config[1]).  However, if
     - the `--exclude-oids-and-modes` option is passed, the mode, object, and
     --stage will be omitted.
     -+stage will be omitted.  If `-z` is passed, the "lines" are terminated
     -+by a NUL character instead of a newline character.
     + the `--name-only` option is passed, the mode, object, and stage will
     +-be omitted.
     ++be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
     ++character instead of a newline character.
       
       Informational messages
       ~~~~~~~~~~~~~~~~~~~~~~
     @@ Documentation/git-merge-tree.txt: This is a sequence of lines with the format
       
      
       ## builtin/merge-tree.c ##
     +@@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
     + 	if (o->show_messages == -1)
     + 		o->show_messages = !result.clean;
     + 
     +-	puts(oid_to_hex(&result.tree->object.oid));
     ++	printf("%s%c", oid_to_hex(&result.tree->object.oid), line_termination);
     + 	if (!result.clean) {
     + 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
     + 		const char *last = NULL;
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		string_list_clear(&conflicted_files, 1);
       	}
       	if (o->show_messages) {
      -		printf("\n");
      +		putchar(line_termination);
     - 		merge_display_update_messages(&opt, &result, stdout);
     + 		merge_display_update_messages(&opt, &result);
       	}
       	merge_finalize(&opt, &result);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			    N_("do a trivial merge only"), 't'),
     + 			    N_("do a trivial merge only"), MODE_TRIVIAL),
       		OPT_BOOL(0, "messages", &o.show_messages,
       			 N_("also show informational/conflict messages")),
      +		OPT_SET_INT('z', NULL, &line_termination,
      +			    N_("separate paths with the NUL character"), '\0'),
     - 		OPT_BOOL_F('l', "exclude-modes-oids-stages",
     - 			   &o.exclude_modes_oids_stages,
     - 			   N_("list conflicted files without modes/oids/stages"),
     + 		OPT_BOOL_F(0, "name-only",
     + 			   &o.name_only,
     + 			   N_("list filenames without modes/oids/stages"),
      
       ## t/t4301-merge-tree-write-tree.sh ##
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and modes without messages' '
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
      +	git commit -m "Renamed numbers" &&
      +
      +	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	# Expected results:
      +	#   "greeting" should merge with conflicts
      +	#   "whatever" has *both* a modify/delete and a file/directory conflict
      +	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
     -+	echo HASH >expect &&
     ++	echo HASH | lf_to_nul >expect &&
      +
      +	q_to_tab <<-EOF | lf_to_nul >>expect &&
      +	100644 HASH 1Qgreeting
 14:  c854ecb5f4a ! 11:  e706cf31c6e merge-tree: add a --allow-unrelated-histories flag
     @@ Documentation/git-merge-tree.txt: OPTIONS
       
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     +@@ builtin/merge-tree.c: enum mode {
       
       struct merge_tree_options {
       	int mode;
      +	int allow_unrelated_histories;
       	int show_messages;
     - 	int exclude_modes_oids_stages;
     + 	int name_only;
       };
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	 * merge_incore_recursive in merge-ort.h
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	for (j = common; j; j = j->next)
       		commit_list_insert(j->item, &merge_bases);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			   &o.exclude_modes_oids_stages,
     - 			   N_("list conflicted files without modes/oids/stages"),
     + 			   &o.name_only,
     + 			   N_("list filenames without modes/oids/stages"),
       			   PARSE_OPT_NONEG),
      +		OPT_BOOL_F(0, "allow-unrelated-histories",
      +			   &o.allow_unrelated_histories,
     @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char
      
       ## t/t4301-merge-tree-write-tree.sh ##
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
     - 	>whatever/empty &&
     - 	git add numbers greeting whatever/empty &&
     + 	git checkout side3 &&
     + 	git mv numbers sequence &&
       	test_tick &&
     --	git commit -m other-modifications
     -+	git commit -m other-modifications &&
     +-	git commit -m rename-numbers
     ++	git commit -m rename-numbers &&
      +
      +	git switch --orphan unrelated &&
      +	>something-else &&
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
      +	git commit -m first-commit
       '
       
     - test_expect_success 'Content merge and a few conflicts' '
     + test_expect_success 'Clean merge' '
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'NUL terminated conflicted file "lines"' '
       	test_cmp expect actual
       '
 15:  bc8591bbb63 = 12:  c279236ab65 git-merge-tree.txt: add a section on potentional usage mistakes

-- 
gitgitgadget

[PATCH v4 01/12] merge-tree: rename merge_trees() to trivial_merge_trees()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget

[PATCH v4 02/12] merge-tree: move logic for existing merge into new function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget

[PATCH v4 03/12] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 84 +++++++++++++++++++++++++++++++++++++++-----
 git.c                |  2 +-
 2 files changed, 76 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..0f9d928e862 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,74 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+enum mode {
+	MODE_UNKNOWN,
+	MODE_TRIVIAL,
+	MODE_REAL,
+};
+
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    MODE_REAL),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	switch (o.mode) {
+	default:
+		BUG("unexpected command mode %d", o.mode);
+	case MODE_UNKNOWN:
+		switch (argc) {
+		default:
+			usage_with_options(merge_tree_usage, mt_options);
+		case 2:
+			o.mode = MODE_REAL;
+			break;
+		case 3:
+			o.mode = MODE_TRIVIAL;
+			break;
+		}
+		expected_remaining_argc = argc;
+		break;
+	case MODE_REAL:
+		expected_remaining_argc = 2;
+		break;
+	case MODE_TRIVIAL:
+		expected_remaining_argc = 3;
+		break;
+	}
+
+	if (argc != expected_remaining_argc)
+		usage_with_options(merge_tree_usage, mt_options);
+
+	/* Do the relevant type of merge */
+	if (o.mode == MODE_REAL)
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget

[PATCH v4 04/12] merge-tree: implement real merges

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability (meaning that NEWTREE would be a lot more than a tree in the
case of conflicts).

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  79 +++++++++++++++++++----
 builtin/merge-tree.c             |  44 ++++++++++++-
 t/t4301-merge-tree-write-tree.sh | 106 +++++++++++++++++++++++++++++++
 3 files changed, 216 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..586733ea564 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,81 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The first form will merge the two branches, doing a real merge.  A real
+merge is distinguished from a trivial merge in that it includes:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, the first form will create a new toplevel
+tree object.  See `OUTPUT` below for details.
+
+The second form is deprecated; it is kept for backward compatibility
+reasons but may be deleted in the future.  Other than the optional
+`--trivial-merge`, it accepts no options.  It can only do a trivial
+merge.  It reads three tree-ish, and outputs trivial merge results and
+conflicting stages to the standard output in a semi-diff format.
+Since this was designed for higher level scripts to consume and merge
+the results back into the index, it omits entries that match
+<branch1>.  The result of this second form is is similar to what
+three-way 'git read-tree -m' does, but instead of storing the results
+in the index, the command outputs the entries to the standard output.
+This form not only has limited applicability, the output format is
+also difficult to work with, and it will generally be less performant
+than the first form even on successful merges (especially if working
+in large repositories).  The remainder of this manual will only
+discuss the first form.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1 (and the output is unspecified).
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
+Thus, it could be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 0f9d928e862..af445cb1576 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -398,7 +401,46 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..e4f9421c105
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+	git branch side3 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications &&
+
+	git checkout side3 &&
+	git mv numbers sequence &&
+	test_tick &&
+	git commit -m rename-numbers
+'
+
+test_expect_success 'Clean merge' '
+	git merge-tree --write-tree side1 side3 >RESULT &&
+	q_to_tab <<-EOF >expect &&
+	100644 blob $(git rev-parse side1:greeting)Qgreeting
+	100644 blob $(git rev-parse side1:numbers)Qsequence
+	100644 blob $(git rev-parse side1:whatever)Qwhatever
+	EOF
+
+	git ls-tree $(cat RESULT) >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(cat .git/AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	cat tmp | sed -e s/HEAD/side1/ >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 invalid 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget

[PATCH v4 05/12] merge-ort: split out a separate display_update_messages() function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 78 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 37 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..ebaed98d53a 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4272,43 +4311,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		fclose(fp);
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
-
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v4 06/12] merge-tree: support including merge messages in output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 21 +++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 37 ++++++++++++++++++++++++++
 3 files changed, 95 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 586733ea564..869c38f8223 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
@@ -47,17 +47,47 @@ than the first form even on successful merges (especially if working
 in large repositories).  The remainder of this manual will only
 discuss the first form.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
+
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -79,6 +109,9 @@ Thus, it could be used as a part of a series of steps such as
        NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
        git update-ref $BRANCH1 $NEWCOMMIT
 
+Note that when the exit status is non-zero, NEWTREE in this sequence
+will contain a lot more output than just a tree.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index af445cb1576..d44e8d087b1 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -396,6 +396,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -438,18 +439,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -459,10 +469,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    MODE_REAL),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc - 1; /* ignoring argv[0] */
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	switch (o.mode) {
@@ -486,8 +499,12 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 		break;
 	case MODE_TRIVIAL:
 		expected_remaining_argc = 3;
+		/* Removal of `--trivial-merge` is expected */
+		original_argc--;
 		break;
 	}
+	if (o.mode == MODE_TRIVIAL && argc < original_argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	if (argc != expected_remaining_argc)
 		usage_with_options(merge_tree_usage, mt_options);
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index e4f9421c105..25adcf36bdf 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -103,4 +103,41 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+anonymize_hash() {
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" "$@"
+}
+
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
+for opt in $(git merge-tree --git-completion-helper-all)
+do
+	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
+	then
+		continue
+	fi
+
+	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
+		test_expect_code 128 git merge-tree --trivial-merge $opt side1 side2 side3
+	'
+done
+
 test_done
-- 
gitgitgadget

[PATCH v4 07/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index ebaed98d53a..e1b647b0a40 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4274,6 +4274,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..ddcc39d7270 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -88,6 +89,26 @@ void merge_switch_to_result(struct merge_options *opt,
 void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v4 08/12] merge-tree: provide a list of which files have conflicts

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 869c38f8223..9f7eb03c6eb 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -67,6 +67,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -78,6 +79,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d44e8d087b1..cb4169d2271 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -400,7 +403,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -444,6 +448,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result);
@@ -511,7 +531,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == MODE_REAL)
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 25adcf36bdf..0964c1145e6 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -117,6 +117,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -140,4 +142,13 @@ do
 	'
 done
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v4 09/12] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide a --name-only option for users to exclude the mode, oid, and
stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 32 ++++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++++--
 3 files changed, 60 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 9f7eb03c6eb..6502ee0669e 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -50,6 +50,13 @@ discuss the first form.
 OPTIONS
 -------
 
+--name-only::
+	In the Conflicted file info section, instead of writing a list
+	of (mode, oid, stage, path) tuples to output for conflicted
+	files, just provide a list of filenames with conflicts (and
+	do not list filenames multiple times if they have multiple
+	conflicting stages).
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -67,7 +74,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -79,18 +86,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--name-only` option is passed, the mode, object, and stage will
+be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -120,6 +132,14 @@ Thus, it could be used as a part of a series of steps such as
 Note that when the exit status is non-zero, NEWTREE in this sequence
 will contain a lot more output than just a tree.
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index cb4169d2271..1d4d6637b90 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -400,6 +400,7 @@ enum mode {
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int name_only;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -456,7 +457,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->name_only)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -491,6 +496,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F(0, "name-only",
+			   &o.name_only,
+			   N_("list filenames without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 0964c1145e6..4ee85439372 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -65,6 +65,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(cat .git/AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -108,7 +109,7 @@ anonymize_hash() {
 }
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	# Expected results:
@@ -143,7 +144,7 @@ do
 done
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -151,4 +152,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v4 10/12] merge-tree: allow `ls-files -u` style info to be NUL terminated

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much as `git ls-files` has a -z option, let's add one to merge-tree so
that the conflict-info section can be NUL terminated (and avoid quoting
of unusual filenames).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 21 +++++++++++++----
 builtin/merge-tree.c             |  6 +++--
 t/t4301-merge-tree-write-tree.sh | 40 ++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 6502ee0669e..ada4595b4fc 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -50,6 +50,12 @@ discuss the first form.
 OPTIONS
 -------
 
+-z::
+	Do not quote filenames in the <Conflicted file info> section,
+	and end each filename with a NUL character rather than
+	newline.  Also begin the messages section with a NUL character
+	instead of a newline.  See OUTPUT below for more information.
+
 --name-only::
 	In the Conflicted file info section, instead of writing a list
 	of (mode, oid, stage, path) tuples to output for conflicted
@@ -84,7 +90,8 @@ OID of toplevel tree
 
 This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
-files within this tree may have embedded conflict markers.
+files within this tree may have embedded conflict markers.  This section
+is always followed by a newline (or NUL if `-z` is passed).
 
 Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
@@ -96,19 +103,25 @@ This is a sequence of lines with the format
 The filename will be quoted as explained for the configuration
 variable `core.quotePath` (see linkgit:git-config[1]).  However, if
 the `--name-only` option is passed, the mode, object, and stage will
-be omitted.
+be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
+character instead of a newline character.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
-This always starts with a blank line to separate it from the previous
-sections, and then has free-form messages about the merge, such as:
+This always starts with a blank line (or NUL if `-z` is passed) to
+separate it from the previous sections, and then has free-form
+messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
   * "Failed to merge submodule <submodule> (<reason>)"
   * "Warning: cannot merge binary files: <filename>"
 
+Note that these free-form messages will never have a NUL character
+in or between them, even if -z is passed.  It is simply a large block
+of text taking up the remainder of the output.
+
 EXIT STATUS
 -----------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 1d4d6637b90..825255667b1 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -448,7 +448,7 @@ static int real_merge(struct merge_tree_options *o,
 	if (o->show_messages == -1)
 		o->show_messages = !result.clean;
 
-	puts(oid_to_hex(&result.tree->object.oid));
+	printf("%s%c", oid_to_hex(&result.tree->object.oid), line_termination);
 	if (!result.clean) {
 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
 		const char *last = NULL;
@@ -470,7 +470,7 @@ static int real_merge(struct merge_tree_options *o,
 		string_list_clear(&conflicted_files, 1);
 	}
 	if (o->show_messages) {
-		printf("\n");
+		putchar(line_termination);
 		merge_display_update_messages(&opt, &result);
 	}
 	merge_finalize(&opt, &result);
@@ -496,6 +496,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_SET_INT('z', NULL, &line_termination,
+			    N_("separate paths with the NUL character"), '\0'),
 		OPT_BOOL_F(0, "name-only",
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 4ee85439372..fe476ed1bcc 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -173,4 +173,44 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'NUL terminated conflicted file "lines"' '
+	git checkout -b tweak1 side1 &&
+	test_write_lines zero 1 2 3 4 5 6 >numbers &&
+	git add numbers &&
+	git mv numbers "Αυτά μου φαίνονται κινέζικα" &&
+	git commit -m "Renamed numbers" &&
+
+	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
+	echo HASH | lf_to_nul >expect &&
+
+	q_to_tab <<-EOF | lf_to_nul >>expect &&
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~tweak1
+	100644 HASH 2Qwhatever~tweak1
+	100644 HASH 1QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 2QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 3QΑυτά μου φαίνονται κινέζικα
+
+	EOF
+
+	cat <<-EOF >>expect &&
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
+	CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
+	Auto-merging Αυτά μου φαίνονται κινέζικα
+	CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v4 11/12] merge-tree: add a --allow-unrelated-histories flag

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index ada4595b4fc..3f566477dcb 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -69,6 +69,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 825255667b1..911504ad694 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -399,6 +399,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int name_only;
 };
@@ -436,7 +437,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -502,6 +503,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index fe476ed1bcc..5cb546083d7 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -44,7 +44,13 @@ test_expect_success setup '
 	git checkout side3 &&
 	git mv numbers sequence &&
 	test_tick &&
-	git commit -m rename-numbers
+	git commit -m rename-numbers &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Clean merge' '
@@ -213,4 +219,20 @@ test_expect_success 'NUL terminated conflicted file "lines"' '
 	test_cmp expect actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v4 12/12] git-merge-tree.txt: add a section on potentional usage mistakes

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 3f566477dcb..4520bbf020a 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -158,6 +158,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

Re: [PATCH v4 04/12] merge-tree: implement real merges

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +<branch1>.  The result of this second form is is similar to what

I haven't read the rest of the patches, but this like somehow stood
out in the interdiff.

Re: [PATCH v4 04/12] merge-tree: implement real merges

[Elijah Newren]

On Mon, Feb 14, 2022 at 9:51 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > +<branch1>.  The result of this second form is is similar to what
>
> I haven't read the rest of the patches, but this like somehow stood
> out in the interdiff.

Are you highlighting the double "is" that I need to correct here?  Or
was there something else that stood out to you?

Re: [PATCH v4 04/12] merge-tree: implement real merges

[Ævar Arnfjörð Bjarmason]

On Sat, Feb 12 2022, Elijah Newren via GitGitGadget wrote:

> +# This test is ort-specific
> +if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"

Nit: Needless braces, left over from an earlier version where you used ${VAR:+...} ?

> +test_expect_success 'Clean merge' '
> +	git merge-tree --write-tree side1 side3 >RESULT &&
> +	q_to_tab <<-EOF >expect &&
> +	100644 blob $(git rev-parse side1:greeting)Qgreeting
> +	100644 blob $(git rev-parse side1:numbers)Qsequence
> +	100644 blob $(git rev-parse side1:whatever)Qwhatever
> +	EOF
> +
> +	git ls-tree $(cat RESULT) >actual &&

Nit: to avoid the "cat":

    oid=$(git merge-tree ...) &&
    [...]
    git ls-tree $oid [...]

> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'Content merge and a few conflicts' '
> +	git checkout side1^0 &&
> +	test_must_fail git merge side2 &&
> +	expected_tree=$(cat .git/AUTO_MERGE) &&

Let's do "git rev-parse AUTO_MERGE", to avoid needing REFFILES here.

> [...]
> +	# greeting should have a merge conflict
> +	git show ${expected_tree}:greeting >tmp &&
> +	cat tmp | sed -e s/HEAD/side1/ >expect &&

Nit: More needless "cat", can just be: "sed ... <tmp >expect".

[PATCH v5 00/12] In-core git merge-tree ("Server side merges")

[Elijah Newren via GitGitGadget]

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs. It could
be used to do merges when in a bare repository (thus potentially making it
of interest to Git hosting sites, i.e. "Server side merges"), or for doing a
merge of branches that aren't checked out.

It does not handle similar functionality for cherry-picks, rebases, or
reverts; that is also of interest, but is being deferred for a future
series.

== Quick Overview ==

 * Patches 1-2: preparatory cleanups
 * Patches 3-4: implement basic real merges
 * Patches 5-6: include informational messages ("CONFLICT" messages and
   such) in output
 * Patches 7-10: add ability to include ls-files -u style of info in the
   output
 * Patch 11: support --allow-unrelated-histories
 * Patch 12: augment the manual with potential usage mistakes

== Updates Log ==

Many thanks to the many reviewers who provided good feedback on the most
recent round -- Junio, Ævar, Josh, Emily, and perhaps some others I've
forgotten from review club.

Updates since v4:

 * Fixed double "is" in documentation.
 * Fixed a few small items with testcases

Updates since v3 (or v5, if you include the rounds at
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):

 * Dropped previous patches 5, 6, and 8 of the old series; they weren't
   being used and opened a can of worms[1]
 * [Patch 3] Restructured argument checking, including using an enum
 * [Patch 4] Restored the extended paragraph about the deprecated form of
   git-merge-tree, mentioned write-tree in plumbing commands, and a few
   other small fixups to the documentation
 * [Patch 4] Also provide an example of a clean merge rather than just a
   conflicted one
 * [Patch 6] Fix the incompatible arguments check and add some tests for it
 * [Patch 6] Introduce an anonymize_hash() shell function to make tests
   easier to read (less repeated sed)
 * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
   option for now
 * [Patch 10] When -z passed, the tree in the first section should have a
   trailing NUL rather than trailing newline [1]
   https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/

Stuff NOT included that reviewers brought up in earlier rounds:

 * Very generic (mode, oid, stage, filename) printing formatting[2]
 * Always printing 3 stages for each filename with conflicts[3]
 * Attempting to group conflict stages by logical conflict rather than by
   affected target filepath[4]
 * Providing similar functionality for doing cherry-picks/rebases/reverts,
   i.e. a scheme for three-way merges with a specified merge-base[5]. That's
   being deferred to a future series. [2]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [3]
   https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/
   [4]
   https://lore.kernel.org/git/CABPp-BGCL0onSmpgKuO1k2spYCkx=v27ed9TSSxFib=OdDcLbw@mail.gmail.com/
   [5]
   https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/

Updates since v2:

 * Improved patches from Dscho for the diff_warn_rename_limit() handling
 * Add a -z option for NUL-terminated conflict info lines (so that filenames
   do not have to be quoted)

Updates since v1 (or v3 depending on how you count; thanks to René, Ævar,
Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

Updates since original submission v2 (thanks to Christian, Dscho, Ramsay,
and René for suggestions and comments):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since original submission v1 (thanks to Johannes Altmanninger and
Fabian for suggestions):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

 Documentation/git-merge-tree.txt | 204 ++++++++++++++++++++++++--
 builtin/merge-tree.c             | 189 ++++++++++++++++++++++--
 git.c                            |   2 +-
 merge-ort.c                      | 109 +++++++++-----
 merge-ort.h                      |  29 ++++
 t/t4301-merge-tree-write-tree.sh | 238 +++++++++++++++++++++++++++++++
 6 files changed, 709 insertions(+), 62 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v4:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  60253745f5c =  3:  60253745f5c merge-tree: add option parsing and initial shell for real merge function
  4:  d7b51da94e6 !  4:  7994775a934 merge-tree: implement real merges
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +conflicting stages to the standard output in a semi-diff format.
      +Since this was designed for higher level scripts to consume and merge
      +the results back into the index, it omits entries that match
     -+<branch1>.  The result of this second form is is similar to what
     ++<branch1>.  The result of this second form is similar to what
      +three-way 'git read-tree -m' does, but instead of storing the results
      +in the index, the command outputs the entries to the standard output.
      +This form not only has limited applicability, the output format is
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +. ./test-lib.sh
      +
      +# This test is ort-specific
     -+if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
     ++if test "$GIT_TEST_MERGE_ALGORITHM" != "ort"
      +then
      +	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
      +	test_done
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +'
      +
      +test_expect_success 'Clean merge' '
     -+	git merge-tree --write-tree side1 side3 >RESULT &&
     ++	TREE_OID=$(git merge-tree --write-tree side1 side3) &&
      +	q_to_tab <<-EOF >expect &&
      +	100644 blob $(git rev-parse side1:greeting)Qgreeting
      +	100644 blob $(git rev-parse side1:numbers)Qsequence
      +	100644 blob $(git rev-parse side1:whatever)Qwhatever
      +	EOF
      +
     -+	git ls-tree $(cat RESULT) >actual &&
     ++	git ls-tree $TREE_OID >actual &&
      +	test_cmp expect actual
      +'
      +
      +test_expect_success 'Content merge and a few conflicts' '
      +	git checkout side1^0 &&
      +	test_must_fail git merge side2 &&
     -+	expected_tree=$(cat .git/AUTO_MERGE) &&
     ++	expected_tree=$(git rev-parse AUTO_MERGE) &&
      +
      +	# We will redo the merge, while we are still in a conflicted state!
      +	test_when_finished "git reset --hard" &&
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +
      +	# greeting should have a merge conflict
      +	git show ${expected_tree}:greeting >tmp &&
     -+	cat tmp | sed -e s/HEAD/side1/ >expect &&
     ++	sed -e s/HEAD/side1/ tmp >expect &&
      +	git show ${actual_tree}:greeting >actual &&
      +	test_cmp expect actual
      +'
  5:  58a5594aeb6 =  5:  e0f95e094cf merge-ort: split out a separate display_update_messages() function
  6:  fa55cb4d644 =  6:  90c4adecb23 merge-tree: support including merge messages in output
  7:  f3ad7add515 =  7:  12e2351092a merge-ort: provide a merge_get_conflicted_files() helper function
  8:  6058190d1b1 =  8:  5bb7d3725ad merge-tree: provide a list of which files have conflicts
  9:  435f66ea699 !  9:  3c2ca198cec merge-tree: provide easy access to `ls-files -u` style info
     @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char
      
       ## t/t4301-merge-tree-write-tree.sh ##
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Content merge and a few conflicts' '
     - 	expected_tree=$(cat .git/AUTO_MERGE) &&
     + 	expected_tree=$(git rev-parse AUTO_MERGE) &&
       
       	# We will redo the merge, while we are still in a conflicted state!
      +	git ls-files -u >conflicted-file-info &&
 10:  5f253e298b3 = 10:  6e89e17693a merge-tree: allow `ls-files -u` style info to be NUL terminated
 11:  e706cf31c6e = 11:  6ddd5ffde9c merge-tree: add a --allow-unrelated-histories flag
 12:  c279236ab65 = 12:  7abf633b638 git-merge-tree.txt: add a section on potentional usage mistakes

-- 
gitgitgadget

[PATCH v5 01/12] merge-tree: rename merge_trees() to trivial_merge_trees()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget

[PATCH v5 02/12] merge-tree: move logic for existing merge into new function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget

[PATCH v5 03/12] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 84 +++++++++++++++++++++++++++++++++++++++-----
 git.c                |  2 +-
 2 files changed, 76 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..0f9d928e862 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,74 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+enum mode {
+	MODE_UNKNOWN,
+	MODE_TRIVIAL,
+	MODE_REAL,
+};
+
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    MODE_REAL),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	switch (o.mode) {
+	default:
+		BUG("unexpected command mode %d", o.mode);
+	case MODE_UNKNOWN:
+		switch (argc) {
+		default:
+			usage_with_options(merge_tree_usage, mt_options);
+		case 2:
+			o.mode = MODE_REAL;
+			break;
+		case 3:
+			o.mode = MODE_TRIVIAL;
+			break;
+		}
+		expected_remaining_argc = argc;
+		break;
+	case MODE_REAL:
+		expected_remaining_argc = 2;
+		break;
+	case MODE_TRIVIAL:
+		expected_remaining_argc = 3;
+		break;
+	}
+
+	if (argc != expected_remaining_argc)
+		usage_with_options(merge_tree_usage, mt_options);
+
+	/* Do the relevant type of merge */
+	if (o.mode == MODE_REAL)
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget

[PATCH v5 04/12] merge-tree: implement real merges

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability (meaning that NEWTREE would be a lot more than a tree in the
case of conflicts).

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  79 +++++++++++++++++++----
 builtin/merge-tree.c             |  44 ++++++++++++-
 t/t4301-merge-tree-write-tree.sh | 106 +++++++++++++++++++++++++++++++
 3 files changed, 216 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..589a83738ce 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,81 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The first form will merge the two branches, doing a real merge.  A real
+merge is distinguished from a trivial merge in that it includes:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, the first form will create a new toplevel
+tree object.  See `OUTPUT` below for details.
+
+The second form is deprecated; it is kept for backward compatibility
+reasons but may be deleted in the future.  Other than the optional
+`--trivial-merge`, it accepts no options.  It can only do a trivial
+merge.  It reads three tree-ish, and outputs trivial merge results and
+conflicting stages to the standard output in a semi-diff format.
+Since this was designed for higher level scripts to consume and merge
+the results back into the index, it omits entries that match
+<branch1>.  The result of this second form is similar to what
+three-way 'git read-tree -m' does, but instead of storing the results
+in the index, the command outputs the entries to the standard output.
+This form not only has limited applicability, the output format is
+also difficult to work with, and it will generally be less performant
+than the first form even on successful merges (especially if working
+in large repositories).  The remainder of this manual will only
+discuss the first form.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1 (and the output is unspecified).
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
+Thus, it could be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 0f9d928e862..af445cb1576 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -398,7 +401,46 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..6d321652e21
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "$GIT_TEST_MERGE_ALGORITHM" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+	git branch side3 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications &&
+
+	git checkout side3 &&
+	git mv numbers sequence &&
+	test_tick &&
+	git commit -m rename-numbers
+'
+
+test_expect_success 'Clean merge' '
+	TREE_OID=$(git merge-tree --write-tree side1 side3) &&
+	q_to_tab <<-EOF >expect &&
+	100644 blob $(git rev-parse side1:greeting)Qgreeting
+	100644 blob $(git rev-parse side1:numbers)Qsequence
+	100644 blob $(git rev-parse side1:whatever)Qwhatever
+	EOF
+
+	git ls-tree $TREE_OID >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(git rev-parse AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	sed -e s/HEAD/side1/ tmp >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 invalid 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget

[PATCH v5 05/12] merge-ort: split out a separate display_update_messages() function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 78 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 37 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..ebaed98d53a 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4272,43 +4311,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		fclose(fp);
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
-
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v5 06/12] merge-tree: support including merge messages in output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 21 +++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 37 ++++++++++++++++++++++++++
 3 files changed, 95 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 589a83738ce..f415d6de5a3 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
@@ -47,17 +47,47 @@ than the first form even on successful merges (especially if working
 in large repositories).  The remainder of this manual will only
 discuss the first form.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
+
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -79,6 +109,9 @@ Thus, it could be used as a part of a series of steps such as
        NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
        git update-ref $BRANCH1 $NEWCOMMIT
 
+Note that when the exit status is non-zero, NEWTREE in this sequence
+will contain a lot more output than just a tree.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index af445cb1576..d44e8d087b1 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -396,6 +396,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -438,18 +439,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -459,10 +469,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    MODE_REAL),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc - 1; /* ignoring argv[0] */
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	switch (o.mode) {
@@ -486,8 +499,12 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 		break;
 	case MODE_TRIVIAL:
 		expected_remaining_argc = 3;
+		/* Removal of `--trivial-merge` is expected */
+		original_argc--;
 		break;
 	}
+	if (o.mode == MODE_TRIVIAL && argc < original_argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	if (argc != expected_remaining_argc)
 		usage_with_options(merge_tree_usage, mt_options);
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 6d321652e21..719d81e7173 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -103,4 +103,41 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+anonymize_hash() {
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" "$@"
+}
+
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
+for opt in $(git merge-tree --git-completion-helper-all)
+do
+	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
+	then
+		continue
+	fi
+
+	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
+		test_expect_code 128 git merge-tree --trivial-merge $opt side1 side2 side3
+	'
+done
+
 test_done
-- 
gitgitgadget

[PATCH v5 07/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index ebaed98d53a..e1b647b0a40 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4274,6 +4274,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..ddcc39d7270 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -88,6 +89,26 @@ void merge_switch_to_result(struct merge_options *opt,
 void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v5 08/12] merge-tree: provide a list of which files have conflicts

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index f415d6de5a3..deaeb49ae05 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -67,6 +67,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -78,6 +79,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d44e8d087b1..cb4169d2271 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -400,7 +403,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -444,6 +448,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result);
@@ -511,7 +531,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == MODE_REAL)
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 719d81e7173..8e6dba44288 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -117,6 +117,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -140,4 +142,13 @@ do
 	'
 done
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v5 09/12] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide a --name-only option for users to exclude the mode, oid, and
stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 32 ++++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++++--
 3 files changed, 60 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index deaeb49ae05..67a135e8f5d 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -50,6 +50,13 @@ discuss the first form.
 OPTIONS
 -------
 
+--name-only::
+	In the Conflicted file info section, instead of writing a list
+	of (mode, oid, stage, path) tuples to output for conflicted
+	files, just provide a list of filenames with conflicts (and
+	do not list filenames multiple times if they have multiple
+	conflicting stages).
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -67,7 +74,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -79,18 +86,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--name-only` option is passed, the mode, object, and stage will
+be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -120,6 +132,14 @@ Thus, it could be used as a part of a series of steps such as
 Note that when the exit status is non-zero, NEWTREE in this sequence
 will contain a lot more output than just a tree.
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index cb4169d2271..1d4d6637b90 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -400,6 +400,7 @@ enum mode {
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int name_only;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -456,7 +457,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->name_only)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -491,6 +496,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F(0, "name-only",
+			   &o.name_only,
+			   N_("list filenames without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 8e6dba44288..0ec5f0d3f7e 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -65,6 +65,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(git rev-parse AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -108,7 +109,7 @@ anonymize_hash() {
 }
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	# Expected results:
@@ -143,7 +144,7 @@ do
 done
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -151,4 +152,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v5 10/12] merge-tree: allow `ls-files -u` style info to be NUL terminated

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much as `git ls-files` has a -z option, let's add one to merge-tree so
that the conflict-info section can be NUL terminated (and avoid quoting
of unusual filenames).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 21 +++++++++++++----
 builtin/merge-tree.c             |  6 +++--
 t/t4301-merge-tree-write-tree.sh | 40 ++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 67a135e8f5d..44024b11b1c 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -50,6 +50,12 @@ discuss the first form.
 OPTIONS
 -------
 
+-z::
+	Do not quote filenames in the <Conflicted file info> section,
+	and end each filename with a NUL character rather than
+	newline.  Also begin the messages section with a NUL character
+	instead of a newline.  See OUTPUT below for more information.
+
 --name-only::
 	In the Conflicted file info section, instead of writing a list
 	of (mode, oid, stage, path) tuples to output for conflicted
@@ -84,7 +90,8 @@ OID of toplevel tree
 
 This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
-files within this tree may have embedded conflict markers.
+files within this tree may have embedded conflict markers.  This section
+is always followed by a newline (or NUL if `-z` is passed).
 
 Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
@@ -96,19 +103,25 @@ This is a sequence of lines with the format
 The filename will be quoted as explained for the configuration
 variable `core.quotePath` (see linkgit:git-config[1]).  However, if
 the `--name-only` option is passed, the mode, object, and stage will
-be omitted.
+be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
+character instead of a newline character.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
-This always starts with a blank line to separate it from the previous
-sections, and then has free-form messages about the merge, such as:
+This always starts with a blank line (or NUL if `-z` is passed) to
+separate it from the previous sections, and then has free-form
+messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
   * "Failed to merge submodule <submodule> (<reason>)"
   * "Warning: cannot merge binary files: <filename>"
 
+Note that these free-form messages will never have a NUL character
+in or between them, even if -z is passed.  It is simply a large block
+of text taking up the remainder of the output.
+
 EXIT STATUS
 -----------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 1d4d6637b90..825255667b1 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -448,7 +448,7 @@ static int real_merge(struct merge_tree_options *o,
 	if (o->show_messages == -1)
 		o->show_messages = !result.clean;
 
-	puts(oid_to_hex(&result.tree->object.oid));
+	printf("%s%c", oid_to_hex(&result.tree->object.oid), line_termination);
 	if (!result.clean) {
 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
 		const char *last = NULL;
@@ -470,7 +470,7 @@ static int real_merge(struct merge_tree_options *o,
 		string_list_clear(&conflicted_files, 1);
 	}
 	if (o->show_messages) {
-		printf("\n");
+		putchar(line_termination);
 		merge_display_update_messages(&opt, &result);
 	}
 	merge_finalize(&opt, &result);
@@ -496,6 +496,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_SET_INT('z', NULL, &line_termination,
+			    N_("separate paths with the NUL character"), '\0'),
 		OPT_BOOL_F(0, "name-only",
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 0ec5f0d3f7e..22e03f0939c 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -173,4 +173,44 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'NUL terminated conflicted file "lines"' '
+	git checkout -b tweak1 side1 &&
+	test_write_lines zero 1 2 3 4 5 6 >numbers &&
+	git add numbers &&
+	git mv numbers "Αυτά μου φαίνονται κινέζικα" &&
+	git commit -m "Renamed numbers" &&
+
+	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
+	echo HASH | lf_to_nul >expect &&
+
+	q_to_tab <<-EOF | lf_to_nul >>expect &&
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~tweak1
+	100644 HASH 2Qwhatever~tweak1
+	100644 HASH 1QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 2QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 3QΑυτά μου φαίνονται κινέζικα
+
+	EOF
+
+	cat <<-EOF >>expect &&
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
+	CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
+	Auto-merging Αυτά μου φαίνονται κινέζικα
+	CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v5 11/12] merge-tree: add a --allow-unrelated-histories flag

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 44024b11b1c..d2ff2fa3035 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -69,6 +69,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 825255667b1..911504ad694 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -399,6 +399,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int name_only;
 };
@@ -436,7 +437,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -502,6 +503,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 22e03f0939c..bd1769c624b 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -44,7 +44,13 @@ test_expect_success setup '
 	git checkout side3 &&
 	git mv numbers sequence &&
 	test_tick &&
-	git commit -m rename-numbers
+	git commit -m rename-numbers &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Clean merge' '
@@ -213,4 +219,20 @@ test_expect_success 'NUL terminated conflicted file "lines"' '
 	test_cmp expect actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v5 12/12] git-merge-tree.txt: add a section on potentional usage mistakes

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index d2ff2fa3035..306149fa0e2 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -158,6 +158,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

Re: [PATCH v5 04/12] merge-tree: implement real merges

[René Scharfe]

Am 20.02.22 um 07:54 schrieb Elijah Newren via GitGitGadget:
> From: Elijah Newren <newren@gmail.com>
>
> This adds the ability to perform real merges rather than just trivial
> merges (meaning handling three way content merges, recursive ancestor
> consolidation, renames, proper directory/file conflict handling, and so
> forth).  However, unlike `git merge`, the working tree and index are
> left alone and no branch is updated.
>
> The only output is:
>   - the toplevel resulting tree printed on stdout
>   - exit status of 0 (clean), 1 (conflicts present), anything else
>     (merge could not be performed; unknown if clean or conflicted)
>
> This output is meant to be used by some higher level script, perhaps in
> a sequence of steps like this:
>
>    NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
>    test $? -eq 0 || die "There were conflicts..."
>    NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
>    git update-ref $BRANCH1 $NEWCOMMIT
>
> Note that higher level scripts may also want to access the
> conflict/warning messages normally output during a merge, or have quick
> access to a list of files with conflicts.  That is not available in this
> preliminary implementation, but subsequent commits will add that
> ability (meaning that NEWTREE would be a lot more than a tree in the
> case of conflicts).
>
> This also marks the traditional trivial merge of merge-tree as
> deprecated.  The trivial merge not only had limited applicability, the
> output format was also difficult to work with (and its format
> undocumented), and will generally be less performant than real merges.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  Documentation/git-merge-tree.txt |  79 +++++++++++++++++++----
>  builtin/merge-tree.c             |  44 ++++++++++++-
>  t/t4301-merge-tree-write-tree.sh | 106 +++++++++++++++++++++++++++++++
>  3 files changed, 216 insertions(+), 13 deletions(-)
>  create mode 100755 t/t4301-merge-tree-write-tree.sh
>
> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 58731c19422..589a83738ce 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -3,26 +3,81 @@ git-merge-tree(1)
>
>  NAME
>  ----
> -git-merge-tree - Show three-way merge without touching index
> +git-merge-tree - Perform merge without touching index or working tree
>
>
>  SYNOPSIS
>  --------
>  [verse]
> -'git merge-tree' <base-tree> <branch1> <branch2>
> +'git merge-tree' [--write-tree] <branch1> <branch2>
> +'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
>
>  DESCRIPTION
>  -----------
> -Reads three tree-ish, and output trivial merge results and
> -conflicting stages to the standard output.  This is similar to
> -what three-way 'git read-tree -m' does, but instead of storing the
> -results in the index, the command outputs the entries to the
> -standard output.
> -
> -This is meant to be used by higher level scripts to compute
> -merge results outside of the index, and stuff the results back into the
> -index.  For this reason, the output from the command omits
> -entries that match the <branch1> tree.
> +
> +Performs a merge, but does not make any new commits and does not read
> +from or write to either the working tree or index.
> +
> +The first form will merge the two branches, doing a real merge.  A real
> +merge is distinguished from a trivial merge in that it includes:
> +
> +  * three way content merges of individual files
> +  * rename detection
> +  * proper directory/file conflict handling
> +  * recursive ancestor consolidation (i.e. when there is more than one
> +    merge base, creating a virtual merge base by merging the merge bases)
> +  * etc.
> +
> +After the merge completes, the first form will create a new toplevel
> +tree object.  See `OUTPUT` below for details.
> +
> +The second form is deprecated; it is kept for backward compatibility
> +reasons but may be deleted in the future.  Other than the optional
> +`--trivial-merge`, it accepts no options.  It can only do a trivial
> +merge.  It reads three tree-ish, and outputs trivial merge results and
> +conflicting stages to the standard output in a semi-diff format.
> +Since this was designed for higher level scripts to consume and merge
> +the results back into the index, it omits entries that match
> +<branch1>.  The result of this second form is similar to what
> +three-way 'git read-tree -m' does, but instead of storing the results
> +in the index, the command outputs the entries to the standard output.
> +This form not only has limited applicability, the output format is
> +also difficult to work with, and it will generally be less performant
> +than the first form even on successful merges (especially if working
> +in large repositories).  The remainder of this manual will only
> +discuss the first form.
> +
> +OUTPUT
> +------
> +
> +For either a successful or conflicted merge, the output from
> +git-merge-tree is simply one line:
> +
> +	<OID of toplevel tree>
> +
> +The printed tree object corresponds to what would be checked out in
> +the working tree at the end of `git merge`, and thus may have files
> +with conflict markers in them.
> +
> +EXIT STATUS
> +-----------
> +
> +For a successful, non-conflicted merge, the exit status is 0.  When the
> +merge has conflicts, the exit status is 1.  If the merge is not able to
> +complete (or start) due to some kind of error, the exit status is
> +something other than 0 or 1 (and the output is unspecified).
> +
> +USAGE NOTES
> +-----------
> +
> +git-merge-tree was written to be low-level plumbing, similar to
> +hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
> +Thus, it could be used as a part of a series of steps such as
> +
> +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> +       test $? -eq 0 || die "There were conflicts..."
> +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> +       git update-ref $BRANCH1 $NEWCOMMIT
>
>  GIT
>  ---
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 0f9d928e862..af445cb1576 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -2,6 +2,9 @@
>  #include "builtin.h"
>  #include "tree-walk.h"
>  #include "xdiff-interface.h"
> +#include "help.h"
> +#include "commit-reach.h"
> +#include "merge-ort.h"
>  #include "object-store.h"
>  #include "parse-options.h"
>  #include "repository.h"
> @@ -398,7 +401,46 @@ struct merge_tree_options {
>  static int real_merge(struct merge_tree_options *o,
>  		      const char *branch1, const char *branch2)
>  {
> -	die(_("real merges are not yet implemented"));
> +	struct commit *parent1, *parent2;
> +	struct commit_list *common;
> +	struct commit_list *merge_bases = NULL;
> +	struct commit_list *j;
> +	struct merge_options opt;
> +	struct merge_result result = { 0 };
> +
> +	parent1 = get_merge_parent(branch1);
> +	if (!parent1)
> +		help_unknown_ref(branch1, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	parent2 = get_merge_parent(branch2);
> +	if (!parent2)
> +		help_unknown_ref(branch2, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	init_merge_options(&opt, the_repository);
> +
> +	opt.show_rename_progress = 0;
> +
> +	opt.branch1 = branch1;
> +	opt.branch2 = branch2;
> +
> +	/*
> +	 * Get the merge bases, in reverse order; see comment above
> +	 * merge_incore_recursive in merge-ort.h
> +	 */
> +	common = get_merge_bases(parent1, parent2);
> +	if (!common)
> +		die(_("refusing to merge unrelated histories"));
> +	for (j = common; j; j = j->next)
> +		commit_list_insert(j->item, &merge_bases);

This loop creates a reversed copy of "common".  You could use
reverse_commit_list() instead to do it in-place and avoid the
allocations.  Only the copy, "merge_bases", is used below.

> +
> +	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> +	if (result.clean < 0)
> +		die(_("failure to merge"));
> +	puts(oid_to_hex(&result.tree->object.oid));
> +	merge_finalize(&opt, &result);
> +	return !result.clean; /* result.clean < 0 handled above */
>  }
>
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
> new file mode 100755
> index 00000000000..6d321652e21
> --- /dev/null
> +++ b/t/t4301-merge-tree-write-tree.sh
> @@ -0,0 +1,106 @@
> +#!/bin/sh
> +
> +test_description='git merge-tree --write-tree'
> +
> +. ./test-lib.sh
> +
> +# This test is ort-specific
> +if test "$GIT_TEST_MERGE_ALGORITHM" != "ort"
> +then
> +	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
> +	test_done
> +fi
> +
> +test_expect_success setup '
> +	test_write_lines 1 2 3 4 5 >numbers &&
> +	echo hello >greeting &&
> +	echo foo >whatever &&
> +	git add numbers greeting whatever &&
> +	test_tick &&
> +	git commit -m initial &&
> +
> +	git branch side1 &&
> +	git branch side2 &&
> +	git branch side3 &&
> +
> +	git checkout side1 &&
> +	test_write_lines 1 2 3 4 5 6 >numbers &&
> +	echo hi >greeting &&
> +	echo bar >whatever &&
> +	git add numbers greeting whatever &&
> +	test_tick &&
> +	git commit -m modify-stuff &&
> +
> +	git checkout side2 &&
> +	test_write_lines 0 1 2 3 4 5 >numbers &&
> +	echo yo >greeting &&
> +	git rm whatever &&
> +	mkdir whatever &&
> +	>whatever/empty &&
> +	git add numbers greeting whatever/empty &&
> +	test_tick &&
> +	git commit -m other-modifications &&
> +
> +	git checkout side3 &&
> +	git mv numbers sequence &&
> +	test_tick &&
> +	git commit -m rename-numbers
> +'
> +
> +test_expect_success 'Clean merge' '
> +	TREE_OID=$(git merge-tree --write-tree side1 side3) &&
> +	q_to_tab <<-EOF >expect &&
> +	100644 blob $(git rev-parse side1:greeting)Qgreeting
> +	100644 blob $(git rev-parse side1:numbers)Qsequence
> +	100644 blob $(git rev-parse side1:whatever)Qwhatever
> +	EOF
> +
> +	git ls-tree $TREE_OID >actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'Content merge and a few conflicts' '
> +	git checkout side1^0 &&
> +	test_must_fail git merge side2 &&
> +	expected_tree=$(git rev-parse AUTO_MERGE) &&
> +
> +	# We will redo the merge, while we are still in a conflicted state!
> +	test_when_finished "git reset --hard" &&
> +
> +	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
> +	actual_tree=$(head -n 1 RESULT) &&
> +
> +	# Due to differences of e.g. "HEAD" vs "side1", the results will not
> +	# exactly match.  Dig into individual files.
> +
> +	# Numbers should have three-way merged cleanly
> +	test_write_lines 0 1 2 3 4 5 6 >expect &&
> +	git show ${actual_tree}:numbers >actual &&
> +	test_cmp expect actual &&
> +
> +	# whatever and whatever~<branch> should have same HASHES
> +	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
> +	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
> +	test_cmp expect actual &&
> +
> +	# greeting should have a merge conflict
> +	git show ${expected_tree}:greeting >tmp &&
> +	sed -e s/HEAD/side1/ tmp >expect &&
> +	git show ${actual_tree}:greeting >actual &&
> +	test_cmp expect actual
> +'
> +
> +test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
> +	# Mis-spell with single "s" instead of double "s"
> +	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
> +
> +	grep "error: unknown option.*mesages" expect
> +'
> +
> +test_expect_success 'Barf on too many arguments' '
> +	test_expect_code 129 git merge-tree --write-tree side1 side2 invalid 2>expect &&
> +
> +	grep "^usage: git merge-tree" expect
> +'
> +
> +test_done

Re: [PATCH v5 00/12] In-core git merge-tree ("Server side merges")

[Ævar Arnfjörð Bjarmason]

On Sun, Feb 20 2022, Elijah Newren via GitGitGadget wrote:

> == Updates Log ==
>
> Many thanks to the many reviewers who provided good feedback on the most
> recent round -- Junio, Ævar, Josh, Emily, and perhaps some others I've
> forgotten from review club.
>
> Updates since v4:
>
>  * Fixed double "is" in documentation.
>  * Fixed a few small items with testcases
>
> Updates since v3 (or v5, if you include the rounds at
> https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):
>
>  * Dropped previous patches 5, 6, and 8 of the old series; they weren't
>    being used and opened a can of worms[1]
>  * [Patch 3] Restructured argument checking, including using an enum
>  * [Patch 4] Restored the extended paragraph about the deprecated form of
>    git-merge-tree, mentioned write-tree in plumbing commands, and a few
>    other small fixups to the documentation
>  * [Patch 4] Also provide an example of a clean merge rather than just a
>    conflicted one
>  * [Patch 6] Fix the incompatible arguments check and add some tests for it
>  * [Patch 6] Introduce an anonymize_hash() shell function to make tests
>    easier to read (less repeated sed)
>  * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
>    option for now
>  * [Patch 10] When -z passed, the tree in the first section should have a
>    trailing NUL rather than trailing newline [1]
>    https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/
>
> Stuff NOT included that reviewers brought up in earlier rounds:
>
>  * Very generic (mode, oid, stage, filename) printing formatting[2]
>  * Always printing 3 stages for each filename with conflicts[3]
>  * Attempting to group conflict stages by logical conflict rather than by
>    affected target filepath[4]
>  * Providing similar functionality for doing cherry-picks/rebases/reverts,
>    i.e. a scheme for three-way merges with a specified merge-base[5]. That's
>    being deferred to a future series. [2]
>    https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
>    [3]
>    https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/
>    [4]
>    https://lore.kernel.org/git/CABPp-BGCL0onSmpgKuO1k2spYCkx=v27ed9TSSxFib=OdDcLbw@mail.gmail.com/
>    [5]
>    https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/

I've looked through this, I think it all looks good overall & that the
things that needed to be addressed (as opposed to my --format rambling)
have been.

I think all the code should be ready for "next".

I suggested (I think around getopts discussion) in an earlier that the
code would have been easier with a new built-in, but if we're
deprecating the existing "mode" I think using the name is probably
better in the end.

I find the resulting documentation to be really hard to grok though
because we're effectively describing two different commands. The current
docs are small: https://git-scm.com/docs/git-merge-tree

I built the tip of this series and read the manpage, and found myself
needing to carefully squint to see what referred to what mode in the
docs.

E.g. by the time it's discussing "-z" and other options the reader needs
to be astute really aware of the context, and infer from the lack of
"<options>" on the "--trivial-merge" that these options refer to the
"--write-tree" only.

The same goes for the rest of "--trivial-merge". I.e. I found myself
needing to read the whole docs word-by-word (no skimming!) to see if
OUTPUT etc. was going to describe its output, or just the "new" mode.

It is my NSHO that man pages should be structured for the impatient
reader :)

Then when you say "git-merge-tree was written to be[...]" I thought "ah
ha! surely this will discuss the since-2005 implemented mode", but "was
written to be" is referring to code new in this series.

The below patch-on-top addresses all those concerns. Basically I just
added a line to the top of the DESCRIPTION saying that you should read a
"DEPRECATED DESCRIPTION" section at the end for "--trivial-merge", and
that all of the rest is talking about the "--write-tree" mode.

I then edited various prose to do away with the now-unnecessary "the
first form" etc.

This diff is better against "master" in that you'll see that the current
merge-tree DESCRIPTION section isn't touched at all (it's now just under
a new heading), but this diff is against the tip of your series.

There's various other small fixes while I was it it here, e.g. all your
cross-section links were using some pseudo-not-quite-ASCIIDOC syntax
that doesn't work. Now it uses the right syntax. Ditto link-ifying
references to "mktag" etc.

I don't know if you'd consider this for a v6, or if I should just submit
this on top myself, but in any case here it is. I'll leave it to you how
you'd like to proceed with it:

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 306149fa0e2..723b1995426 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,17 +9,24 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
-'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
+'git merge-tree' --write-tree [<options>] <branch1> <branch2>
+'git merge-tree' --trivial-merge <base-tree> <branch1> <branch2>
 
+[[NEWMERGE]]
 DESCRIPTION
 -----------
 
-Performs a merge, but does not make any new commits and does not read
-from or write to either the working tree or index.
+This command has a modern `--write-tree` mode and a deprecated
+`--trivial-merge` mode. The rest of this documentation describes
+modern `--write-tree` mode unless otherwise specified. see
+<<DEPMERGE,DEPRECATED DESCRIPTION>> below for a summary of the
+`--trivial-merge` mode.
 
-The first form will merge the two branches, doing a real merge.  A real
-merge is distinguished from a trivial merge in that it includes:
+Performs a "real" merge, but does not make any new commits and does
+not read from or write to either the working tree or index.
+
+The performed merge will use the same feature as the "real"
+linkgit:git-merge[1], including:
 
   * three way content merges of individual files
   * rename detection
@@ -28,24 +35,8 @@ merge is distinguished from a trivial merge in that it includes:
     merge base, creating a virtual merge base by merging the merge bases)
   * etc.
 
-After the merge completes, the first form will create a new toplevel
-tree object.  See `OUTPUT` below for details.
-
-The second form is deprecated; it is kept for backward compatibility
-reasons but may be deleted in the future.  Other than the optional
-`--trivial-merge`, it accepts no options.  It can only do a trivial
-merge.  It reads three tree-ish, and outputs trivial merge results and
-conflicting stages to the standard output in a semi-diff format.
-Since this was designed for higher level scripts to consume and merge
-the results back into the index, it omits entries that match
-<branch1>.  The result of this second form is similar to what
-three-way 'git read-tree -m' does, but instead of storing the results
-in the index, the command outputs the entries to the standard output.
-This form not only has limited applicability, the output format is
-also difficult to work with, and it will generally be less performant
-than the first form even on successful merges (especially if working
-in large repositories).  The remainder of this manual will only
-discuss the first form.
+After the merge completes, a newtoplevel tree object is created.  See
+`OUTPUT` below for details.
 
 OPTIONS
 -------
@@ -54,7 +45,7 @@ OPTIONS
 	Do not quote filenames in the <Conflicted file info> section,
 	and end each filename with a NUL character rather than
 	newline.  Also begin the messages section with a NUL character
-	instead of a newline.  See OUTPUT below for more information.
+	instead of a newline.  See <<OUTPUT>> below for more information.
 
 --name-only::
 	In the Conflicted file info section, instead of writing a list
@@ -74,11 +65,12 @@ OPTIONS
 	share no common history.  This flag can be given to override that
 	check and make the merge proceed anyway.
 
+[[OUTPUT]]
 OUTPUT
 ------
 
-By default, for a successful merge, the output from git-merge-tree is
-simply one line:
+For a successful merge, the output from git-merge-tree is simply one
+line:
 
 	<OID of toplevel tree>
 
@@ -90,6 +82,7 @@ Whereas for a conflicted merge, the output is by default of the form:
 
 These are discussed individually below.
 
+[[OIDTLT]]
 OID of toplevel tree
 ~~~~~~~~~~~~~~~~~~~~
 
@@ -98,6 +91,7 @@ working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.  This section
 is always followed by a newline (or NUL if `-z` is passed).
 
+[[CFI]]
 Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
@@ -111,6 +105,7 @@ the `--name-only` option is passed, the mode, object, and stage will
 be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
 character instead of a newline character.
 
+[[IM]]
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
@@ -138,72 +133,94 @@ something other than 0 or 1 (and the output is unspecified).
 USAGE NOTES
 -----------
 
-git-merge-tree was written to be low-level plumbing, similar to
-hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
-Thus, it could be used as a part of a series of steps such as
+This command is intended as low-level plumbing, similar to
+linkgit:git-hash-object[1], linkgit:git-mktree[1],
+linkgit:git-commit-tree[1], linkgit:git-write-tree[1],
+linkgit:git-update-ref[1], and linkgit:git-mktag[1].  Thus, it can be
+used as a part of a series of steps such as:
 
        NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
        test $? -eq 0 || die "There were conflicts..."
        NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
        git update-ref $BRANCH1 $NEWCOMMIT
 
-Note that when the exit status is non-zero, NEWTREE in this sequence
+Note that when the exit status is non-zero, `NEWTREE` in this sequence
 will contain a lot more output than just a tree.
 
-git-merge-tree was written to provide users with the same information
-that they'd have access to if using `git merge`:
-  * what would be written to the working tree (the <OID of toplevel tree>)
+The output will include the same information that you'd get with
+linkgit:git-merge[1]:
+
+  * what would be written to the working tree (the <<OIDTLT,OID of toplevel tree>>)
   * the higher order stages that would be written to the index (the
-    <Conflicted file info>)
-  * any messages that would have been printed to stdout (the <Informational
-    messages>)
+    <<CFI,Conflicted file info>>)
+  * any messages that would have been printed to stdout (the <<IM,Informational
+    messages>>)
 
 MISTAKES TO AVOID
 -----------------
 
 Do NOT look through the resulting toplevel tree to try to find which
-files conflict; parse the <Conflicted file info> section instead.  Not
+files conflict; parse the <<CFI,Conflicted file info>> section instead.  Not
 only would parsing an entire tree be horrendously slow in large
 repositories, there are numerous types of conflicts not representable by
 conflict markers (modify/delete, mode conflict, binary file changed on
 both sides, file/directory conflicts, various rename conflict
 permutations, etc.)
 
-Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+Do NOT interpret an empty <<CFI,Conflicted file info>> list as a clean merge;
 check the exit status.  A merge can have conflicts without having
 individual files conflict (there are a few types of directory rename
 conflicts that fall into this category, and others might also be added
 in the future).
 
 Do NOT attempt to guess or make the user guess the conflict types from
-the <Conflicted file info> list.  The information there is insufficient
+the <<CFI,Conflicted file info>> list.  The information there is insufficient
 to do so.  For example: Rename/rename(1to2) conflicts (both sides
 renamed the same file differently) will result in three different file
 having higher order stages (but each only has one higher order stage),
-with no way (short of the <Informational messages> section) to determine
+with no way (short of the <<IM,Informational messages>> section) to determine
 which three files are related.  File/directory conflicts also result in
 a file with exactly one higher order stage.
 Possibly-involved-in-directory-rename conflicts (when
 "merge.directoryRenames" is unset or set to "conflicts") also result in
 a file with exactly one higher order stage.  In all cases, the
-<Informational messages> section has the necessary info, though it is
+<<IM,Informational messages>> section has the necessary info, though it is
 not designed to be machine parseable.
 
-Do NOT assume all filenames listed in the <Informational messages>
+Do NOT assume all filenames listed in the <<IM,Informational messages>>
 section had conflicts.  Messages can be included for files that have no
 conflicts, such as "Auto-merging <file>".
 
-AVOID taking the OIDS from the <Conflicted file info> and re-merging
+AVOID taking the OIDS from the <<CFI,Conflicted file info>> and re-merging
 them to present the conflicts to the user.  This will lose information.
-Instead, look up the version of the file found within the <OID of
-toplevel tree> and show that instead.  In particular, the latter will
+Instead, look up the version of the file found within the <<OIDTLT,OID of
+toplevel tree>> and show that instead.  In particular, the latter will
 have conflict markers annotated with the original branch/commit being
 merged and, if renames were involved, the original filename.  While you
 could include the original branch/commit in the conflict marker
 annotations when re-merging, the original filename is not available from
-the <Conflicted file info> and thus you would be losing information that
+the <<CFI,Conflicted file info>> and thus you would be losing information that
 might help the user resolve the conflict.
 
+[[DEPMERGE]]
+DEPRECATED DESCRIPTION
+----------------------
+
+Per the <<NEWMERGE,DESCRIPTION>> and unlike the rest of this
+documentation this section describes describes the deprecated
+`--trivial-merge` mode.
+
+Reads three tree-ish, and output trivial merge results and
+conflicting stages to the standard output.  This is similar to
+what three-way 'git read-tree -m' does, but instead of storing the
+results in the index, the command outputs the entries to the
+standard output.
+
+This is meant to be used by higher level scripts to compute
+merge results outside of the index, and stuff the results back into the
+index.  For this reason, the output from the command omits
+entries that match the <branch1> tree.
+
 GIT
 ---
 Part of the linkgit:git[1] suite

Re: [PATCH v2 04/13] merge-tree: implement real merges

[Johannes Schindelin]

Hi,

On Wed, 2 Feb 2022, Elijah Newren wrote:

> On Wed, Feb 2, 2022 at 1:30 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
> >
> > > +Performs a merge, but does not make any new commits and does not read
> > > +from or write to either the working tree or index.
> > > +
> > > +The second form is deprecated and supported only for backward
> > > +compatibility.  It will likely be removed in the future, and will not
> > > +be discussed further in this manual.
> >
> > This, especially the deletion of the original description on what
> > trivial merge does, may be premature, especially if it is still
> > "supported for backward compatibility".
>
> I actually extended it, but Dscho suggested removing it entirely --
> https://lore.kernel.org/git/nycvar.QRO.7.76.6.2201251804250.2121@tvgsbejvaqbjf.bet/.
> I can restore it; does that paragraph look good to you (you can see
> the full thing even if it's split by Dscho's commentary).

I do not necessarily feel strong about keeping the paragraph, but I would
appreciate a better rationale for that than "because it was there
previously".

At this point there cannot be any doubt that the "trivial mode" is worth
deprecating and getting rid off because it has been in little or no use.

I mean, elsewhere we are overzealously removing stuff "because _core Git_
does not use it", and here we keep something for... reasons?

> > > +test_expect_success 'Content merge and a few conflicts' '
> > > +     git checkout side1^0 &&
> > > +     test_must_fail git merge side2 &&
> > > +     expected_tree=$(cat .git/AUTO_MERGE) &&
> > > +
> > > +     # We will redo the merge, while we are still in a conflicted state!
> > > +     test_when_finished "git reset --hard" &&
> > > +
> > > +     test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
> > > +     actual_tree=$(head -n 1 RESULT) &&
> > > +
> > > +     # Due to differences of e.g. "HEAD" vs "side1", the results will not
> > > +     # exactly match.  Dig into individual files.
> > > +
> > > +     # Numbers should have three-way merged cleanly
> > > +     test_write_lines 0 1 2 3 4 5 6 >expect &&
> > > +     git show ${actual_tree}:numbers >actual &&
> > > +     test_cmp expect actual &&
> > > +
> > > +     # whatever and whatever~<branch> should have same HASHES
> > > +     git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
> > > +     git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
> > > +     test_cmp expect actual &&
> > > +
> > > +     # greeting should have a merge conflict
> > > +     git show ${expected_tree}:greeting >tmp &&
> > > +     cat tmp | sed -e s/HEAD/side1/ >expect &&
> > > +     git show ${actual_tree}:greeting >actual &&
> > > +     test_cmp expect actual
> > > +'
> >
> > It is somewhat sad that we need to reivent merge test cases over and
> > over, instead of easily reuse an existing one by replacing
> >
> >         git checkout one &&
> >         git merge two
> >
> > with
> >
> >         git checkout one &&
> >         T=$(git merge-tree HEAD two) &&
> >         C=$(git commit-tree $T -p HEAD -p two) &&
> >         git reset --hard $C
> >
> > ;-)
>
> Sorry...I'm afraid I'm not following.

I cannot speak for Junio (nor do I want to). Having said that, I interpret
the quoted section as a lament on the somewhat repetitive complexity that
is required by our tests merely to set up a scenario where we can run the
actual command we want to test.

If my interpretation is correct, then it is probably fair to suggest using
`test_merge` in a setup-type of test case (because that not only runs `git
merge` but also tags the result), and then use `git reset --hard <tag>` in
subsequent test cases instead of re-running the `checkout && merge` dance.

Ciao,
Dscho

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Johannes Schindelin]

Hi,

On Thu, 3 Feb 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 2:42 AM Johannes Altmanninger <aclopte@gmail.com> wrote:
> >
> > On Wed, Feb 02, 2022 at 04:18:39PM -0800, Elijah Newren wrote:
> > > On Wed, Feb 2, 2022 at 2:01 PM Junio C Hamano <gitster@pobox.com> wrote:
> > > >
> > > > Elijah Newren <newren@gmail.com> writes:
> > > >
> > > > > Yes, you are reading right.  I think the cherry-pick/rebase
> > > > > replacement actually deserves a separate command from what merges
> > > > > should use; replaying a sequence of commits just has a number of UI
> > > > > differences and abilities that I think pull it in a different
> > > > > direction.
> > > >
> > > > I completely disagree.  Each individual step in a sequence of
> > > > replaying commits in order (or in reverse order) should be
> > > > scriptable as a single merge-tree that takes "apply the change to go
> > > > from A^ to A on X".  Sequencing and placing UI around it is a job
> > > > for the script that drives merge-tree.
> > >
> > > Adding such an ability to merge-tree would be trivial -- it basically
> > > involves just two things: (1) accepting one extra argument, and (2)
> > > calling merge_incore_nonrecursive() instead of
> > > merge_incore_recursive().
> > >
> > > However, I think forking a subprocess for every merge of a series of
> > > commits is a completely unreasonable overhead, so even if we provide
> > > such an option to merge-tree, I still want a separate plumbing-ish
> > > tool that does non-worktree/non-index replaying of commits which is
> > > not written as a driver of merge-tree.  That other tool should just
> > > call merge_incore_nonrecursive() directly.  And such a tool, since it
> > > should handle an arbitrary number of commits, should certainly be able
> > > to handle just one commit.  From that angle, it feels like adding
> > > another mode to merge-tree would just be a partial duplication of the
> > > other tool.
> >
> > I wonder how the UI of a tool that does non-worktree/non-index cherry-picks
> > will look like.  I'd expect it to produce the same output as merge-tree,
> > except cherry-pick should probably output a commit OID, not a tree.
> >
> > Maybe we want a unified command that produces commits from any sequence of
> > merge/cherry-pick/revert/reword steps. The obvious UI would use something
> > like the rebase-todo list as input.  For example:
> >
> >         $ echo '
> >         pick commit1
> >         reword commit2  # edit commit message in $GIT_EDITOR
> >         merge commit3 -m "log message"
> >         ' | git create-commit commit0
> >         <OID of final commit>
> >
> > we start from commit0 and apply steps one-by-one. Obviously, one unsolved
> > problem is how to pass parameters like commit messages if no editor should
> > be invoked (my sketch uses -m).
> > If any of the steps fails when merging merge, then we get the tree with
> > conflicts
> >
> >         $ echo '
> >         pick commit1
> >         pick commit2
> >         pick commit-that-does-not-apply
> >         ' | git create-commit commit0
> >         <OID of commit after step 2>
> >         <OID of toplevel tree after failed merge>
> >         <Conflicted file info>
> >         <Informational messages>
> >
> > Replaying a series of commits might look like this:
> >
> >         $ echo 'pick commit1 ^commit0' | git create-commit new-base
> >
> > I'm concluding that this is a difficult UI problem
>
> I agree.  I've got a lot of thoughts on it, and some work in progress
> towards it (https://github.com/newren/git/tree/replay -- _very_ hacky,
> not even close to alpha quality, lots of fixup commits, todo comments,
> random brain dump files added to the tree, based on a previous round
> of this patch series, not updated for weeks, etc., etc.)

Just chiming in that I find that very exciting. But it's a tangent, and
slightly distracting from the topic at hand, so I would like to ask to
focus back on server-side merges.

> > and having a merge-tree command that accepts a "common ancestor"
> > parameter could make it easier to experiment.  Of course that depends
> > on who is experimenting.
>
> I think that would result in experiments and eventually full-blown
> scripts designed around forking subprocesses for every merge, and
> pushes us back into the world of having a scripted-rebase again.  Yes,
> I know people can transliterate shell back to C; it seems to always be
> done as a half-way measure with the forking just being done from C or
> have other UI-warts guided by the shell design.  In fact, *that* was
> the primary reason for me not providing a merge-tree option based on
> merge_incore_nonrecursive(), despite how trivial it'd be to provide
> it.  If someone wanted a merge_incore_nonrecursive() mode for
> merge-tree for reasons other than attempting to build a
> rebase/cherry-pick replacement based on it, then I'd be much happier
> to provide it.
>
> If someone wants to experiment with what a plumbing-ish
> rebase/cherry-pick would look like, the _right_ way to do it would be
> making using of merge_incore_nonrecursive() directly.  If they want
> example code, I already provided some a year and a half ago and got it
> merged into git.git in the form of t/helper/test-fast-rebase.c.  My
> "replay" branch is based on that code, but (a) moves it from t/helper
> to a real builtin, (b) removes the hardcoded very strict input, (c)
> removes the line of code doing the index & working tree updates, and
> (d) modifies the output to be a more plumbing-ish style.

I actually implemented that so I could provide apples-to-apples
speed comparisons between libgit2 and merge-ort:

-- snip --
From 6a865c691810b67dc15ddb57ad110bd6fdfc2f12 Mon Sep 17 00:00:00 2001
From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Fri, 28 Jan 2022 23:28:20 +0100
Subject: [PATCH] merge-tree: optionally force a simple 3-way merge

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/merge-tree.c | 72 ++++++++++++++++++++++++++++++--------------
 1 file changed, 50 insertions(+), 22 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 58c0ddc5a3..1007aaaede 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -396,6 +396,7 @@ struct merge_tree_options {
 	int allow_unrelated_histories;
 	int show_messages;
 	int exclude_modes_oids_stages;
+	const char *nonrecursive_base;
 };

 static int real_merge(struct merge_tree_options *o,
@@ -409,34 +410,58 @@ static int real_merge(struct merge_tree_options *o,
 	struct merge_options opt;
 	struct merge_result result = { 0 };

-	parent1 = get_merge_parent(branch1);
-	if (!parent1)
-		help_unknown_ref(branch1, "merge-tree",
-				 _("not something we can merge"));
-
-	parent2 = get_merge_parent(branch2);
-	if (!parent2)
-		help_unknown_ref(branch2, "merge-tree",
-				 _("not something we can merge"));
-
 	init_merge_options(&opt, the_repository);

 	opt.show_rename_progress = 0;

-	opt.branch1 = branch1;
-	opt.branch2 = branch2;
+	if (o->nonrecursive_base) {
+		struct object_id base_oid, head_oid, merge_oid;
+		struct tree *base_tree, *head_tree, *merge_tree;
+
+		opt.ancestor = "(base)";
+		opt.branch1 = "(branch1)";
+		opt.branch2 = "(branch2)";
+
+		if (get_oid_treeish(o->nonrecursive_base, &base_oid))
+			die("could not parse base '%s'", o->nonrecursive_base);
+		base_tree = parse_tree_indirect(&base_oid);
+		if (get_oid_treeish(branch1, &head_oid))
+			die("could not parse head '%s'", branch1);
+		head_tree = parse_tree_indirect(&head_oid);
+		if (get_oid_treeish(branch2, &merge_oid))
+			die("could not parse merge '%s'", branch2);
+		merge_tree = parse_tree_indirect(&merge_oid);
+
+		merge_incore_nonrecursive(&opt,
+					  base_tree, head_tree, merge_tree,
+					  &result);
+	} else {
+		parent1 = get_merge_parent(branch1);
+		if (!parent1)
+			help_unknown_ref(branch1, "merge-tree",
+					 _("not something we can merge"));
+
+		parent2 = get_merge_parent(branch2);
+		if (!parent2)
+			help_unknown_ref(branch2, "merge-tree",
+					 _("not something we can merge"));
+
+		opt.branch1 = branch1;
+		opt.branch2 = branch2;

-	/*
-	 * Get the merge bases, in reverse order; see comment above
-	 * merge_incore_recursive in merge-ort.h
-	 */
-	common = get_merge_bases(parent1, parent2);
-	if (!common && !o->allow_unrelated_histories)
-		die(_("refusing to merge unrelated histories"));
-	for (j = common; j; j = j->next)
-		commit_list_insert(j->item, &merge_bases);
+		/*
+		 * Get the merge bases, in reverse order; see comment above
+		 * merge_incore_recursive in merge-ort.h
+		 */
+		common = get_merge_bases(parent1, parent2);
+		if (!common && !o->allow_unrelated_histories)
+			die(_("refusing to merge unrelated histories"));
+		for (j = common; j; j = j->next)
+			commit_list_insert(j->item, &merge_bases);
+
+		merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	}

-	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));

@@ -501,6 +526,9 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.allow_unrelated_histories,
 			   N_("allow merging unrelated histories"),
 			   PARSE_OPT_NONEG),
+		OPT_STRING(0, "force-non-recursive-base", &o.nonrecursive_base,
+			   N_("base-tree"),
+			   N_("force a simple three-way merge")),
 		OPT_END()
 	};

-- snap --

I do strongly agree that this should _not_ enter core Git's code, I just
provide this in case someone else wants to play with merge-ort on the
server side in an existing code base.

> We'll certainly have discussions on what that should look like.  But a
> plumbing-ish replacement for merge was much simpler, and made sense to
> do first.  I would prefer to concentrate on getting that hammered down
> first.  Then I'll start discussions on a plumbing-ish
> rebase/cherry-pick.  And if that doesn't fulfill all the needs that
> folks think they want out of merge-tree, then we can add a
> merge_incore_nonrecursive()-based mode to merge-tree.  It's all
> coming, but having fought transliterations-of-scripts in
> merge-recursive.c, sequencer.c, stash.c, rebase.c, etc. for years I
> really, really don't want any more of that.  Let's end that insanity.

Being the driving force behind many a "built-in-ification" of scripted
commands, I wholeheartedly agree. You can still see the fall-out of
designing commands in a scripted fashion, without any way to represent
data structures other than strings. I wish we had come up with a better
design to prototype commands than to write shell scripts. But I have to
admit that even I do not have any better idea than to work on a proper API
for libgit.a (which has historically invariably seen push-back from
Junio).

While I agree that this discussion is a valuable one, right now I would
like to focus on getting the server-side merges done, and once that has
happened, move on to the replay/sequencer/API discussion (which will
probably be a big one, not so much for technical reasons but more for all
too human ones).

Ciao,
Dscho

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1039 bytes --]

Hi Elijah,

On Thu, 3 Feb 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 8:24 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >
> > On Thu, Feb 03 2022, Elijah Newren wrote:
> >
> > > Man, what a can of worms this all is.  Maybe I really should just drop
> > > patches 5, 6, and 8 for now...
> >
> > Yeah, I really think it's worth it to just sprinkle a tiny bit of
> > if/else (or a macro) here and print to stderr inline or not. We can make
> > some use of some usage.c when there's good reason to do so, but this bit
> > just seems like a needless digression.
> >
> > I hope all of this has helped somewhat ...
>
> Absolutely; thanks for reviewing!  These parts may just end up in me
> dropping some patches for now (since they're not actually being used
> anyway), but I think it's all good feedback.

So we dropped some useful patches future-proofing `merge-tree` for the
sake of appeasing a refactoring with no immediately obvious benefit? I
really don't like that direction.

Ciao,
Dscho

Re: [PATCH v5 00/12] In-core git merge-tree ("Server side merges")

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 852 bytes --]

Hi,

On Sun, 20 Feb 2022, Ævar Arnfjörð Bjarmason wrote:

> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 306149fa0e2..723b1995426 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -9,17 +9,24 @@ git-merge-tree - Perform merge without touching index or working tree
>  SYNOPSIS
>  --------
>  [verse]
> -'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
> -'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
> +'git merge-tree' --write-tree [<options>] <branch1> <branch2>
> +'git merge-tree' --trivial-merge <base-tree> <branch1> <branch2>

Given that we want to get away from `--trivial-merge` (and probably even
deprecating and then dropping it), this direction makes no sense.

Ciao,
Johannes

Re: [PATCH v5 04/12] merge-tree: implement real merges

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1246 bytes --]

Hi,

On Sun, 20 Feb 2022, René Scharfe wrote:

> Am 20.02.22 um 07:54 schrieb Elijah Newren via GitGitGadget:
>
> > +	/*
> > +	 * Get the merge bases, in reverse order; see comment above
> > +	 * merge_incore_recursive in merge-ort.h
> > +	 */
> > +	common = get_merge_bases(parent1, parent2);
> > +	if (!common)
> > +		die(_("refusing to merge unrelated histories"));
> > +	for (j = common; j; j = j->next)
> > +		commit_list_insert(j->item, &merge_bases);
>
> This loop creates a reversed copy of "common".  You could use
> reverse_commit_list() instead to do it in-place and avoid the
> allocations.  Only the copy, "merge_bases", is used below.

Curious. When I read this first, I immediately assumed this was
copy-pasted from `merge-recursive.c`, but it wasn't
(https://github.com/git/git/blob/v2.35.1/merge-recursive.c#L3591-L3592):

		merge_bases = get_merge_bases(h1, h2);
		merge_bases = reverse_commit_list(merge_bases);

I tried to figure out where the manual reversal might have been
copy-pasted from, but came up empty-handed. The comment in
https://github.com/git/git/blob/v2.35.1/merge-ort.h#L40-L48 did not shed
any light on it (but took me down memory lane, all the way to 2006!).

Ciao,
Dscho

Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts

[Johannes Schindelin]

Hi Elijah,

[reinstating the Cc: list]

On Sat, 12 Feb 2022, Elijah Newren wrote:

> On Fri, Feb 4, 2022 at 3:12 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Fri, 28 Jan 2022, Elijah Newren wrote:
> >
> > > On Fri, Jan 28, 2022 at 8:57 AM Johannes Schindelin
> > > <Johannes.Schindelin@gmx.de> wrote:
> > > >
> > > > Hi Elijah,
> > > >
> > > > On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
> > > >
> > > > > From: Elijah Newren <newren@gmail.com>
> > > > >
> > > [...]
> > > > > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > > > > index fd7a867de60..041a4ac2785 100644
> > > > > --- a/Documentation/git-merge-tree.txt
> > > > > +++ b/Documentation/git-merge-tree.txt
> > > > > @@ -58,6 +58,7 @@ simply one line:
> > > > >  Whereas for a conflicted merge, the output is by default of the form:
> > > > >
> > > > >       <OID of toplevel tree>
> > > > > +     <Conflicted file list>
> > > > >       <Informational messages>
> > > >
> > > > To distinguish between the list of conflicted files and the informational
> > > > messages, I think it would be good to insert an empty line, as a
> > > > separator, like.
> > >
> > > Yes, I agree; that's why I did so.  :-)
> >
> > My concern was that I did not see this empty line reflected in the quoted
> > diff. I would have expected an empty line between the `<Conflicted [...]>`
> > and the `<Informational [...]>` line.
>
> As stated later in the same email, the newline is only printed if the
> <Informational messages> section is printed.  As such, it's part of
> the <Informational messages> section and listing it between the
> sections could be misleading.

Thank you for the clarification. I still think that it might cause
confusion in the documentation not to see the explicit newline, quite
possible when _I_ re-read this in six months from now. But then, if `-z`
is in effect, it won't be an explicit newline, it will be a NUL instead.

In short: I am fine with leaving this as-is, it might actually reduce even
_my_ confusion to the minimum possible.

Thanks,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Fri, 4 Feb 2022, Elijah Newren wrote:

> On Fri, Feb 4, 2022 at 3:10 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Sat, 29 Jan 2022, Elijah Newren wrote:
> >
> > > On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
> > > >
> > > > Just a heckling from the peanut gallery...
> > > >
> > > > Am 29.01.22 um 07:08 schrieb Elijah Newren:
> > > > > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> > > > > <Johannes.Schindelin@gmx.de> wrote:
> > > > >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> > > > >> missing from the second conflict, in the output we would see stages 1, 2,
> > > > >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> > > > >> different conflicts.
> > > > >
> > > > > I don't understand why you're fixating on the stage here.  Why would
> > > > > you want to group all the stage 2s together, count them up, and then
> > > > > determine there are N conflicting files because there are N stage 2's?
> > > >
> > > > Looks like you are misunderstanding Dscho's point: When you have two
> > > > conflicts, the first with stages 1 and 2, the second with stages 2 and
> > > > 3, then the 2s occur lumped together when the 4 lines are printed in a
> > > > row, and that is the cue to the parser where the new conflict begins.
> > > > Dscho did not mean that all N 2s of should be listed together.
> > >
> > > Ah, so...I didn't understand his misunderstanding?  Using stages as a
> > > cue to the parser where the new conflict begins is broken; you should
> > > instead check for when the filename listed on a line does not match
> > > the filename on the previous line.
> >
> > But that would break down in case of rename/rename conflicts, right?
> >
> > > In particular, if one conflict has stages 1 and 2, and the next conflict
> > > has only stage 3, then looking at stages only might cause you to
> > > accidentally lump unrelated conflicts together.
> >
> > Precisely. That's why I would love to have a way to deviate from the
> > output of `ls-files -u`'s format, and have a reliable way to indicate
> > stages that belong to the same merge conflict.
>
> Ah, attempting to somehow identify and present logical separate
> conflicts?  That could be awesome, but I'm not sure it's technically
> possible.  It certainly isn't with today's merge-ort.
>
> Let me ask some questions first...
>
> If I understand you correctly then in the event of a rename/rename,
> i.e. foo->bar & foo->baz, then you want foo's, bar's, & baz's stages
> all listed together.  Right?  And in some way that you can identify
> them as related?

Yes, that was kind of my idea ;-)

> If we do so, how do we mark the beginning and the end of what you call
> "the same merge conflict"?  If you say it's always 3 stages (with the
> possibility of all-zero modes/oids), then what about the rename/rename
> case above modified so that the side that did foo->baz also added a
> different 'bar'?  That'd be 4 non-zero modes/oids, all of them
> relevant.  Or what if the side that did foo->bar also renamed
> something else to 'baz', giving us even more non-zero stages for these
> three paths?  Perhaps you consider these different conflicts and want
> them listed separately -- if so, where does one conflict begin and
> another start and which stages are parts of which conflict?

Thank you for calling me out on an only half-finished thought.

To be quite honest, I previously did not have much time to think of the
non-trivial nature of representing merge conflicts in a web UI when
performing merges with rename detection, in particular when performing
_recursive_ merges (where, as you point out so correctly, an arbitrary
number of rename conflicts can happen _for the same file_).

Of course, this gets even more complicated when we're also thinking about
type changes (file -> symlink, file -> directory, and vice versa).

> If you are attempting to somehow present the stuff that "belongs to
> the same merge conflict" are you also trying to identify what kind of
> merge conflict it is?  If so, do you want each type of merge conflict
> listed?  For example, let's switch from the example above of logically
> disjoint paths coming together to result in more than 3 stages, and
> instead pick an example with a single logical path with less than
> three stages.  And now let's say that path has multiple conflicts
> associated with it; let's use an example with 3: rename/delete +
> modify/delete + directory/file (one side renames foo->bar while
> modifying the contents, the other side deletes foo and adds the
> directory 'bar/').  In this case, there is a target file 'bar' that
> has two non-zero modes/oids in the ls-files-u output.  If all three
> types of conflicts need to be listed, does each need to be listed with
> the two non-zero modes/oids (and perhaps one zero mode/oid), resulting
> in six listings for 'bar'?  Or would the duplication be confusing
> enough that we instead decide to list some merge conflicts with no
> stages associated with them?
>
> Thinking about both sets of questions in the last two paragraphs from
> a higher level -- should we focus on and group the higher order stages
> by the individual conflicts that happen, or should we group them by
> the paths that they happen to (which is what `ls-files -u` happens to
> do), or should we not bother grouping them and instead duplicate the
> higher order stages for each logical conflict it is part of?
>
> As an alternative to duplicating higher order stages, do we sometimes
> decide to "lump" separate conflicts together and treat them as one
> conflict?  If so, what are the rules on how we decide to lump
> conflicts and when not to?  Is there a bright line boundary?  And can
> it be done without sucking in arbitrarily more stages for a single
> conflict?
>
>
> Some testcases that might be useful while considering the above
> questions: take a look at the "rad", "rrdd", and "mod6" tests of
> t6422.  How many "same merge conflicts" are there for each of those,
> and what's the boundary between them?  And can you give the answer in
> the form of rules that generically handle all cases, rather than just
> answering these three specific cases?
>
>
> I've thought about this problem long and hard before (in part because
> of some conversations I had with Edward Thompson about libgit2 and

Not a big deal for _me_, but I seem to remember that Ed cared a lot about
having no p in their surname ;-)

> merging at Git Merge 2020).  It wasn't at all clear to me that libgit2
> had considered anything beyond simple rename cases.  The only rules I
> ever figured out that made sense to me was "group the stages by target
> filename rather than by logical conflict" (so we get `ls -files -u`
> populated) and print a meant-for-human message for each logical
> conflict (found in the <Informational Messages> section for
> merge-tree), and make NO attempt to connect stages by conflict type.
>
> I'm sure that's not what you wanted to hear, and maybe doesn't even
> play nicely with your design.  But short of ignoring the edge and
> corner cases, I don't see how to solve that problem.  If you do just
> want to ignore edge and corner cases, then just ignore the
> rename/rename case you brought up in the first place and just use
> `ls-files -u`-type output as-is within your design.  If you don't want
> to ignore edge cases and want something that works with a specific
> design that somehow groups conflicted file stages by conflict type,
> then we're going to have to dig into all these questions above and do
> some big replumbing within merge-ort.

There is sometimes a big difference between what I want to hear and what I
need to hear. Thank you for providing so many details that I needed to
hear.

So let's take a step back and look at my goal here, as in: the
over-arching goal: to use merge-ort on the server side.

From what you said above, it becomes very clear to me that there is very
little chance to resolve such conflicts on the server side.

For example, if a topic branch renames a file differently than the main
branch, there is a really good chance that the user tasked with merging
the topic branch will have to do a whole lot more than just click a few
buttons to perform that task. There might very well be the need to edit
files that do not contain merge conflict markers (I like to call those
cases "non-semantic merge conflicts"), and almost certainly local testing
will be necessary.

So I guess the best we can do in those complicated cases is to give a
comprehensive overview of the problems in the web UI, with the note that
this merge conflict has to be resolved on the local side.

Which brings me to the next concern: since `merge-tree` is a low-level
tool meant to be called by programs rather than humans, we need to make
sure that those messages remain machine-parseable, even if they contain
file names.

Concretely: while I am not currently aware of any web UI that allows to
resolve simple rename/rename conflicts, it is easily conceivable how to
implement such a thing. When that happens, we will need to be able to
teach the server-side code to discern between the cases that can be
handled in the web UI (trivial merge conflicts, trivial rename/rename
conflicts) as compared to scenarios where the conflicts are just too
complex.

Here's an excerpt from t4301:

-- snip --
Auto-merging greeting
CONFLICT (content): Merge conflict in greeting
Auto-merging numbers
CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
-- snap --

This is the complete set of messages provided in the `test conflict
notices and such` test case.

I immediately notice that every line contains at least one file name.
Looking at https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1899, it
does not seem as if the file names are quoted:

		path_msg(opt, path, 1, _("Auto-merging %s"), path);

(where `path` is used verbatim in a call to `merge_3way()` before that,
i.e. it must not have been quoted)

I would like to register a wish to ensure that file names with special
characters (such as most notably line-feed characters) are quoted in these
messages, so that a simple server-side parser can handle messages starting
with `Auto-merging` and with `CONFLICT (content): Merge conflict in `, and
"throw the hands up in the air" if any other message prefix is seen.

Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
For the `Auto-merging` one, it would be trivial, but I fear that we will
have to work a bit on the `path_msg()` function
(https://github.com/git/git/blob/v2.35.1/merge-ort.c#L630-L649) because it
accepts a variable list of arguments without any clue whether the
arguments refer to paths or not. (And I would be loathe to switch _all_
callers to do the quoting themselves.)

I see 28 calls to that function, and at least a couple that pass not only
a path but also an OID (e.g.
https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1611-L1613).

We could of course be sloppy and pass even OIDs through
`sq_quote_buf_pretty()` in `path_msg()`, knowing that there won't be any
special characters in them, but it gets more complicated e.g. in
https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1648-L1651, where we
pass an `strbuf` that contains a somewhat free-form commit message.

I guess we could still pass those through `sq_quote_buf_pretty()`, even if
they are not paths, to ensure that there are no special characters in the
machine-parseable lines.

What do you think?

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Ævar Arnfjörð Bjarmason]

On Mon, Feb 21 2022, Johannes Schindelin wrote:

> Hi Elijah,
>
> On Fri, 4 Feb 2022, Elijah Newren wrote:
>
>> On Fri, Feb 4, 2022 at 3:10 PM Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> >
>> > On Sat, 29 Jan 2022, Elijah Newren wrote:
>> >
>> > > On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
>> > > >
>> > > > Just a heckling from the peanut gallery...
>> > > >
>> > > > Am 29.01.22 um 07:08 schrieb Elijah Newren:
>> > > > > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
>> > > > > <Johannes.Schindelin@gmx.de> wrote:
>> > > > >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
>> > > > >> missing from the second conflict, in the output we would see stages 1, 2,
>> > > > >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
>> > > > >> different conflicts.
>> > > > >
>> > > > > I don't understand why you're fixating on the stage here.  Why would
>> > > > > you want to group all the stage 2s together, count them up, and then
>> > > > > determine there are N conflicting files because there are N stage 2's?
>> > > >
>> > > > Looks like you are misunderstanding Dscho's point: When you have two
>> > > > conflicts, the first with stages 1 and 2, the second with stages 2 and
>> > > > 3, then the 2s occur lumped together when the 4 lines are printed in a
>> > > > row, and that is the cue to the parser where the new conflict begins.
>> > > > Dscho did not mean that all N 2s of should be listed together.
>> > >
>> > > Ah, so...I didn't understand his misunderstanding?  Using stages as a
>> > > cue to the parser where the new conflict begins is broken; you should
>> > > instead check for when the filename listed on a line does not match
>> > > the filename on the previous line.
>> >
>> > But that would break down in case of rename/rename conflicts, right?
>> >
>> > > In particular, if one conflict has stages 1 and 2, and the next conflict
>> > > has only stage 3, then looking at stages only might cause you to
>> > > accidentally lump unrelated conflicts together.
>> >
>> > Precisely. That's why I would love to have a way to deviate from the
>> > output of `ls-files -u`'s format, and have a reliable way to indicate
>> > stages that belong to the same merge conflict.
>>
>> Ah, attempting to somehow identify and present logical separate
>> conflicts?  That could be awesome, but I'm not sure it's technically
>> possible.  It certainly isn't with today's merge-ort.
>>
>> Let me ask some questions first...
>>
>> If I understand you correctly then in the event of a rename/rename,
>> i.e. foo->bar & foo->baz, then you want foo's, bar's, & baz's stages
>> all listed together.  Right?  And in some way that you can identify
>> them as related?
>
> Yes, that was kind of my idea ;-)
>
>> If we do so, how do we mark the beginning and the end of what you call
>> "the same merge conflict"?  If you say it's always 3 stages (with the
>> possibility of all-zero modes/oids), then what about the rename/rename
>> case above modified so that the side that did foo->baz also added a
>> different 'bar'?  That'd be 4 non-zero modes/oids, all of them
>> relevant.  Or what if the side that did foo->bar also renamed
>> something else to 'baz', giving us even more non-zero stages for these
>> three paths?  Perhaps you consider these different conflicts and want
>> them listed separately -- if so, where does one conflict begin and
>> another start and which stages are parts of which conflict?
>
> Thank you for calling me out on an only half-finished thought.
>
> To be quite honest, I previously did not have much time to think of the
> non-trivial nature of representing merge conflicts in a web UI when
> performing merges with rename detection, in particular when performing
> _recursive_ merges (where, as you point out so correctly, an arbitrary
> number of rename conflicts can happen _for the same file_).
>
> Of course, this gets even more complicated when we're also thinking about
> type changes (file -> symlink, file -> directory, and vice versa).
>
>> If you are attempting to somehow present the stuff that "belongs to
>> the same merge conflict" are you also trying to identify what kind of
>> merge conflict it is?  If so, do you want each type of merge conflict
>> listed?  For example, let's switch from the example above of logically
>> disjoint paths coming together to result in more than 3 stages, and
>> instead pick an example with a single logical path with less than
>> three stages.  And now let's say that path has multiple conflicts
>> associated with it; let's use an example with 3: rename/delete +
>> modify/delete + directory/file (one side renames foo->bar while
>> modifying the contents, the other side deletes foo and adds the
>> directory 'bar/').  In this case, there is a target file 'bar' that
>> has two non-zero modes/oids in the ls-files-u output.  If all three
>> types of conflicts need to be listed, does each need to be listed with
>> the two non-zero modes/oids (and perhaps one zero mode/oid), resulting
>> in six listings for 'bar'?  Or would the duplication be confusing
>> enough that we instead decide to list some merge conflicts with no
>> stages associated with them?
>>
>> Thinking about both sets of questions in the last two paragraphs from
>> a higher level -- should we focus on and group the higher order stages
>> by the individual conflicts that happen, or should we group them by
>> the paths that they happen to (which is what `ls-files -u` happens to
>> do), or should we not bother grouping them and instead duplicate the
>> higher order stages for each logical conflict it is part of?
>>
>> As an alternative to duplicating higher order stages, do we sometimes
>> decide to "lump" separate conflicts together and treat them as one
>> conflict?  If so, what are the rules on how we decide to lump
>> conflicts and when not to?  Is there a bright line boundary?  And can
>> it be done without sucking in arbitrarily more stages for a single
>> conflict?
>>
>>
>> Some testcases that might be useful while considering the above
>> questions: take a look at the "rad", "rrdd", and "mod6" tests of
>> t6422.  How many "same merge conflicts" are there for each of those,
>> and what's the boundary between them?  And can you give the answer in
>> the form of rules that generically handle all cases, rather than just
>> answering these three specific cases?
>>
>>
>> I've thought about this problem long and hard before (in part because
>> of some conversations I had with Edward Thompson about libgit2 and
>
> Not a big deal for _me_, but I seem to remember that Ed cared a lot about
> having no p in their surname ;-)
>
>> merging at Git Merge 2020).  It wasn't at all clear to me that libgit2
>> had considered anything beyond simple rename cases.  The only rules I
>> ever figured out that made sense to me was "group the stages by target
>> filename rather than by logical conflict" (so we get `ls -files -u`
>> populated) and print a meant-for-human message for each logical
>> conflict (found in the <Informational Messages> section for
>> merge-tree), and make NO attempt to connect stages by conflict type.
>>
>> I'm sure that's not what you wanted to hear, and maybe doesn't even
>> play nicely with your design.  But short of ignoring the edge and
>> corner cases, I don't see how to solve that problem.  If you do just
>> want to ignore edge and corner cases, then just ignore the
>> rename/rename case you brought up in the first place and just use
>> `ls-files -u`-type output as-is within your design.  If you don't want
>> to ignore edge cases and want something that works with a specific
>> design that somehow groups conflicted file stages by conflict type,
>> then we're going to have to dig into all these questions above and do
>> some big replumbing within merge-ort.
>
> There is sometimes a big difference between what I want to hear and what I
> need to hear. Thank you for providing so many details that I needed to
> hear.
>
> So let's take a step back and look at my goal here, as in: the
> over-arching goal: to use merge-ort on the server side.
>
> From what you said above, it becomes very clear to me that there is very
> little chance to resolve such conflicts on the server side.
>
> For example, if a topic branch renames a file differently than the main
> branch, there is a really good chance that the user tasked with merging
> the topic branch will have to do a whole lot more than just click a few
> buttons to perform that task. There might very well be the need to edit
> files that do not contain merge conflict markers (I like to call those
> cases "non-semantic merge conflicts"), and almost certainly local testing
> will be necessary.
>
> So I guess the best we can do in those complicated cases is to give a
> comprehensive overview of the problems in the web UI, with the note that
> this merge conflict has to be resolved on the local side.
>
> Which brings me to the next concern: since `merge-tree` is a low-level
> tool meant to be called by programs rather than humans, we need to make
> sure that those messages remain machine-parseable, even if they contain
> file names.
>
> Concretely: while I am not currently aware of any web UI that allows to
> resolve simple rename/rename conflicts, it is easily conceivable how to
> implement such a thing. When that happens, we will need to be able to
> teach the server-side code to discern between the cases that can be
> handled in the web UI (trivial merge conflicts, trivial rename/rename
> conflicts) as compared to scenarios where the conflicts are just too
> complex.
>
> Here's an excerpt from t4301:
>
> -- snip --
> Auto-merging greeting
> CONFLICT (content): Merge conflict in greeting
> Auto-merging numbers
> CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
> CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
> -- snap --
>
> This is the complete set of messages provided in the `test conflict
> notices and such` test case.
>
> I immediately notice that every line contains at least one file name.
> Looking at https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1899, it
> does not seem as if the file names are quoted:
>
> 		path_msg(opt, path, 1, _("Auto-merging %s"), path);
>
> (where `path` is used verbatim in a call to `merge_3way()` before that,
> i.e. it must not have been quoted)
>
> I would like to register a wish to ensure that file names with special
> characters (such as most notably line-feed characters) are quoted in these
> messages, so that a simple server-side parser can handle messages starting
> with `Auto-merging` and with `CONFLICT (content): Merge conflict in `, and
> "throw the hands up in the air" if any other message prefix is seen.
>
> Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
> For the `Auto-merging` one, it would be trivial, but I fear that we will
> have to work a bit on the `path_msg()` function
> (https://github.com/git/git/blob/v2.35.1/merge-ort.c#L630-L649) because it
> accepts a variable list of arguments without any clue whether the
> arguments refer to paths or not. (And I would be loathe to switch _all_
> callers to do the quoting themselves.)
>
> I see 28 calls to that function, and at least a couple that pass not only
> a path but also an OID (e.g.
> https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1611-L1613).
>
> We could of course be sloppy and pass even OIDs through
> `sq_quote_buf_pretty()` in `path_msg()`, knowing that there won't be any
> special characters in them, but it gets more complicated e.g. in
> https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1648-L1651, where we
> pass an `strbuf` that contains a somewhat free-form commit message.
>
> I guess we could still pass those through `sq_quote_buf_pretty()`, even if
> they are not paths, to ensure that there are no special characters in the
> machine-parseable lines.
>
> What do you think?
>
> Ciao,
> Dscho

machine-parsable git-merge-tree messages (was: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function)

[Ævar Arnfjörð Bjarmason]

On Mon, Feb 21 2022, Johannes Schindelin wrote:

[I sent out an empty reply to this earlier by mistake, sorry about that]

> [...]
> Which brings me to the next concern: since `merge-tree` is a low-level
> tool meant to be called by programs rather than humans, we need to make
> sure that those messages remain machine-parseable, even if they contain
> file names.
>
> Concretely: while I am not currently aware of any web UI that allows to
> resolve simple rename/rename conflicts, it is easily conceivable how to
> implement such a thing. When that happens, we will need to be able to
> teach the server-side code to discern between the cases that can be
> handled in the web UI (trivial merge conflicts, trivial rename/rename
> conflicts) as compared to scenarios where the conflicts are just too
> complex.
>
> Here's an excerpt from t4301:
>
> -- snip --
> Auto-merging greeting
> CONFLICT (content): Merge conflict in greeting
> Auto-merging numbers
> CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
> CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
> -- snap --
>
> This is the complete set of messages provided in the `test conflict
> notices and such` test case.
>
> I immediately notice that every line contains at least one file name.
> Looking at https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1899, it
> does not seem as if the file names are quoted:
>
> 		path_msg(opt, path, 1, _("Auto-merging %s"), path);
>
> (where `path` is used verbatim in a call to `merge_3way()` before that,
> i.e. it must not have been quoted)
>
> I would like to register a wish to ensure that file names with special
> characters (such as most notably line-feed characters) are quoted in these
> messages, so that a simple server-side parser can handle messages starting
> with `Auto-merging` and with `CONFLICT (content): Merge conflict in `, and
> "throw the hands up in the air" if any other message prefix is seen.
>
> Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
> For the `Auto-merging` one, it would be trivial, but I fear that we will
> have to work a bit on the `path_msg()` function
> (https://github.com/git/git/blob/v2.35.1/merge-ort.c#L630-L649) because it
> accepts a variable list of arguments without any clue whether the
> arguments refer to paths or not. (And I would be loathe to switch _all_
> callers to do the quoting themselves.)
>
> I see 28 calls to that function, and at least a couple that pass not only
> a path but also an OID (e.g.
> https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1611-L1613).
>
> We could of course be sloppy and pass even OIDs through
> `sq_quote_buf_pretty()` in `path_msg()`, knowing that there won't be any
> special characters in them, but it gets more complicated e.g. in
> https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1648-L1651, where we
> pass an `strbuf` that contains a somewhat free-form commit message.
>
> I guess we could still pass those through `sq_quote_buf_pretty()`, even if
> they are not paths, to ensure that there are no special characters in the
> machine-parseable lines.
>
> What do you think?

That sounds like a rather nasty hack, this is too, but demonstrates that
we can pretty easily extract this in a machine-readable format with just
a few lines now):

diff --git a/merge-ort.c b/merge-ort.c
index 8a5f201d190..a906881f9b3 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -633,7 +633,7 @@ static void path_msg(struct merge_options *opt,
 		     int omittable_hint, /* skippable under --remerge-diff */
 		     const char *fmt, ...)
 {
-	va_list ap;
+	va_list ap, cp;
 	struct strbuf *sb, *dest;
 	struct strbuf tmp = STRBUF_INIT;
 
@@ -650,7 +650,9 @@ static void path_msg(struct merge_options *opt,
 
 	dest = (opt->record_conflict_msgs_as_headers ? &tmp : sb);
 
+	va_copy(cp, ap);
 	va_start(ap, fmt);
+
 	if (opt->priv->call_depth) {
 		strbuf_addchars(dest, ' ', 2);
 		strbuf_addstr(dest, "From inner merge:");
@@ -659,6 +661,15 @@ static void path_msg(struct merge_options *opt,
 	strbuf_vaddf(dest, fmt, ap);
 	va_end(ap);
 
+	va_start(cp, fmt);
+	trace2_region_enter_printf("merge", "conflict/path", opt->repo, "%s", path);
+	trace2_region_leave("merge", "conflict/path", opt->repo);
+	trace2_region_enter_printf("merge", "conflict/fmt", opt->repo, "%s", fmt);
+	trace2_region_leave("merge", "conflict/fmt", opt->repo);
+	trace2_region_enter_printf_va("merge", "conflict/msg", opt->repo, fmt, cp);
+	trace2_region_leave("merge", "conflict/msg", opt->repo);
+	va_end(cp);
+
 	if (opt->record_conflict_msgs_as_headers) {
 		int i_sb = 0, i_tmp = 0;
 
You can run that with one of the tests added in this series to get the
output as JSON, e.g.:

     GIT_TRACE2_EVENT=/dev/stderr GIT_TRACE2_EVENT_NESTING=10 ~/g/git/git merge-tree --write-tree --no-messages --name-only --messages side1 side2 2>&1|jq -r .| grep '"msg"'
      "msg": "whatever~side1"
      "msg": "CONFLICT (file/directory): directory in the way of %s from %s; moving it to %s instead."
      "msg": "CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead."
      "msg": "whatever~side1"
      "msg": "CONFLICT (modify/delete): %s deleted in %s and modified in %s.  Version %s of %s left in tree."
      "msg": "CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree."
      "msg": "numbers"
      "msg": "Auto-merging %s"
      "msg": "Auto-merging numbers"
      "msg": "greeting"
      "msg": "Auto-merging %s"
      "msg": "Auto-merging greeting"
      "msg": "greeting"
      "msg": "CONFLICT (%s): Merge conflict in %s"
      "msg": "CONFLICT (content): Merge conflict in greeting"

A "proper" fix for this doesn't sound too hard, we'd just instrument the
path_msg() function to pass along some "message category", see
e.g. unpack_plumbing_errors in unpack-trees.c for one example of such a
thing, or the "enum fsck_msg_id".

Then we'd just allow you to emit any of the sprintf() format itself, or
the expanded version, the path, or an id like "CONFLICT:file/directory"
or "auto-merging" etc.

I think that would be particularly useful in conjuction with the
--format changes I was proposing for this (and hacked up a WIP patch
for). You could just have a similar --format-messages or whatever.

Then you could pick \0\0 as a delimiter for your "main" --format, and
"\0" as the delimiter for your --format-messages, and thus be able to
parse N-nested \0-delimited content.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Adding such an ability to merge-tree would be trivial -- it basically
> involves just two things: (1) accepting one extra argument, and (2)
> calling merge_incore_nonrecursive() instead of
> merge_incore_recursive().
>
> However, I think forking a subprocess for every merge of a series of
> commits is a completely unreasonable overhead, so even if we provide
> such an option to merge-tree, I still want a separate plumbing-ish
> tool that does non-worktree/non-index replaying of commits which is
> not written as a driver of merge-tree.  That other tool should just
> call merge_incore_nonrecursive() directly.  And such a tool, since it
> should handle an arbitrary number of commits, should certainly be able
> to handle just one commit.  From that angle, it feels like adding
> another mode to merge-tree would just be a partial duplication of the
> other tool.

The above does not make much sense to me.

I am hearing that "multi-step cherry-picks and reverts need to be
fast and we need something like sequencer that is all written in C,
and single-step cherry-pick is merely a special case that does not
deserve a plumbing".

But that argument leads to "and the same something-like-sequencer
that is all written in C would need '--rebase-merges' that can pick
multi-step merge sequences, and single-step merge does not deserve a
plumbing", which is an argument against this topic that is utterly
absurd.

So why isn't your objection not equally absurd against having a
single step cherry-pick or revert primitive as a plumbing?

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Elijah Newren]

On Mon, Feb 21, 2022 at 1:13 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Thu, 3 Feb 2022, Elijah Newren wrote:
>
> > On Thu, Feb 3, 2022 at 8:24 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> > >
> > > On Thu, Feb 03 2022, Elijah Newren wrote:
> > >
> > > > Man, what a can of worms this all is.  Maybe I really should just drop
> > > > patches 5, 6, and 8 for now...
> > >
> > > Yeah, I really think it's worth it to just sprinkle a tiny bit of
> > > if/else (or a macro) here and print to stderr inline or not. We can make
> > > some use of some usage.c when there's good reason to do so, but this bit
> > > just seems like a needless digression.
> > >
> > > I hope all of this has helped somewhat ...
> >
> > Absolutely; thanks for reviewing!  These parts may just end up in me
> > dropping some patches for now (since they're not actually being used
> > anyway), but I think it's all good feedback.
>
> So we dropped some useful patches future-proofing `merge-tree` for the
> sake of appeasing a refactoring with no immediately obvious benefit? I
> really don't like that direction.

Even before any of Ævar's comments, I had already noted on my cover
letter[1] that "to be honest, patches 5, 6, & 8 may be less relevant
since we're now including these messages on stdout anyway" -- so I was
already wondering if I should defer them to some future series.  Then
when Ævar reviewed the series, he noted (1) that I lacked tests of
these changes (which is true, because nothing uses them, and I can't
easily add a test as I have no current usecase in mind), and (2) these
patches would print a "warning: " prefix when printing to stdout but
not print such a prefix otherwise, which felt inconsistent.  Those
seemed to reinforce the comment I had already made that these changes
were unused in my series and maybe should be separated.  I still like
the general idea behind the future proofing you did here, so maybe I
was just being lazy, but between those factors I decided that punting
until later made sense.

[1] https://lore.kernel.org/git/pull.1122.v3.git.1643787281.gitgitgadget@gmail.com/

Re: [PATCH v5 00/12] In-core git merge-tree ("Server side merges")

[Elijah Newren]

On Sun, Feb 20, 2022 at 4:35 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sun, Feb 20 2022, Elijah Newren via GitGitGadget wrote:
>
> > == Updates Log ==
> >
> > Many thanks to the many reviewers who provided good feedback on the most
> > recent round -- Junio, Ævar, Josh, Emily, and perhaps some others I've
> > forgotten from review club.
> >
> > Updates since v4:
> >
> >  * Fixed double "is" in documentation.
> >  * Fixed a few small items with testcases
> >
> > Updates since v3 (or v5, if you include the rounds at
> > https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):
> >
> >  * Dropped previous patches 5, 6, and 8 of the old series; they weren't
> >    being used and opened a can of worms[1]
> >  * [Patch 3] Restructured argument checking, including using an enum
> >  * [Patch 4] Restored the extended paragraph about the deprecated form of
> >    git-merge-tree, mentioned write-tree in plumbing commands, and a few
> >    other small fixups to the documentation
> >  * [Patch 4] Also provide an example of a clean merge rather than just a
> >    conflicted one
> >  * [Patch 6] Fix the incompatible arguments check and add some tests for it
> >  * [Patch 6] Introduce an anonymize_hash() shell function to make tests
> >    easier to read (less repeated sed)
> >  * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
> >    option for now
> >  * [Patch 10] When -z passed, the tree in the first section should have a
> >    trailing NUL rather than trailing newline [1]
> >    https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/
> >
> > Stuff NOT included that reviewers brought up in earlier rounds:
> >
> >  * Very generic (mode, oid, stage, filename) printing formatting[2]
> >  * Always printing 3 stages for each filename with conflicts[3]
> >  * Attempting to group conflict stages by logical conflict rather than by
> >    affected target filepath[4]
> >  * Providing similar functionality for doing cherry-picks/rebases/reverts,
> >    i.e. a scheme for three-way merges with a specified merge-base[5]. That's
> >    being deferred to a future series. [2]
> >    https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
> >    [3]
> >    https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/
> >    [4]
> >    https://lore.kernel.org/git/CABPp-BGCL0onSmpgKuO1k2spYCkx=v27ed9TSSxFib=OdDcLbw@mail.gmail.com/
> >    [5]
> >    https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/
>
> I've looked through this, I think it all looks good overall & that the
> things that needed to be addressed (as opposed to my --format rambling)
> have been.
>
> I think all the code should be ready for "next".
>
> I suggested (I think around getopts discussion) in an earlier that the
> code would have been easier with a new built-in, but if we're
> deprecating the existing "mode" I think using the name is probably
> better in the end.
>
> I find the resulting documentation to be really hard to grok though
> because we're effectively describing two different commands. The current
> docs are small: https://git-scm.com/docs/git-merge-tree
>
> I built the tip of this series and read the manpage, and found myself
> needing to carefully squint to see what referred to what mode in the
> docs.
>
> E.g. by the time it's discussing "-z" and other options the reader needs
> to be astute really aware of the context, and infer from the lack of
> "<options>" on the "--trivial-merge" that these options refer to the
> "--write-tree" only.
>
> The same goes for the rest of "--trivial-merge". I.e. I found myself
> needing to read the whole docs word-by-word (no skimming!) to see if
> OUTPUT etc. was going to describe its output, or just the "new" mode.
>
> It is my NSHO that man pages should be structured for the impatient
> reader :)
>
> Then when you say "git-merge-tree was written to be[...]" I thought "ah
> ha! surely this will discuss the since-2005 implemented mode", but "was
> written to be" is referring to code new in this series.
>
> The below patch-on-top addresses all those concerns. Basically I just
> added a line to the top of the DESCRIPTION saying that you should read a
> "DEPRECATED DESCRIPTION" section at the end for "--trivial-merge", and
> that all of the rest is talking about the "--write-tree" mode.
>
> I then edited various prose to do away with the now-unnecessary "the
> first form" etc.
>
> This diff is better against "master" in that you'll see that the current
> merge-tree DESCRIPTION section isn't touched at all (it's now just under
> a new heading), but this diff is against the tip of your series.
>
> There's various other small fixes while I was it it here, e.g. all your
> cross-section links were using some pseudo-not-quite-ASCIIDOC syntax
> that doesn't work. Now it uses the right syntax. Ditto link-ifying
> references to "mktag" etc.
>
> I don't know if you'd consider this for a v6, or if I should just submit
> this on top myself, but in any case here it is. I'll leave it to you how
> you'd like to proceed with it:

Overall the diff below looks good; thanks!  I'll include it in a v6.
Two exceptions, listed below:

> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 306149fa0e2..723b1995426 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -9,17 +9,24 @@ git-merge-tree - Perform merge without touching index or working tree
>  SYNOPSIS
>  --------
>  [verse]
> -'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
> -'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
> +'git merge-tree' --write-tree [<options>] <branch1> <branch2>
> +'git merge-tree' --trivial-merge <base-tree> <branch1> <branch2>

I dislike all three changes to the synopsis; I'll discard these.

> +[[NEWMERGE]]
>  DESCRIPTION
>  -----------
>
> -Performs a merge, but does not make any new commits and does not read
> -from or write to either the working tree or index.
> +This command has a modern `--write-tree` mode and a deprecated
> +`--trivial-merge` mode. The rest of this documentation describes
> +modern `--write-tree` mode unless otherwise specified. see
> +<<DEPMERGE,DEPRECATED DESCRIPTION>> below for a summary of the
> +`--trivial-merge` mode.
>
> -The first form will merge the two branches, doing a real merge.  A real
> -merge is distinguished from a trivial merge in that it includes:
> +Performs a "real" merge, but does not make any new commits and does
> +not read from or write to either the working tree or index.
> +
> +The performed merge will use the same feature as the "real"
> +linkgit:git-merge[1], including:
>
>    * three way content merges of individual files
>    * rename detection
> @@ -28,24 +35,8 @@ merge is distinguished from a trivial merge in that it includes:
>      merge base, creating a virtual merge base by merging the merge bases)
>    * etc.
>
> -After the merge completes, the first form will create a new toplevel
> -tree object.  See `OUTPUT` below for details.
> -
> -The second form is deprecated; it is kept for backward compatibility
> -reasons but may be deleted in the future.  Other than the optional
> -`--trivial-merge`, it accepts no options.  It can only do a trivial
> -merge.  It reads three tree-ish, and outputs trivial merge results and
> -conflicting stages to the standard output in a semi-diff format.
> -Since this was designed for higher level scripts to consume and merge
> -the results back into the index, it omits entries that match
> -<branch1>.  The result of this second form is similar to what
> -three-way 'git read-tree -m' does, but instead of storing the results
> -in the index, the command outputs the entries to the standard output.
> -This form not only has limited applicability, the output format is
> -also difficult to work with, and it will generally be less performant
> -than the first form even on successful merges (especially if working
> -in large repositories).  The remainder of this manual will only
> -discuss the first form.
> +After the merge completes, a newtoplevel tree object is created.  See
> +`OUTPUT` below for details.
>
>  OPTIONS
>  -------
> @@ -54,7 +45,7 @@ OPTIONS
>         Do not quote filenames in the <Conflicted file info> section,
>         and end each filename with a NUL character rather than
>         newline.  Also begin the messages section with a NUL character
> -       instead of a newline.  See OUTPUT below for more information.
> +       instead of a newline.  See <<OUTPUT>> below for more information.
>
>  --name-only::
>         In the Conflicted file info section, instead of writing a list
> @@ -74,11 +65,12 @@ OPTIONS
>         share no common history.  This flag can be given to override that
>         check and make the merge proceed anyway.
>
> +[[OUTPUT]]
>  OUTPUT
>  ------
>
> -By default, for a successful merge, the output from git-merge-tree is
> -simply one line:
> +For a successful merge, the output from git-merge-tree is simply one
> +line:
>
>         <OID of toplevel tree>
>
> @@ -90,6 +82,7 @@ Whereas for a conflicted merge, the output is by default of the form:
>
>  These are discussed individually below.
>
> +[[OIDTLT]]
>  OID of toplevel tree
>  ~~~~~~~~~~~~~~~~~~~~
>
> @@ -98,6 +91,7 @@ working tree at the end of `git merge`.  If there were conflicts, then
>  files within this tree may have embedded conflict markers.  This section
>  is always followed by a newline (or NUL if `-z` is passed).
>
> +[[CFI]]
>  Conflicted file info
>  ~~~~~~~~~~~~~~~~~~~~
>
> @@ -111,6 +105,7 @@ the `--name-only` option is passed, the mode, object, and stage will
>  be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
>  character instead of a newline character.
>
> +[[IM]]
>  Informational messages
>  ~~~~~~~~~~~~~~~~~~~~~~
>
> @@ -138,72 +133,94 @@ something other than 0 or 1 (and the output is unspecified).
>  USAGE NOTES
>  -----------
>
> -git-merge-tree was written to be low-level plumbing, similar to
> -hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
> -Thus, it could be used as a part of a series of steps such as
> +This command is intended as low-level plumbing, similar to
> +linkgit:git-hash-object[1], linkgit:git-mktree[1],
> +linkgit:git-commit-tree[1], linkgit:git-write-tree[1],
> +linkgit:git-update-ref[1], and linkgit:git-mktag[1].  Thus, it can be
> +used as a part of a series of steps such as:
>
>         NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
>         test $? -eq 0 || die "There were conflicts..."
>         NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
>         git update-ref $BRANCH1 $NEWCOMMIT
>
> -Note that when the exit status is non-zero, NEWTREE in this sequence
> +Note that when the exit status is non-zero, `NEWTREE` in this sequence
>  will contain a lot more output than just a tree.
>
> -git-merge-tree was written to provide users with the same information
> -that they'd have access to if using `git merge`:
> -  * what would be written to the working tree (the <OID of toplevel tree>)
> +The output will include the same information that you'd get with
> +linkgit:git-merge[1]:
> +
> +  * what would be written to the working tree (the <<OIDTLT,OID of toplevel tree>>)
>    * the higher order stages that would be written to the index (the
> -    <Conflicted file info>)
> -  * any messages that would have been printed to stdout (the <Informational
> -    messages>)
> +    <<CFI,Conflicted file info>>)
> +  * any messages that would have been printed to stdout (the <<IM,Informational
> +    messages>>)
>
>  MISTAKES TO AVOID
>  -----------------
>
>  Do NOT look through the resulting toplevel tree to try to find which
> -files conflict; parse the <Conflicted file info> section instead.  Not
> +files conflict; parse the <<CFI,Conflicted file info>> section instead.  Not
>  only would parsing an entire tree be horrendously slow in large
>  repositories, there are numerous types of conflicts not representable by
>  conflict markers (modify/delete, mode conflict, binary file changed on
>  both sides, file/directory conflicts, various rename conflict
>  permutations, etc.)
>
> -Do NOT interpret an empty <Conflicted file info> list as a clean merge;
> +Do NOT interpret an empty <<CFI,Conflicted file info>> list as a clean merge;
>  check the exit status.  A merge can have conflicts without having
>  individual files conflict (there are a few types of directory rename
>  conflicts that fall into this category, and others might also be added
>  in the future).
>
>  Do NOT attempt to guess or make the user guess the conflict types from
> -the <Conflicted file info> list.  The information there is insufficient
> +the <<CFI,Conflicted file info>> list.  The information there is insufficient
>  to do so.  For example: Rename/rename(1to2) conflicts (both sides
>  renamed the same file differently) will result in three different file
>  having higher order stages (but each only has one higher order stage),
> -with no way (short of the <Informational messages> section) to determine
> +with no way (short of the <<IM,Informational messages>> section) to determine
>  which three files are related.  File/directory conflicts also result in
>  a file with exactly one higher order stage.
>  Possibly-involved-in-directory-rename conflicts (when
>  "merge.directoryRenames" is unset or set to "conflicts") also result in
>  a file with exactly one higher order stage.  In all cases, the
> -<Informational messages> section has the necessary info, though it is
> +<<IM,Informational messages>> section has the necessary info, though it is
>  not designed to be machine parseable.
>
> -Do NOT assume all filenames listed in the <Informational messages>
> +Do NOT assume all filenames listed in the <<IM,Informational messages>>
>  section had conflicts.  Messages can be included for files that have no
>  conflicts, such as "Auto-merging <file>".
>
> -AVOID taking the OIDS from the <Conflicted file info> and re-merging
> +AVOID taking the OIDS from the <<CFI,Conflicted file info>> and re-merging
>  them to present the conflicts to the user.  This will lose information.
> -Instead, look up the version of the file found within the <OID of
> -toplevel tree> and show that instead.  In particular, the latter will
> +Instead, look up the version of the file found within the <<OIDTLT,OID of
> +toplevel tree>> and show that instead.  In particular, the latter will
>  have conflict markers annotated with the original branch/commit being
>  merged and, if renames were involved, the original filename.  While you
>  could include the original branch/commit in the conflict marker
>  annotations when re-merging, the original filename is not available from
> -the <Conflicted file info> and thus you would be losing information that
> +the <<CFI,Conflicted file info>> and thus you would be losing information that
>  might help the user resolve the conflict.
>
> +[[DEPMERGE]]
> +DEPRECATED DESCRIPTION
> +----------------------
> +
> +Per the <<NEWMERGE,DESCRIPTION>> and unlike the rest of this
> +documentation this section describes describes the deprecated
> +`--trivial-merge` mode.
> +
> +Reads three tree-ish, and output trivial merge results and
> +conflicting stages to the standard output.  This is similar to
> +what three-way 'git read-tree -m' does, but instead of storing the
> +results in the index, the command outputs the entries to the
> +standard output.
> +
> +This is meant to be used by higher level scripts to compute
> +merge results outside of the index, and stuff the results back into the
> +index.  For this reason, the output from the command omits
> +entries that match the <branch1> tree.

What you have in the new section is fine, but what you've deleted
should probably be included here.  In particular, the brief
explanation about why this mode is deprecated.


> +
>  GIT
>  ---
>  Part of the linkgit:git[1] suite
>

Re: [PATCH v5 04/12] merge-tree: implement real merges

[Elijah Newren]

On Sun, Feb 20, 2022 at 1:03 AM René Scharfe <l.s.r@web.de> wrote:
>
> Am 20.02.22 um 07:54 schrieb Elijah Newren via GitGitGadget:
[...]
> > +     /*
> > +      * Get the merge bases, in reverse order; see comment above
> > +      * merge_incore_recursive in merge-ort.h
> > +      */
> > +     common = get_merge_bases(parent1, parent2);
> > +     if (!common)
> > +             die(_("refusing to merge unrelated histories"));
> > +     for (j = common; j; j = j->next)
> > +             commit_list_insert(j->item, &merge_bases);
>
> This loop creates a reversed copy of "common".  You could use
> reverse_commit_list() instead to do it in-place and avoid the
> allocations.  Only the copy, "merge_bases", is used below.

Oh, good catch.  I probably should have been aware of this since
someone requested I move the reverse_commit_list() function from
merge-recursive.c to commit.c as part of my merge-ort work, but looks
like I forgot about it and copied this command snippet from
builtin/merge.c instead.  I have no excuse.

However, I wonder if that means we could also apply this
simplification to the code snippets in builtin/merge.c and sequencer.c
that you can find with
    git grep commit_list_insert.*reversed
?  Maybe #leftoverbits for that part?

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren]

On Mon, Feb 21, 2022 at 1:06 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi,
>
> On Thu, 3 Feb 2022, Elijah Newren wrote:

[...]
> > I agree.  I've got a lot of thoughts on it, and some work in progress
> > towards it (https://github.com/newren/git/tree/replay -- _very_ hacky,
> > not even close to alpha quality, lots of fixup commits, todo comments,
> > random brain dump files added to the tree, based on a previous round
> > of this patch series, not updated for weeks, etc., etc.)
>
> Just chiming in that I find that very exciting. But it's a tangent, and
> slightly distracting from the topic at hand, so I would like to ask to
> focus back on server-side merges.

+1 for refocusing.

> > We'll certainly have discussions on what that should look like.  But a
> > plumbing-ish replacement for merge was much simpler, and made sense to
> > do first.  I would prefer to concentrate on getting that hammered down
> > first.  Then I'll start discussions on a plumbing-ish
> > rebase/cherry-pick.  And if that doesn't fulfill all the needs that
> > folks think they want out of merge-tree, then we can add a
> > merge_incore_nonrecursive()-based mode to merge-tree.  It's all
> > coming, but having fought transliterations-of-scripts in
> > merge-recursive.c, sequencer.c, stash.c, rebase.c, etc. for years I
> > really, really don't want any more of that.  Let's end that insanity.
>
> Being the driving force behind many a "built-in-ification" of scripted
> commands, I wholeheartedly agree. You can still see the fall-out of
> designing commands in a scripted fashion, without any way to represent
> data structures other than strings. I wish we had come up with a better
> design to prototype commands than to write shell scripts. But I have to
> admit that even I do not have any better idea than to work on a proper API
> for libgit.a (which has historically invariably seen push-back from
> Junio).
>
> While I agree that this discussion is a valuable one, right now I would
> like to focus on getting the server-side merges done, and once that has
> happened, move on to the replay/sequencer/API discussion (which will
> probably be a big one, not so much for technical reasons but more for all
> too human ones).

I'm just guessing, but I suspect your prediction for the future
replay/sequencer/rebase/API discussion will be spot on.  :-)

Re: [PATCH v5 00/12] In-core git merge-tree ("Server side merges")

[Ævar Arnfjörð Bjarmason]

On Mon, Feb 21 2022, Elijah Newren wrote:

> On Sun, Feb 20, 2022 at 4:35 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> On Sun, Feb 20 2022, Elijah Newren via GitGitGadget wrote:
>>
>> > == Updates Log ==
>> >
>> > Many thanks to the many reviewers who provided good feedback on the most
>> > recent round -- Junio, Ævar, Josh, Emily, and perhaps some others I've
>> > forgotten from review club.
>> >
>> > Updates since v4:
>> >
>> >  * Fixed double "is" in documentation.
>> >  * Fixed a few small items with testcases
>> >
>> > Updates since v3 (or v5, if you include the rounds at
>> > https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):
>> >
>> >  * Dropped previous patches 5, 6, and 8 of the old series; they weren't
>> >    being used and opened a can of worms[1]
>> >  * [Patch 3] Restructured argument checking, including using an enum
>> >  * [Patch 4] Restored the extended paragraph about the deprecated form of
>> >    git-merge-tree, mentioned write-tree in plumbing commands, and a few
>> >    other small fixups to the documentation
>> >  * [Patch 4] Also provide an example of a clean merge rather than just a
>> >    conflicted one
>> >  * [Patch 6] Fix the incompatible arguments check and add some tests for it
>> >  * [Patch 6] Introduce an anonymize_hash() shell function to make tests
>> >    easier to read (less repeated sed)
>> >  * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
>> >    option for now
>> >  * [Patch 10] When -z passed, the tree in the first section should have a
>> >    trailing NUL rather than trailing newline [1]
>> >    https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/
>> >
>> > Stuff NOT included that reviewers brought up in earlier rounds:
>> >
>> >  * Very generic (mode, oid, stage, filename) printing formatting[2]
>> >  * Always printing 3 stages for each filename with conflicts[3]
>> >  * Attempting to group conflict stages by logical conflict rather than by
>> >    affected target filepath[4]
>> >  * Providing similar functionality for doing cherry-picks/rebases/reverts,
>> >    i.e. a scheme for three-way merges with a specified merge-base[5]. That's
>> >    being deferred to a future series. [2]
>> >    https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
>> >    [3]
>> >    https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/
>> >    [4]
>> >    https://lore.kernel.org/git/CABPp-BGCL0onSmpgKuO1k2spYCkx=v27ed9TSSxFib=OdDcLbw@mail.gmail.com/
>> >    [5]
>> >    https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/
>>
>> I've looked through this, I think it all looks good overall & that the
>> things that needed to be addressed (as opposed to my --format rambling)
>> have been.
>>
>> I think all the code should be ready for "next".
>>
>> I suggested (I think around getopts discussion) in an earlier that the
>> code would have been easier with a new built-in, but if we're
>> deprecating the existing "mode" I think using the name is probably
>> better in the end.
>>
>> I find the resulting documentation to be really hard to grok though
>> because we're effectively describing two different commands. The current
>> docs are small: https://git-scm.com/docs/git-merge-tree
>>
>> I built the tip of this series and read the manpage, and found myself
>> needing to carefully squint to see what referred to what mode in the
>> docs.
>>
>> E.g. by the time it's discussing "-z" and other options the reader needs
>> to be astute really aware of the context, and infer from the lack of
>> "<options>" on the "--trivial-merge" that these options refer to the
>> "--write-tree" only.
>>
>> The same goes for the rest of "--trivial-merge". I.e. I found myself
>> needing to read the whole docs word-by-word (no skimming!) to see if
>> OUTPUT etc. was going to describe its output, or just the "new" mode.
>>
>> It is my NSHO that man pages should be structured for the impatient
>> reader :)
>>
>> Then when you say "git-merge-tree was written to be[...]" I thought "ah
>> ha! surely this will discuss the since-2005 implemented mode", but "was
>> written to be" is referring to code new in this series.
>>
>> The below patch-on-top addresses all those concerns. Basically I just
>> added a line to the top of the DESCRIPTION saying that you should read a
>> "DEPRECATED DESCRIPTION" section at the end for "--trivial-merge", and
>> that all of the rest is talking about the "--write-tree" mode.
>>
>> I then edited various prose to do away with the now-unnecessary "the
>> first form" etc.
>>
>> This diff is better against "master" in that you'll see that the current
>> merge-tree DESCRIPTION section isn't touched at all (it's now just under
>> a new heading), but this diff is against the tip of your series.
>>
>> There's various other small fixes while I was it it here, e.g. all your
>> cross-section links were using some pseudo-not-quite-ASCIIDOC syntax
>> that doesn't work. Now it uses the right syntax. Ditto link-ifying
>> references to "mktag" etc.
>>
>> I don't know if you'd consider this for a v6, or if I should just submit
>> this on top myself, but in any case here it is. I'll leave it to you how
>> you'd like to proceed with it:
>
> Overall the diff below looks good; thanks!  I'll include it in a v6.
> Two exceptions, listed below:

Thanks.

>> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
>> index 306149fa0e2..723b1995426 100644
>> --- a/Documentation/git-merge-tree.txt
>> +++ b/Documentation/git-merge-tree.txt
>> @@ -9,17 +9,24 @@ git-merge-tree - Perform merge without touching index or working tree
>>  SYNOPSIS
>>  --------
>>  [verse]
>> -'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
>> -'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
>> +'git merge-tree' --write-tree [<options>] <branch1> <branch2>
>> +'git merge-tree' --trivial-merge <base-tree> <branch1> <branch2>
>
> I dislike all three changes to the synopsis; I'll discard these.

I don't know why I changed --write-tree to [--write-tree], I think
that's just incorrect.

But what I was going for was removing "(deprecated)" there.

Maybe it's confusing to noone else, but it is ambiguous in the way we
usually use parens in SYNOPSIS. I.e. per that syntax it means that the
string "deprecated" is a mandotory paramater after <branch2>.

Maybe just having a "DEPRECATED SYNOPSIS" section after "SYNOPSIS" would
be better? That would also allow for more lines of examples without
re-wording everything that comes after with "the first form" etc.

>> +[[NEWMERGE]]
>>  DESCRIPTION
>>  -----------
>>
>> -Performs a merge, but does not make any new commits and does not read
>> -from or write to either the working tree or index.
>> +This command has a modern `--write-tree` mode and a deprecated
>> +`--trivial-merge` mode. The rest of this documentation describes
>> +modern `--write-tree` mode unless otherwise specified. see
>> +<<DEPMERGE,DEPRECATED DESCRIPTION>> below for a summary of the
>> +`--trivial-merge` mode.
>>
>> -The first form will merge the two branches, doing a real merge.  A real
>> -merge is distinguished from a trivial merge in that it includes:
>> +Performs a "real" merge, but does not make any new commits and does
>> +not read from or write to either the working tree or index.
>> +
>> +The performed merge will use the same feature as the "real"
>> +linkgit:git-merge[1], including:
>>
>>    * three way content merges of individual files
>>    * rename detection
>> @@ -28,24 +35,8 @@ merge is distinguished from a trivial merge in that it includes:
>>      merge base, creating a virtual merge base by merging the merge bases)
>>    * etc.
>>
>> -After the merge completes, the first form will create a new toplevel
>> -tree object.  See `OUTPUT` below for details.
>> -
>> -The second form is deprecated; it is kept for backward compatibility
>> -reasons but may be deleted in the future.  Other than the optional
>> -`--trivial-merge`, it accepts no options.  It can only do a trivial
>> -merge.  It reads three tree-ish, and outputs trivial merge results and
>> -conflicting stages to the standard output in a semi-diff format.
>> -Since this was designed for higher level scripts to consume and merge
>> -the results back into the index, it omits entries that match
>> -<branch1>.  The result of this second form is similar to what
>> -three-way 'git read-tree -m' does, but instead of storing the results
>> -in the index, the command outputs the entries to the standard output.
>> -This form not only has limited applicability, the output format is
>> -also difficult to work with, and it will generally be less performant
>> -than the first form even on successful merges (especially if working
>> -in large repositories).  The remainder of this manual will only
>> -discuss the first form.
>> +After the merge completes, a newtoplevel tree object is created.  See
>> +`OUTPUT` below for details.
>>
>>  OPTIONS
>>  -------
>> @@ -54,7 +45,7 @@ OPTIONS
>>         Do not quote filenames in the <Conflicted file info> section,
>>         and end each filename with a NUL character rather than
>>         newline.  Also begin the messages section with a NUL character
>> -       instead of a newline.  See OUTPUT below for more information.
>> +       instead of a newline.  See <<OUTPUT>> below for more information.
>>
>>  --name-only::
>>         In the Conflicted file info section, instead of writing a list
>> @@ -74,11 +65,12 @@ OPTIONS
>>         share no common history.  This flag can be given to override that
>>         check and make the merge proceed anyway.
>>
>> +[[OUTPUT]]
>>  OUTPUT
>>  ------
>>
>> -By default, for a successful merge, the output from git-merge-tree is
>> -simply one line:
>> +For a successful merge, the output from git-merge-tree is simply one
>> +line:
>>
>>         <OID of toplevel tree>
>>
>> @@ -90,6 +82,7 @@ Whereas for a conflicted merge, the output is by default of the form:
>>
>>  These are discussed individually below.
>>
>> +[[OIDTLT]]
>>  OID of toplevel tree
>>  ~~~~~~~~~~~~~~~~~~~~
>>
>> @@ -98,6 +91,7 @@ working tree at the end of `git merge`.  If there were conflicts, then
>>  files within this tree may have embedded conflict markers.  This section
>>  is always followed by a newline (or NUL if `-z` is passed).
>>
>> +[[CFI]]
>>  Conflicted file info
>>  ~~~~~~~~~~~~~~~~~~~~
>>
>> @@ -111,6 +105,7 @@ the `--name-only` option is passed, the mode, object, and stage will
>>  be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
>>  character instead of a newline character.
>>
>> +[[IM]]
>>  Informational messages
>>  ~~~~~~~~~~~~~~~~~~~~~~
>>
>> @@ -138,72 +133,94 @@ something other than 0 or 1 (and the output is unspecified).
>>  USAGE NOTES
>>  -----------
>>
>> -git-merge-tree was written to be low-level plumbing, similar to
>> -hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
>> -Thus, it could be used as a part of a series of steps such as
>> +This command is intended as low-level plumbing, similar to
>> +linkgit:git-hash-object[1], linkgit:git-mktree[1],
>> +linkgit:git-commit-tree[1], linkgit:git-write-tree[1],
>> +linkgit:git-update-ref[1], and linkgit:git-mktag[1].  Thus, it can be
>> +used as a part of a series of steps such as:
>>
>>         NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
>>         test $? -eq 0 || die "There were conflicts..."
>>         NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
>>         git update-ref $BRANCH1 $NEWCOMMIT
>>
>> -Note that when the exit status is non-zero, NEWTREE in this sequence
>> +Note that when the exit status is non-zero, `NEWTREE` in this sequence
>>  will contain a lot more output than just a tree.
>>
>> -git-merge-tree was written to provide users with the same information
>> -that they'd have access to if using `git merge`:
>> -  * what would be written to the working tree (the <OID of toplevel tree>)
>> +The output will include the same information that you'd get with
>> +linkgit:git-merge[1]:
>> +
>> +  * what would be written to the working tree (the <<OIDTLT,OID of toplevel tree>>)
>>    * the higher order stages that would be written to the index (the
>> -    <Conflicted file info>)
>> -  * any messages that would have been printed to stdout (the <Informational
>> -    messages>)
>> +    <<CFI,Conflicted file info>>)
>> +  * any messages that would have been printed to stdout (the <<IM,Informational
>> +    messages>>)
>>
>>  MISTAKES TO AVOID
>>  -----------------
>>
>>  Do NOT look through the resulting toplevel tree to try to find which
>> -files conflict; parse the <Conflicted file info> section instead.  Not
>> +files conflict; parse the <<CFI,Conflicted file info>> section instead.  Not
>>  only would parsing an entire tree be horrendously slow in large
>>  repositories, there are numerous types of conflicts not representable by
>>  conflict markers (modify/delete, mode conflict, binary file changed on
>>  both sides, file/directory conflicts, various rename conflict
>>  permutations, etc.)
>>
>> -Do NOT interpret an empty <Conflicted file info> list as a clean merge;
>> +Do NOT interpret an empty <<CFI,Conflicted file info>> list as a clean merge;
>>  check the exit status.  A merge can have conflicts without having
>>  individual files conflict (there are a few types of directory rename
>>  conflicts that fall into this category, and others might also be added
>>  in the future).
>>
>>  Do NOT attempt to guess or make the user guess the conflict types from
>> -the <Conflicted file info> list.  The information there is insufficient
>> +the <<CFI,Conflicted file info>> list.  The information there is insufficient
>>  to do so.  For example: Rename/rename(1to2) conflicts (both sides
>>  renamed the same file differently) will result in three different file
>>  having higher order stages (but each only has one higher order stage),
>> -with no way (short of the <Informational messages> section) to determine
>> +with no way (short of the <<IM,Informational messages>> section) to determine
>>  which three files are related.  File/directory conflicts also result in
>>  a file with exactly one higher order stage.
>>  Possibly-involved-in-directory-rename conflicts (when
>>  "merge.directoryRenames" is unset or set to "conflicts") also result in
>>  a file with exactly one higher order stage.  In all cases, the
>> -<Informational messages> section has the necessary info, though it is
>> +<<IM,Informational messages>> section has the necessary info, though it is
>>  not designed to be machine parseable.
>>
>> -Do NOT assume all filenames listed in the <Informational messages>
>> +Do NOT assume all filenames listed in the <<IM,Informational messages>>
>>  section had conflicts.  Messages can be included for files that have no
>>  conflicts, such as "Auto-merging <file>".
>>
>> -AVOID taking the OIDS from the <Conflicted file info> and re-merging
>> +AVOID taking the OIDS from the <<CFI,Conflicted file info>> and re-merging
>>  them to present the conflicts to the user.  This will lose information.
>> -Instead, look up the version of the file found within the <OID of
>> -toplevel tree> and show that instead.  In particular, the latter will
>> +Instead, look up the version of the file found within the <<OIDTLT,OID of
>> +toplevel tree>> and show that instead.  In particular, the latter will
>>  have conflict markers annotated with the original branch/commit being
>>  merged and, if renames were involved, the original filename.  While you
>>  could include the original branch/commit in the conflict marker
>>  annotations when re-merging, the original filename is not available from
>> -the <Conflicted file info> and thus you would be losing information that
>> +the <<CFI,Conflicted file info>> and thus you would be losing information that
>>  might help the user resolve the conflict.
>>
>> +[[DEPMERGE]]
>> +DEPRECATED DESCRIPTION
>> +----------------------
>> +
>> +Per the <<NEWMERGE,DESCRIPTION>> and unlike the rest of this
>> +documentation this section describes describes the deprecated
>> +`--trivial-merge` mode.
>> +
>> +Reads three tree-ish, and output trivial merge results and
>> +conflicting stages to the standard output.  This is similar to
>> +what three-way 'git read-tree -m' does, but instead of storing the
>> +results in the index, the command outputs the entries to the
>> +standard output.
>> +
>> +This is meant to be used by higher level scripts to compute
>> +merge results outside of the index, and stuff the results back into the
>> +index.  For this reason, the output from the command omits
>> +entries that match the <branch1> tree.
>
> What you have in the new section is fine, but what you've deleted
> should probably be included here.  In particular, the brief
> explanation about why this mode is deprecated.

Ah, yes. That was overzelous. Sorry. I skimmed that and IIRC thought it
was a re-flowing of the old DESCRIPTION section.

Re: [PATCH v5 04/12] merge-tree: implement real merges

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

Hi Elijah,

On Mon, 21 Feb 2022, Elijah Newren wrote:

> On Sun, Feb 20, 2022 at 1:03 AM René Scharfe <l.s.r@web.de> wrote:
> >
> > Am 20.02.22 um 07:54 schrieb Elijah Newren via GitGitGadget:
> [...]
> > > +     /*
> > > +      * Get the merge bases, in reverse order; see comment above
> > > +      * merge_incore_recursive in merge-ort.h
> > > +      */
> > > +     common = get_merge_bases(parent1, parent2);
> > > +     if (!common)
> > > +             die(_("refusing to merge unrelated histories"));
> > > +     for (j = common; j; j = j->next)
> > > +             commit_list_insert(j->item, &merge_bases);
> >
> > This loop creates a reversed copy of "common".  You could use
> > reverse_commit_list() instead to do it in-place and avoid the
> > allocations.  Only the copy, "merge_bases", is used below.
>
> Oh, good catch.  I probably should have been aware of this since
> someone requested I move the reverse_commit_list() function from
> merge-recursive.c to commit.c as part of my merge-ort work, but looks
> like I forgot about it and copied this command snippet from
> builtin/merge.c instead.  I have no excuse.

Ooops! I missed that the `reverse_commit_list()` function was moved to
`commit.c` by _you_, and had not been there all along (my fault, of
course, see 8918b0c9c26 (merge-recur: try to merge older merge bases
first, 2006-08-09)).

> However, I wonder if that means we could also apply this
> simplification to the code snippets in builtin/merge.c and sequencer.c
> that you can find with
>     git grep commit_list_insert.*reversed
> ?  Maybe #leftoverbits for that part?

Yes, that's a good idea. I summarized this left-over-bit in
https://github.com/gitgitgadget/git/issues/1156

Ciao,
Dscho

Re: [PATCH v5 12/12] git-merge-tree.txt: add a section on potentional usage mistakes

[Johannes Schindelin]

Hi Elijah,

On Sun, 20 Feb 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)

Thank you for this. It addresses the concern I had about printing out the
tree object ID in the conflicted case.

Thanks,
Dscho

>
> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index d2ff2fa3035..306149fa0e2 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -158,6 +158,52 @@ that they'd have access to if using `git merge`:
>    * any messages that would have been printed to stdout (the <Informational
>      messages>)
>
> +MISTAKES TO AVOID
> +-----------------
> +
> +Do NOT look through the resulting toplevel tree to try to find which
> +files conflict; parse the <Conflicted file info> section instead.  Not
> +only would parsing an entire tree be horrendously slow in large
> +repositories, there are numerous types of conflicts not representable by
> +conflict markers (modify/delete, mode conflict, binary file changed on
> +both sides, file/directory conflicts, various rename conflict
> +permutations, etc.)
> +
> +Do NOT interpret an empty <Conflicted file info> list as a clean merge;
> +check the exit status.  A merge can have conflicts without having
> +individual files conflict (there are a few types of directory rename
> +conflicts that fall into this category, and others might also be added
> +in the future).
> +
> +Do NOT attempt to guess or make the user guess the conflict types from
> +the <Conflicted file info> list.  The information there is insufficient
> +to do so.  For example: Rename/rename(1to2) conflicts (both sides
> +renamed the same file differently) will result in three different file
> +having higher order stages (but each only has one higher order stage),
> +with no way (short of the <Informational messages> section) to determine
> +which three files are related.  File/directory conflicts also result in
> +a file with exactly one higher order stage.
> +Possibly-involved-in-directory-rename conflicts (when
> +"merge.directoryRenames" is unset or set to "conflicts") also result in
> +a file with exactly one higher order stage.  In all cases, the
> +<Informational messages> section has the necessary info, though it is
> +not designed to be machine parseable.
> +
> +Do NOT assume all filenames listed in the <Informational messages>
> +section had conflicts.  Messages can be included for files that have no
> +conflicts, such as "Auto-merging <file>".
> +
> +AVOID taking the OIDS from the <Conflicted file info> and re-merging
> +them to present the conflicts to the user.  This will lose information.
> +Instead, look up the version of the file found within the <OID of
> +toplevel tree> and show that instead.  In particular, the latter will
> +have conflict markers annotated with the original branch/commit being
> +merged and, if renames were involved, the original filename.  While you
> +could include the original branch/commit in the conflict marker
> +annotations when re-merging, the original filename is not available from
> +the <Conflicted file info> and thus you would be losing information that
> +might help the user resolve the conflict.
> +
>  GIT
>  ---
>  Part of the linkgit:git[1] suite
> --
> gitgitgadget
>

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren]

On Mon, Feb 21, 2022 at 10:55 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > Adding such an ability to merge-tree would be trivial -- it basically
> > involves just two things: (1) accepting one extra argument, and (2)
> > calling merge_incore_nonrecursive() instead of
> > merge_incore_recursive().
> >
> > However, I think forking a subprocess for every merge of a series of
> > commits is a completely unreasonable overhead, so even if we provide
> > such an option to merge-tree, I still want a separate plumbing-ish
> > tool that does non-worktree/non-index replaying of commits which is
> > not written as a driver of merge-tree.  That other tool should just
> > call merge_incore_nonrecursive() directly.  And such a tool, since it
> > should handle an arbitrary number of commits, should certainly be able
> > to handle just one commit.  From that angle, it feels like adding
> > another mode to merge-tree would just be a partial duplication of the
> > other tool.
>
> The above does not make much sense to me.
>
> I am hearing that "multi-step cherry-picks and reverts need to be
> fast and we need something like sequencer that is all written in C,

Yes, I agree with that part so far.  jj is kicking our butt on rebase
speed; I'm not sure if we can catch it, but it'd be nice to see us not
be more than a hundred times slower.

> and single-step cherry-pick is merely a special case that does not
> deserve a plumbing".

Well, apparently I failed at communication if that's what you heard.
Perhaps I can step back and provide my high-level goals, and then
mention how this series fits in.  My high-level goals:

  * new sequencer-like replay tool, including multiple abilities
today's rebase/cherry-pick tools don't have
  * enable folks to use merging machinery for server side operations
(merge, rebase, cherry-pick, revert)
  * do not repeat or encourage the rebase-as-shell-script mistakes of yesteryear
  * somehow split this up into reviewable chunks

Now, in particular, the "merge divergent branches" piece seemed like a
really simple portion of the problem space for which I could get some
early feedback without having to address the whole problem space all
at once, and which doesn't seem to have any downside risk.

And even with my attempt to narrow it in scope, and even despite lots
of early feedback from the Git Virtual Contributor Summit six months
ago, it's been nearly two months of active discussions including all
kinds of intrinsic and tangential points about the UI and design.  Why
try to prematurely widen the scope?  Can we just focus on merging
divergent branches for now, and cover the rest later?

> But that argument leads to "and the same something-like-sequencer
> that is all written in C would need '--rebase-merges' that can pick
> multi-step merge sequences, and single-step merge does not deserve a
> plumbing", which is an argument against this topic that is utterly
> absurd.
>
> So why isn't your objection not equally absurd against having a
> single step cherry-pick or revert primitive as a plumbing?

The objection you are arguing against is not my position.  In fact,
I'm not even objecting to having a single-step cherry-pick, I'm
objecting to providing it _now_, which I thought would have been clear
from the portion of my email you snipped ("...I'm happy to add [a
single step cherry-pick primitive] along with the tool I submit
later...").  Since that wasn't clear, and since that wasn't my only
communication failure here, let me attempt to be clearer about my
objection(s):

1. I'm really trying to pick off a small piece of the problem space
and get feedback on it without unnecessarily complicating things with
unrelated issues.  Thus, this series is _only_ about merging branches
that have diverged, and leaves commit replaying for later.

2. Two folks have chimed in about the single step cherry-pick, and the
ONLY reason given for wanting such a thing was to create a
rebasing/cherry-picking script which was driven by repeatedly invoking
this low-level primitive command.  That's also the only usecase I can
currently think of for such a primitive.  To me, that means providing
such a low-level command now would be likely to result in the
rebase-as-a-script mistake of yesteryear.  I think we can avoid that
pitfall by first providing a tool that avoids the
repeatedly-fork-git-subprocesses model.  (Also, providing a low-level
single-step cherry-pick command also has the added negative of further
distracting from the focus on merging divergent branches.)

3. The merge primitive in this series is useful completely independent
of any rebasing script (it would not be used solely for rebasing
merges, if it's used for that purpose at all, as evidenced by the fact
that dscho is already trying to use it for doing new real merges).

4. Once we have a git-replay tool that can replay a sequence of
commits, there _might_ not be a need for a single commit replaying
primitive.  If we provided one as you and Johannes Altimanninger were
asking for, and it turned out to be deemed useless because the later
tool I provide can do everything it can and more, haven't we just
wasted time in providing it?  And perhaps also wasted future time as
we then have work to do to deprecate and remove the new command or
mode? (NOTE: I did *not* say there was "no need" for a single-commit
replaying primitive -- I said there "might not" be a need.)

Also, since you bring up --rebase-merges, there's an additional point
about it that might be relevant:

5. While you could implement a naive --rebase-merges in terms of a
primitive for merging divergent branches (or vice-versa, i.e.
implement merging divergent branches from a naive --rebase-merges
implementation), I think replaying merges more intelligently[*] is
actually a distinct operation from doing a new merge of divergent
branches and that you probably can't implement one in terms of the
other.  (I'm not certain on this, and definitely don't want to argue
the finer points on it while my implementation is still half-baked,
but I really do think they are different things right now.)

[*] https://lore.kernel.org/git/CABPp-BHp+d62dCyAaJfh1cZ8xVpGyb97mZryd02aCOX=Qn=Ltw@mail.gmail.com/

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Johannes Schindelin]

Hi Junio,

On Mon, 21 Feb 2022, Junio C Hamano wrote:

> Elijah Newren <newren@gmail.com> writes:
>
> > Adding such an ability to merge-tree would be trivial -- it basically
> > involves just two things: (1) accepting one extra argument, and (2)
> > calling merge_incore_nonrecursive() instead of
> > merge_incore_recursive().
> >
> > However, I think forking a subprocess for every merge of a series of
> > commits is a completely unreasonable overhead, so even if we provide
> > such an option to merge-tree, I still want a separate plumbing-ish
> > tool that does non-worktree/non-index replaying of commits which is
> > not written as a driver of merge-tree.  That other tool should just
> > call merge_incore_nonrecursive() directly.  And such a tool, since it
> > should handle an arbitrary number of commits, should certainly be able
> > to handle just one commit.  From that angle, it feels like adding
> > another mode to merge-tree would just be a partial duplication of the
> > other tool.
>
> The above does not make much sense to me.
>
> I am hearing that "multi-step cherry-picks and reverts need to be
> fast and we need something like sequencer that is all written in C,
> and single-step cherry-pick is merely a special case that does not
> deserve a plumbing".

Correct. The single cherry-pick will be a trivial fall-out of such a
sequencer, and the opposite is not true: if we taught `merge-tree` the
options `--cherry-pick` or `--revert`, the result would be a dead end
because it does not make sense to extend `merge-tree` to do what the
`sequencer` would do.

> But that argument leads to "and the same something-like-sequencer
> that is all written in C would need '--rebase-merges' that can pick
> multi-step merge sequences, and single-step merge does not deserve a
> plumbing", which is an argument against this topic that is utterly
> absurd.

But that `--rebase-merges`-like behavior is far off in the future, and the
sequencer is not.

If you step back for a moment and think about the existing use cases where
we want to use `merge-tree` on the server side, there are GitHub's Pull
Requests (and I suspect that all other Git hosting services followed
suite), where you have three options:

- merge
- squash
- rebase

The first two options are actually pretty much done, as we already have
a way with the current iteration of `merge-tree` to get the tree, and
that's all we need from `merge-tree`, the rest can be done by calling
`commit-tree` with the appropriate parent(s) and commit messages.

In contrast, `rebase` will require not only `tree` objects to be
generated, but much more. It is a fundamentally more complex operation
because of that.

Now, if there were already server-side user interfaces to cherry-pick, I
could potentially see that the next logical step would be to support
something like the `--force-non-recursive-base=<tree-ish>` option I
have already implemented over here (for separate reasons).

But I am unaware of such a user interface. I _am_ aware of the `rebase`
option to apply Pull Requests. So I think that's the logical direction
we're going from here.

> So why isn't your objection not equally absurd against having a
> single step cherry-pick or revert primitive as a plumbing?

Well, if you care so deeply about it, I will offer up that patch to
support `--force-non-recursive-base=<tree-ish>` (where the `cherry-pick`
and `revert` primitive would not need to exist but be the special case of
passing `CHERRY_PICK_HEAD^` as the appropriate argument).

What gets me excited much more, though, is the `rebase` operation. And
therefore I would like to spend more focus on that, and focus is a limited
resource (especially here on the Git mailing list :-)).

Ciao,
Dscho

Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1671 bytes --]

Hi Elijah,

On Mon, 21 Feb 2022, Elijah Newren wrote:

> On Mon, Feb 21, 2022 at 1:13 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Hi Elijah,
> >
> > On Thu, 3 Feb 2022, Elijah Newren wrote:
> >
> > > On Thu, Feb 3, 2022 at 8:24 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> > > >
> > > > On Thu, Feb 03 2022, Elijah Newren wrote:
> > > >
> > > > > Man, what a can of worms this all is.  Maybe I really should just drop
> > > > > patches 5, 6, and 8 for now...
> > > >
> > > > Yeah, I really think it's worth it to just sprinkle a tiny bit of
> > > > if/else (or a macro) here and print to stderr inline or not. We can make
> > > > some use of some usage.c when there's good reason to do so, but this bit
> > > > just seems like a needless digression.
> > > >
> > > > I hope all of this has helped somewhat ...
> > >
> > > Absolutely; thanks for reviewing!  These parts may just end up in me
> > > dropping some patches for now (since they're not actually being used
> > > anyway), but I think it's all good feedback.
> >
> > So we dropped some useful patches future-proofing `merge-tree` for the
> > sake of appeasing a refactoring with no immediately obvious benefit? I
> > really don't like that direction.
>
> Even before any of Ævar's comments, I had already noted on my cover
> letter[1] that "to be honest, patches 5, 6, & 8 may be less relevant
> since we're now including these messages on stdout anyway" -- so I was
> already wondering if I should defer them to some future series.

Ah, that was not clear to me. In that case, I retract my objections.

Thanks for clarifying,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Mon, 21 Feb 2022, Johannes Schindelin wrote:

> [...] since `merge-tree` is a low-level tool meant to be called by
> programs rather than humans, we need to make sure that those messages
> remain machine-parseable, even if they contain file names.
>
> [...]
>
> Do you think we can switch to `sq_quote_buf_pretty()` for these messages?

Or maybe much better: use NUL to separate those messages if `-z` is passed
to `merge-tree`? That would address the issue in one elegant diff.

What do you think?

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Mon, Feb 21, 2022 at 2:46 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Fri, 4 Feb 2022, Elijah Newren wrote:
>
> > On Fri, Feb 4, 2022 at 3:10 PM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > On Sat, 29 Jan 2022, Elijah Newren wrote:
[...]
> > I've thought about this problem long and hard before (in part because
> > of some conversations I had with Edward Thompson about libgit2 and
>
> Not a big deal for _me_, but I seem to remember that Ed cared a lot about
> having no p in their surname ;-)

Eek!  My apologies to Ed; I'll try to remember and do better.

> > merging at Git Merge 2020).  It wasn't at all clear to me that libgit2
> > had considered anything beyond simple rename cases.  The only rules I
> > ever figured out that made sense to me was "group the stages by target
> > filename rather than by logical conflict" (so we get `ls -files -u`
> > populated) and print a meant-for-human message for each logical
> > conflict (found in the <Informational Messages> section for
> > merge-tree), and make NO attempt to connect stages by conflict type.
> >
> > I'm sure that's not what you wanted to hear, and maybe doesn't even
> > play nicely with your design.  But short of ignoring the edge and
> > corner cases, I don't see how to solve that problem.  If you do just
> > want to ignore edge and corner cases, then just ignore the
> > rename/rename case you brought up in the first place and just use
> > `ls-files -u`-type output as-is within your design.  If you don't want
> > to ignore edge cases and want something that works with a specific
> > design that somehow groups conflicted file stages by conflict type,
> > then we're going to have to dig into all these questions above and do
> > some big replumbing within merge-ort.
>
> There is sometimes a big difference between what I want to hear and what I
> need to hear. Thank you for providing so many details that I needed to
> hear.
>
> So let's take a step back and look at my goal here, as in: the
> over-arching goal: to use merge-ort on the server side.
>
> From what you said above, it becomes very clear to me that there is very
> little chance to resolve such conflicts on the server side.

No, it's still resolvable on the server side.  It's just that attempts
to break up the information into individual logical conflicts is
problematic; if you provide _all_ the informational conflict messages
to the user, and iterate through the paths with conflicts, then things
are fine.  It's only when you attempt to find "the relevant subset"
that things become hard.  Of course, providing it all at once might be
a UI that you hate (perhaps because it's too much like how the command
line behaves...)

> For example, if a topic branch renames a file differently than the main
> branch, there is a really good chance that the user tasked with merging
> the topic branch will have to do a whole lot more than just click a few
> buttons to perform that task. There might very well be the need to edit
> files that do not contain merge conflict markers (I like to call those
> cases "non-semantic merge conflicts"), and almost certainly local testing
> will be necessary.

Sidenote: Do you lump in binary merge conflicts with "non-semantic
merge conflicts"?  You would by your definition, but I'm not sure it
matches.

I tend to call things either content-based conflicts or path-based
conflicts, where content-based usually means textual-based but also
includes merges of binaries.

> So I guess the best we can do in those complicated cases is to give a
> comprehensive overview of the problems in the web UI, with the note that
> this merge conflict has to be resolved on the local side.
>
> Which brings me to the next concern: since `merge-tree` is a low-level
> tool meant to be called by programs rather than humans, we need to make
> sure that those messages remain machine-parseable, even if they contain
> file names.
>
> Concretely: while I am not currently aware of any web UI that allows to
> resolve simple rename/rename conflicts, it is easily conceivable how to
> implement such a thing. When that happens, we will need to be able to
> teach the server-side code to discern between the cases that can be
> handled in the web UI (trivial merge conflicts, trivial rename/rename
> conflicts) as compared to scenarios where the conflicts are just too
> complex.

Um, I'm really worried about attempting to make the conflict notices
machine parseable.  I don't like that idea at all, and I even tried to
rule that out already with my wording:
"""
In all cases, the
<Informational messages> section has the necessary info, though it is
not designed to be machine parseable.
"""
though maybe I should have been even more explicit.  The restrictions
that those messages be stable is too rigid, I think.  I also think
they're a poor way to communicate information to a higher level tool.
I would much rather us add some kind of additional return data
structures from merge ort and use them if we want extra info.

> Here's an excerpt from t4301:
>
> -- snip --
> Auto-merging greeting
> CONFLICT (content): Merge conflict in greeting
> Auto-merging numbers
> CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
> CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
> -- snap --
>
> This is the complete set of messages provided in the `test conflict
> notices and such` test case.
>
> I immediately notice that every line contains at least one file name.
> Looking at https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1899, it
> does not seem as if the file names are quoted:
>
>                 path_msg(opt, path, 1, _("Auto-merging %s"), path);
>
> (where `path` is used verbatim in a call to `merge_3way()` before that,
> i.e. it must not have been quoted)
>
> I would like to register a wish to ensure that file names with special
> characters (such as most notably line-feed characters) are quoted in these
> messages, so that a simple server-side parser can handle messages starting
> with `Auto-merging` and with `CONFLICT (content): Merge conflict in `, and
> "throw the hands up in the air" if any other message prefix is seen.
>
> Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
> For the `Auto-merging` one, it would be trivial, but I fear that we will
> have to work a bit on the `path_msg()` function
> (https://github.com/git/git/blob/v2.35.1/merge-ort.c#L630-L649) because it
> accepts a variable list of arguments without any clue whether the
> arguments refer to paths or not. (And I would be loathe to switch _all_
> callers to do the quoting themselves.)
>
> I see 28 calls to that function, and at least a couple that pass not only
> a path but also an OID (e.g.
> https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1611-L1613).
>
> We could of course be sloppy and pass even OIDs through
> `sq_quote_buf_pretty()` in `path_msg()`, knowing that there won't be any
> special characters in them, but it gets more complicated e.g. in
> https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1648-L1651, where we
> pass an `strbuf` that contains a somewhat free-form commit message.
>
> I guess we could still pass those through `sq_quote_buf_pretty()`, even if
> they are not paths, to ensure that there are no special characters in the
> machine-parseable lines.
>
> What do you think?

Switching to single quoting paths as a matter of style might make
sense, but only if we go through and change every caller to do so so
that we can make sure it applies to all paths.  And only paths and not
OIDs.

But I'm going to reserve the right in merge-ort to modify, add, or
delete any of those messages passed to path_msg(), which might wreak
havoc on your attempts to parse those strings.  I think they're a bad
form for communicating information to a script or program, and trying
to transform them into such risks making them suboptimal at
communicating info to humans.  These messages should optimize the
latter, and if we want something for the former, it should probably be
a new independent bit of info.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Tue, Feb 22, 2022 at 8:54 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Mon, 21 Feb 2022, Johannes Schindelin wrote:
>
> > [...] since `merge-tree` is a low-level tool meant to be called by
> > programs rather than humans, we need to make sure that those messages
> > remain machine-parseable, even if they contain file names.
> >
> > [...]
> >
> > Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
>
> Or maybe much better: use NUL to separate those messages if `-z` is passed
> to `merge-tree`? That would address the issue in one elegant diff.
>
> What do you think?

Separating the combination of messages for a single target path from
the combination of messages for a different target path by a NUL
character may make sense.  Would we also want the messages for a path
to be prepended by the pathname and a NUL character, in this case, to
make it easier to determine which path the group of messages are for?

I'm not sure if that does exactly what you are asking, though.

The thing that is stored (in opt->priv->output) is a strbuf per path,
not an array of strbufs per path.  So, if we have a rename/delete and
a rename/add and a mode conflict for the same "path" (A->B on one
side, other side deletes A and adds a symlink B, resulting in three
messages for path "B" that are all appended into a single strbuf),
then we'll have a single "message" which has three newlines.  We can
add a NUL character at the end of that, but not between the messages
without restructuring things a bit.

There's also at least one example, with submodules, where there are
two path_msg() calls for the same individual conflict in order to
split conflict info from resolution advice, and those really shouldn't
be thought of as messages for different conflicts.  (I'm starting to
wonder if the resolution advice should just be tossed; I kept it
because merge-recursive had it, but it might not make sense with
merge-ort being used by server side merges.  But even if we toss that
one, I'm not sure I want to commit to one path_msg() call per "logical
conflict".)

But...maybe this would be good enough for some kind of use you have?
Because if you only want to care about "simple" cases, you could
potentially define those as ones with only one newline  in them.

Re: machine-parsable git-merge-tree messages (was: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function)

[Elijah Newren]

On Mon, Feb 21, 2022 at 6:37 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Mon, Feb 21 2022, Johannes Schindelin wrote:
>
> [I sent out an empty reply to this earlier by mistake, sorry about that]
>
> > [...]
> > Which brings me to the next concern: since `merge-tree` is a low-level
> > tool meant to be called by programs rather than humans, we need to make
> > sure that those messages remain machine-parseable, even if they contain
> > file names.
> >
> > Concretely: while I am not currently aware of any web UI that allows to
> > resolve simple rename/rename conflicts, it is easily conceivable how to
> > implement such a thing. When that happens, we will need to be able to
> > teach the server-side code to discern between the cases that can be
> > handled in the web UI (trivial merge conflicts, trivial rename/rename
> > conflicts) as compared to scenarios where the conflicts are just too
> > complex.
> >
> > Here's an excerpt from t4301:
> >
> > -- snip --
> > Auto-merging greeting
> > CONFLICT (content): Merge conflict in greeting
> > Auto-merging numbers
> > CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
> > CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
> > -- snap --
> >
> > This is the complete set of messages provided in the `test conflict
> > notices and such` test case.
> >
> > I immediately notice that every line contains at least one file name.
> > Looking at https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1899, it
> > does not seem as if the file names are quoted:
> >
> >               path_msg(opt, path, 1, _("Auto-merging %s"), path);
> >
> > (where `path` is used verbatim in a call to `merge_3way()` before that,
> > i.e. it must not have been quoted)
> >
> > I would like to register a wish to ensure that file names with special
> > characters (such as most notably line-feed characters) are quoted in these
> > messages, so that a simple server-side parser can handle messages starting
> > with `Auto-merging` and with `CONFLICT (content): Merge conflict in `, and
> > "throw the hands up in the air" if any other message prefix is seen.
> >
> > Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
> > For the `Auto-merging` one, it would be trivial, but I fear that we will
> > have to work a bit on the `path_msg()` function
> > (https://github.com/git/git/blob/v2.35.1/merge-ort.c#L630-L649) because it
> > accepts a variable list of arguments without any clue whether the
> > arguments refer to paths or not. (And I would be loathe to switch _all_
> > callers to do the quoting themselves.)
> >
> > I see 28 calls to that function, and at least a couple that pass not only
> > a path but also an OID (e.g.
> > https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1611-L1613).
> >
> > We could of course be sloppy and pass even OIDs through
> > `sq_quote_buf_pretty()` in `path_msg()`, knowing that there won't be any
> > special characters in them, but it gets more complicated e.g. in
> > https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1648-L1651, where we
> > pass an `strbuf` that contains a somewhat free-form commit message.
> >
> > I guess we could still pass those through `sq_quote_buf_pretty()`, even if
> > they are not paths, to ensure that there are no special characters in the
> > machine-parseable lines.
> >
> > What do you think?
>
> That sounds like a rather nasty hack, this is too, but demonstrates that
> we can pretty easily extract this in a machine-readable format with just
> a few lines now):
>
> diff --git a/merge-ort.c b/merge-ort.c
> index 8a5f201d190..a906881f9b3 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -633,7 +633,7 @@ static void path_msg(struct merge_options *opt,
>                      int omittable_hint, /* skippable under --remerge-diff */
>                      const char *fmt, ...)
>  {
> -       va_list ap;
> +       va_list ap, cp;
>         struct strbuf *sb, *dest;
>         struct strbuf tmp = STRBUF_INIT;
>
> @@ -650,7 +650,9 @@ static void path_msg(struct merge_options *opt,
>
>         dest = (opt->record_conflict_msgs_as_headers ? &tmp : sb);
>
> +       va_copy(cp, ap);
>         va_start(ap, fmt);
> +
>         if (opt->priv->call_depth) {
>                 strbuf_addchars(dest, ' ', 2);
>                 strbuf_addstr(dest, "From inner merge:");
> @@ -659,6 +661,15 @@ static void path_msg(struct merge_options *opt,
>         strbuf_vaddf(dest, fmt, ap);
>         va_end(ap);
>
> +       va_start(cp, fmt);
> +       trace2_region_enter_printf("merge", "conflict/path", opt->repo, "%s", path);
> +       trace2_region_leave("merge", "conflict/path", opt->repo);
> +       trace2_region_enter_printf("merge", "conflict/fmt", opt->repo, "%s", fmt);
> +       trace2_region_leave("merge", "conflict/fmt", opt->repo);
> +       trace2_region_enter_printf_va("merge", "conflict/msg", opt->repo, fmt, cp);
> +       trace2_region_leave("merge", "conflict/msg", opt->repo);
> +       va_end(cp);
> +
>         if (opt->record_conflict_msgs_as_headers) {
>                 int i_sb = 0, i_tmp = 0;
>
> You can run that with one of the tests added in this series to get the
> output as JSON, e.g.:
>
>      GIT_TRACE2_EVENT=/dev/stderr GIT_TRACE2_EVENT_NESTING=10 ~/g/git/git merge-tree --write-tree --no-messages --name-only --messages side1 side2 2>&1|jq -r .| grep '"msg"'
>       "msg": "whatever~side1"
>       "msg": "CONFLICT (file/directory): directory in the way of %s from %s; moving it to %s instead."
>       "msg": "CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead."
>       "msg": "whatever~side1"
>       "msg": "CONFLICT (modify/delete): %s deleted in %s and modified in %s.  Version %s of %s left in tree."
>       "msg": "CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree."
>       "msg": "numbers"
>       "msg": "Auto-merging %s"
>       "msg": "Auto-merging numbers"
>       "msg": "greeting"
>       "msg": "Auto-merging %s"
>       "msg": "Auto-merging greeting"
>       "msg": "greeting"
>       "msg": "CONFLICT (%s): Merge conflict in %s"
>       "msg": "CONFLICT (content): Merge conflict in greeting"
>
> A "proper" fix for this doesn't sound too hard, we'd just instrument the
> path_msg() function to pass along some "message category", see
> e.g. unpack_plumbing_errors in unpack-trees.c for one example of such a
> thing, or the "enum fsck_msg_id".
>
> Then we'd just allow you to emit any of the sprintf() format itself, or
> the expanded version, the path, or an id like "CONFLICT:file/directory"
> or "auto-merging" etc.

I don't see how this helps solve the problem Dscho was bringing up at
all.  Your reference to "the path" means you've missed his whole
complaint -- that with more complex conflicts (renames, directory/file
conflicts resolved via moving the file out of the way, mode conflicts
resolved by moving both files out of the way, etc) there are multiple
paths involved and he's trying to determine what those paths are.
He's particularly focusing on rename/rename cases where a single path
was renamed differently by the two sides of history (which results in
a conflict message only being associated with the path from the merge
base in order to avoid repeating the same message 2-3 times, but that
one message has three distinct paths embedded in the string).

Also, the additional paths is not part of the API to path_msg; it's
merely embedded in a string.  (And, in case it bears repeating: as
mentioned elsewhere, we cannot assume there will only be one
path_msg() call per path, and we at least currently can't assume that
each path_msg() call is for a separate logical conflict; there might
be two for a single "conflict".)

I agree that parsing these meant-for-human-consumption (and not
promised to be stable) messages is not a good way to go, but
pretending the current API has enough info to answer his questions
isn't right either.

> I think that would be particularly useful in conjuction with the
> --format changes I was proposing for this (and hacked up a WIP patch
> for). You could just have a similar --format-messages or whatever.
>
> Then you could pick \0\0 as a delimiter for your "main" --format, and
> "\0" as the delimiter for your --format-messages, and thus be able to
> parse N-nested \0-delimited content.

To be honest, the --format stuff is sounding a little bit like a
solution in search of a problem.

[PATCH v6 00/12] In-core git merge-tree ("Server side merges")

[Elijah Newren via GitGitGadget]

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs. It could
be used to do merges when in a bare repository (thus potentially making it
of interest to Git hosting sites, i.e. "Server side merges"), or for doing a
merge of branches that aren't checked out.

It does not handle similar functionality for cherry-picks, rebases, or
reverts; that is also of interest, but is being deferred for a future
series.

== Quick Overview ==

 * Patches 1-2: preparatory cleanups
 * Patches 3-4: implement basic real merges
 * Patches 5-6: include informational messages ("CONFLICT" messages and
   such) in output
 * Patches 7-10: add ability to include ls-files -u style of info in the
   output
 * Patch 11: support --allow-unrelated-histories
 * Patch 12: augment the manual with potential usage mistakes

== Updates Log ==

Stuff NOT included that reviewers brought up in various rounds (and which
might still be an open question):

 * Having -z affect the informational messages section[1]
 * Having a machine-parseable variant of information from the informational
   messages section[2]
 * Very generic (mode, oid, stage, filename) printing formatting[3]
 * Providing similar functionality for doing cherry-picks/rebases/reverts,
   i.e. a scheme for three-way merges with a specified merge-base[4]. That's
   being deferred to a future series. [1]
   https://lore.kernel.org/git/CABPp-BG7id0GfpDee_7ETZ_94BC_i-e_=-u=PrYJeD7d4sVbiw@mail.gmail.com/
   [2]
   https://lore.kernel.org/git/nycvar.QRO.7.76.6.2202211059430.26495@tvgsbejvaqbjf.bet/,
   https://lore.kernel.org/git/220221.86y224b80b.gmgdl@evledraar.gmail.com/
   [3]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [4]
   https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/

Updates since v5:

 * Used reverse_commit_list() to reverse a commit_list without extra
   allocations
 * Several documentation updates

Updates since v4:

 * Fixed double "is" in documentation.
 * Fixed a few small items with testcases

Updates since v3:

 * Dropped previous patches 5, 6, and 8 of the old series; they weren't
   being used and opened a can of worms[1]
 * [Patch 3] Restructured argument checking, including using an enum
 * [Patch 4] Restored the extended paragraph about the deprecated form of
   git-merge-tree, mentioned write-tree in plumbing commands, and a few
   other small fixups to the documentation
 * [Patch 4] Also provide an example of a clean merge rather than just a
   conflicted one
 * [Patch 6] Fix the incompatible arguments check and add some tests for it
 * [Patch 6] Introduce an anonymize_hash() shell function to make tests
   easier to read (less repeated sed)
 * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
   option for now
 * [Patch 10] When -z passed, the tree in the first section should have a
   trailing NUL rather than trailing newline [1]
   https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/

Updates since v2:

 * Improved patches from Dscho for the diff_warn_rename_limit() handling
 * Add a -z option for NUL-terminated conflict info lines (so that filenames
   do not have to be quoted)

Updates since v1 (or v3 depending on how you count; thanks to René, Ævar,
Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

[There were also two submissions of a previous series; see
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/]
[https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/%5D]

Updates since original submission v2 (thanks to Christian, Dscho, Ramsay,
and René for suggestions and comments):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since original submission v1 (thanks to Johannes Altmanninger and
Fabian for suggestions):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

 Documentation/git-merge-tree.txt | 228 +++++++++++++++++++++++++++--
 builtin/merge-tree.c             | 186 ++++++++++++++++++++++--
 git.c                            |   2 +-
 merge-ort.c                      | 109 +++++++++-----
 merge-ort.h                      |  29 ++++
 t/t4301-merge-tree-write-tree.sh | 238 +++++++++++++++++++++++++++++++
 6 files changed, 730 insertions(+), 62 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v5:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  60253745f5c =  3:  60253745f5c merge-tree: add option parsing and initial shell for real merge function
  4:  7994775a934 !  4:  f8266d39c1b merge-tree: implement real merges
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +'git merge-tree' [--write-tree] <branch1> <branch2>
      +'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
     ++[[NEWMERGE]]
       DESCRIPTION
       -----------
      -Reads three tree-ish, and output trivial merge results and
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      -index.  For this reason, the output from the command omits
      -entries that match the <branch1> tree.
      +
     ++This command has a modern `--write-tree` mode and a deprecated
     ++`--trivial-merge` mode.  With the exception of the
     ++<<DEPMERGE,DEPRECATED DESCRIPTION>> section at the end, the rest of
     ++this documentation describes modern `--write-tree` mode.
     ++
      +Performs a merge, but does not make any new commits and does not read
      +from or write to either the working tree or index.
      +
     -+The first form will merge the two branches, doing a real merge.  A real
     -+merge is distinguished from a trivial merge in that it includes:
     ++The performed merge will use the same feature as the "real"
     ++linkgit:git-merge[1], including:
      +
      +  * three way content merges of individual files
      +  * rename detection
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +    merge base, creating a virtual merge base by merging the merge bases)
      +  * etc.
      +
     -+After the merge completes, the first form will create a new toplevel
     -+tree object.  See `OUTPUT` below for details.
     -+
     -+The second form is deprecated; it is kept for backward compatibility
     -+reasons but may be deleted in the future.  Other than the optional
     -+`--trivial-merge`, it accepts no options.  It can only do a trivial
     -+merge.  It reads three tree-ish, and outputs trivial merge results and
     -+conflicting stages to the standard output in a semi-diff format.
     -+Since this was designed for higher level scripts to consume and merge
     -+the results back into the index, it omits entries that match
     -+<branch1>.  The result of this second form is similar to what
     -+three-way 'git read-tree -m' does, but instead of storing the results
     -+in the index, the command outputs the entries to the standard output.
     -+This form not only has limited applicability, the output format is
     -+also difficult to work with, and it will generally be less performant
     -+than the first form even on successful merges (especially if working
     -+in large repositories).  The remainder of this manual will only
     -+discuss the first form.
     ++After the merge completes, a new toplevel tree object is created.  See
     ++`OUTPUT` below for details.
      +
     ++[[OUTPUT]]
      +OUTPUT
      +------
      +
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +USAGE NOTES
      +-----------
      +
     -+git-merge-tree was written to be low-level plumbing, similar to
     -+hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
     -+Thus, it could be used as a part of a series of steps such as
     ++This command is intended as low-level plumbing, similar to
     ++linkgit:git-hash-object[1], linkgit:git-mktree[1],
     ++linkgit:git-commit-tree[1], linkgit:git-write-tree[1],
     ++linkgit:git-update-ref[1], and linkgit:git-mktag[1].  Thus, it can be
     ++used as a part of a series of steps such as:
      +
      +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
      +       test $? -eq 0 || die "There were conflicts..."
      +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
      +       git update-ref $BRANCH1 $NEWCOMMIT
     ++
     ++[[DEPMERGE]]
     ++DEPRECATED DESCRIPTION
     ++----------------------
     ++
     ++Per the <<NEWMERGE,DESCRIPTION>> and unlike the rest of this
     ++documentation, this section describes the deprecated `--trivial-merge`
     ++mode.
     ++
     ++Other than the optional `--trivial-merge`, this mode accepts no
     ++options.
     ++
     ++This mode reads three tree-ish, and outputs trivial merge results and
     ++conflicting stages to the standard output in a semi-diff format.
     ++Since this was designed for higher level scripts to consume and merge
     ++the results back into the index, it omits entries that match
     ++<branch1>.  The result of this second form is similar to what
     ++three-way 'git read-tree -m' does, but instead of storing the results
     ++in the index, the command outputs the entries to the standard output.
     ++
     ++This form not only has limited applicability (a trivial merge cannot
     ++handle content merges of individual files, rename detection, proper
     ++directory/file conflict handling, etc.), the output format is also
     ++difficult to work with, and it will generally be less performant than
     ++the first form even on successful merges (especially if working in
     ++large repositories).
       
       GIT
       ---
     @@ builtin/merge-tree.c: struct merge_tree_options {
       {
      -	die(_("real merges are not yet implemented"));
      +	struct commit *parent1, *parent2;
     -+	struct commit_list *common;
      +	struct commit_list *merge_bases = NULL;
     -+	struct commit_list *j;
      +	struct merge_options opt;
      +	struct merge_result result = { 0 };
      +
     @@ builtin/merge-tree.c: struct merge_tree_options {
      +	 * Get the merge bases, in reverse order; see comment above
      +	 * merge_incore_recursive in merge-ort.h
      +	 */
     -+	common = get_merge_bases(parent1, parent2);
     -+	if (!common)
     ++	merge_bases = get_merge_bases(parent1, parent2);
     ++	if (!merge_bases)
      +		die(_("refusing to merge unrelated histories"));
     -+	for (j = common; j; j = j->next)
     -+		commit_list_insert(j->item, &merge_bases);
     ++	merge_bases = reverse_commit_list(merge_bases);
      +
      +	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
      +	if (result.clean < 0)
  5:  e0f95e094cf =  5:  6629af14919 merge-ort: split out a separate display_update_messages() function
  6:  90c4adecb23 !  6:  17b57efb714 merge-tree: support including merge messages in output
     @@ Documentation/git-merge-tree.txt: git-merge-tree - Perform merge without touchin
      +'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
       'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
     - DESCRIPTION
     -@@ Documentation/git-merge-tree.txt: than the first form even on successful merges (especially if working
     - in large repositories).  The remainder of this manual will only
     - discuss the first form.
     + [[NEWMERGE]]
     +@@ Documentation/git-merge-tree.txt: linkgit:git-merge[1], including:
     + After the merge completes, a new toplevel tree object is created.  See
     + `OUTPUT` below for details.
       
      +OPTIONS
      +-------
     @@ Documentation/git-merge-tree.txt: than the first form even on successful merges
      +	default is to include these messages if there are merge
      +	conflicts, and to omit them otherwise.
      +
     + [[OUTPUT]]
       OUTPUT
       ------
       
      -For either a successful or conflicted merge, the output from
      -git-merge-tree is simply one line:
     -+By default, for a successful merge, the output from git-merge-tree is
     -+simply one line:
     ++For a successful merge, the output from git-merge-tree is simply one
     ++line:
      +
      +	<OID of toplevel tree>
      +
     @@ Documentation/git-merge-tree.txt: than the first form even on successful merges
      -The printed tree object corresponds to what would be checked out in
      -the working tree at the end of `git merge`, and thus may have files
      -with conflict markers in them.
     ++[[OIDTLT]]
      +OID of toplevel tree
      +~~~~~~~~~~~~~~~~~~~~
      +
     @@ Documentation/git-merge-tree.txt: than the first form even on successful merges
      +working tree at the end of `git merge`.  If there were conflicts, then
      +files within this tree may have embedded conflict markers.
      +
     ++[[IM]]
      +Informational messages
      +~~~~~~~~~~~~~~~~~~~~~~
      +
     @@ Documentation/git-merge-tree.txt: than the first form even on successful merges
       
       EXIT STATUS
       -----------
     -@@ Documentation/git-merge-tree.txt: Thus, it could be used as a part of a series of steps such as
     +@@ Documentation/git-merge-tree.txt: used as a part of a series of steps such as:
              NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
              git update-ref $BRANCH1 $NEWCOMMIT
       
     -+Note that when the exit status is non-zero, NEWTREE in this sequence
     ++Note that when the exit status is non-zero, `NEWTREE` in this sequence
      +will contain a lot more output than just a tree.
      +
     - GIT
     - ---
     - Part of the linkgit:git[1] suite
     + [[DEPMERGE]]
     + DEPRECATED DESCRIPTION
     + ----------------------
      
       ## builtin/merge-tree.c ##
      @@ builtin/merge-tree.c: enum mode {
  7:  12e2351092a =  7:  4c8f42372dd merge-ort: provide a merge_get_conflicted_files() helper function
  8:  5bb7d3725ad !  8:  7b1ee417f3d merge-tree: provide a list of which files have conflicts
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: simply one line:
     +@@ Documentation/git-merge-tree.txt: line:
       Whereas for a conflicted merge, the output is by default of the form:
       
       	<OID of toplevel tree>
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
       working tree at the end of `git merge`.  If there were conflicts, then
       files within this tree may have embedded conflict markers.
       
     ++[[CFI]]
      +Conflicted file list
      +~~~~~~~~~~~~~~~~~~~~
      +
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
      +as explained for the configuration variable `core.quotePath` (see
      +linkgit:git-config[1]).
      +
     + [[IM]]
       Informational messages
       ~~~~~~~~~~~~~~~~~~~~~~
     - 
      
       ## builtin/merge-tree.c ##
      @@
     @@ builtin/merge-tree.c: struct merge_tree_options {
      +		      const char *prefix)
       {
       	struct commit *parent1, *parent2;
     - 	struct commit_list *common;
     + 	struct commit_list *merge_bases = NULL;
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		o->show_messages = !result.clean;
       
  9:  3c2ca198cec !  9:  f1231a8fbc8 merge-tree: provide easy access to `ls-files -u` style info
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: discuss the first form.
     +@@ Documentation/git-merge-tree.txt: After the merge completes, a new toplevel tree object is created.  See
       OPTIONS
       -------
       
     @@ Documentation/git-merge-tree.txt: discuss the first form.
       --[no-]messages::
       	Write any informational messages such as "Auto-merging <path>"
       	or CONFLICT notices to the end of stdout.  If unspecified, the
     -@@ Documentation/git-merge-tree.txt: simply one line:
     +@@ Documentation/git-merge-tree.txt: line:
       Whereas for a conflicted merge, the output is by default of the form:
       
       	<OID of toplevel tree>
     @@ Documentation/git-merge-tree.txt: simply one line:
       	<Informational messages>
       
       These are discussed individually below.
     -@@ Documentation/git-merge-tree.txt: This is a tree object that represents what would be checked out in the
     - working tree at the end of `git merge`.  If there were conflicts, then
     +@@ Documentation/git-merge-tree.txt: working tree at the end of `git merge`.  If there were conflicts, then
       files within this tree may have embedded conflict markers.
       
     + [[CFI]]
      -Conflicted file list
      +Conflicted file info
       ~~~~~~~~~~~~~~~~~~~~
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
      +the `--name-only` option is passed, the mode, object, and stage will
      +be omitted.
       
     + [[IM]]
       Informational messages
       ~~~~~~~~~~~~~~~~~~~~~~
       
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
       
         * "Auto-merging <file>"
         * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
     -@@ Documentation/git-merge-tree.txt: Thus, it could be used as a part of a series of steps such as
     - Note that when the exit status is non-zero, NEWTREE in this sequence
     +@@ Documentation/git-merge-tree.txt: used as a part of a series of steps such as:
     + Note that when the exit status is non-zero, `NEWTREE` in this sequence
       will contain a lot more output than just a tree.
       
     -+git-merge-tree was written to provide users with the same information
     -+that they'd have access to if using `git merge`:
     -+  * what would be written to the working tree (the <OID of toplevel tree>)
     ++For conflicts, the output includes the same information that you'd get
     ++with linkgit:git-merge[1]:
     ++
     ++  * what would be written to the working tree (the
     ++    <<OIDTLT,OID of toplevel tree>>)
      +  * the higher order stages that would be written to the index (the
     -+    <Conflicted file info>)
     -+  * any messages that would have been printed to stdout (the <Informational
     -+    messages>)
     ++    <<CFI,Conflicted file info>>)
     ++  * any messages that would have been printed to stdout (the
     ++    <<IM,Informational messages>>)
      +
     - GIT
     - ---
     - Part of the linkgit:git[1] suite
     + [[DEPMERGE]]
     + DEPRECATED DESCRIPTION
     + ----------------------
      
       ## builtin/merge-tree.c ##
      @@ builtin/merge-tree.c: enum mode {
 10:  6e89e17693a ! 10:  22297e6ce75 merge-tree: allow `ls-files -u` style info to be NUL terminated
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: discuss the first form.
     +@@ Documentation/git-merge-tree.txt: After the merge completes, a new toplevel tree object is created.  See
       OPTIONS
       -------
       
     @@ Documentation/git-merge-tree.txt: discuss the first form.
      +	Do not quote filenames in the <Conflicted file info> section,
      +	and end each filename with a NUL character rather than
      +	newline.  Also begin the messages section with a NUL character
     -+	instead of a newline.  See OUTPUT below for more information.
     ++	instead of a newline.  See <<OUTPUT>> below for more information.
      +
       --name-only::
       	In the Conflicted file info section, instead of writing a list
     @@ Documentation/git-merge-tree.txt: OID of toplevel tree
      +files within this tree may have embedded conflict markers.  This section
      +is always followed by a newline (or NUL if `-z` is passed).
       
     + [[CFI]]
       Conflicted file info
     - ~~~~~~~~~~~~~~~~~~~~
      @@ Documentation/git-merge-tree.txt: This is a sequence of lines with the format
       The filename will be quoted as explained for the configuration
       variable `core.quotePath` (see linkgit:git-config[1]).  However, if
     @@ Documentation/git-merge-tree.txt: This is a sequence of lines with the format
      +be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
      +character instead of a newline character.
       
     + [[IM]]
       Informational messages
       ~~~~~~~~~~~~~~~~~~~~~~
       
 11:  6ddd5ffde9c ! 11:  db73c6dd823 merge-tree: add a --allow-unrelated-histories flag
     @@ Documentation/git-merge-tree.txt: OPTIONS
      +	share no common history.  This flag can be given to override that
      +	check and make the merge proceed anyway.
      +
     + [[OUTPUT]]
       OUTPUT
       ------
     - 
      
       ## builtin/merge-tree.c ##
      @@ builtin/merge-tree.c: enum mode {
     @@ builtin/merge-tree.c: enum mode {
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	 * merge_incore_recursive in merge-ort.h
       	 */
     - 	common = get_merge_bases(parent1, parent2);
     --	if (!common)
     -+	if (!common && !o->allow_unrelated_histories)
     + 	merge_bases = get_merge_bases(parent1, parent2);
     +-	if (!merge_bases)
     ++	if (!merge_bases && !o->allow_unrelated_histories)
       		die(_("refusing to merge unrelated histories"));
     - 	for (j = common; j; j = j->next)
     - 		commit_list_insert(j->item, &merge_bases);
     + 	merge_bases = reverse_commit_list(merge_bases);
     + 
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
       			   &o.name_only,
       			   N_("list filenames without modes/oids/stages"),
 12:  7abf633b638 ! 12:  d58a7c7a9f6 git-merge-tree.txt: add a section on potentional usage mistakes
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: that they'd have access to if using `git merge`:
     -   * any messages that would have been printed to stdout (the <Informational
     -     messages>)
     +@@ Documentation/git-merge-tree.txt: with linkgit:git-merge[1]:
     +   * any messages that would have been printed to stdout (the
     +     <<IM,Informational messages>>)
       
      +MISTAKES TO AVOID
      +-----------------
      +
      +Do NOT look through the resulting toplevel tree to try to find which
     -+files conflict; parse the <Conflicted file info> section instead.  Not
     -+only would parsing an entire tree be horrendously slow in large
     ++files conflict; parse the <<CFI,Conflicted file info>> section instead.
     ++Not only would parsing an entire tree be horrendously slow in large
      +repositories, there are numerous types of conflicts not representable by
      +conflict markers (modify/delete, mode conflict, binary file changed on
      +both sides, file/directory conflicts, various rename conflict
      +permutations, etc.)
      +
     -+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
     -+check the exit status.  A merge can have conflicts without having
     ++Do NOT interpret an empty <<CFI,Conflicted file info>> list as a clean
     ++merge; check the exit status.  A merge can have conflicts without having
      +individual files conflict (there are a few types of directory rename
      +conflicts that fall into this category, and others might also be added
      +in the future).
      +
      +Do NOT attempt to guess or make the user guess the conflict types from
     -+the <Conflicted file info> list.  The information there is insufficient
     -+to do so.  For example: Rename/rename(1to2) conflicts (both sides
     -+renamed the same file differently) will result in three different file
     -+having higher order stages (but each only has one higher order stage),
     -+with no way (short of the <Informational messages> section) to determine
     -+which three files are related.  File/directory conflicts also result in
     -+a file with exactly one higher order stage.
     ++the <<CFI,Conflicted file info>> list.  The information there is
     ++insufficient to do so.  For example: Rename/rename(1to2) conflicts (both
     ++sides renamed the same file differently) will result in three different
     ++file having higher order stages (but each only has one higher order
     ++stage), with no way (short of the <<IM,Informational messages>> section)
     ++to determine which three files are related.  File/directory conflicts
     ++also result in a file with exactly one higher order stage.
      +Possibly-involved-in-directory-rename conflicts (when
      +"merge.directoryRenames" is unset or set to "conflicts") also result in
      +a file with exactly one higher order stage.  In all cases, the
     -+<Informational messages> section has the necessary info, though it is
     -+not designed to be machine parseable.
     ++<<IM,Informational messages>> section has the necessary info, though it
     ++is not designed to be machine parseable.
      +
     -+Do NOT assume all filenames listed in the <Informational messages>
     ++Do NOT assume all filenames listed in the <<IM,Informational messages>>
      +section had conflicts.  Messages can be included for files that have no
      +conflicts, such as "Auto-merging <file>".
      +
     -+AVOID taking the OIDS from the <Conflicted file info> and re-merging
     -+them to present the conflicts to the user.  This will lose information.
     -+Instead, look up the version of the file found within the <OID of
     -+toplevel tree> and show that instead.  In particular, the latter will
     -+have conflict markers annotated with the original branch/commit being
     -+merged and, if renames were involved, the original filename.  While you
     -+could include the original branch/commit in the conflict marker
     -+annotations when re-merging, the original filename is not available from
     -+the <Conflicted file info> and thus you would be losing information that
     -+might help the user resolve the conflict.
     ++AVOID taking the OIDS from the <<CFI,Conflicted file info>> and
     ++re-merging them to present the conflicts to the user.  This will lose
     ++information.  Instead, look up the version of the file found within the
     ++<<OIDTLT,OID of toplevel tree>> and show that instead.  In particular,
     ++the latter will have conflict markers annotated with the original
     ++branch/commit being merged and, if renames were involved, the original
     ++filename.  While you could include the original branch/commit in the
     ++conflict marker annotations when re-merging, the original filename is
     ++not available from the <<CFI,Conflicted file info>> and thus you would
     ++be losing information that might help the user resolve the conflict.
      +
     - GIT
     - ---
     - Part of the linkgit:git[1] suite
     + [[DEPMERGE]]
     + DEPRECATED DESCRIPTION
     + ----------------------

-- 
gitgitgadget

[PATCH v6 01/12] merge-tree: rename merge_trees() to trivial_merge_trees()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget

[PATCH v6 02/12] merge-tree: move logic for existing merge into new function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget

[PATCH v6 03/12] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 84 +++++++++++++++++++++++++++++++++++++++-----
 git.c                |  2 +-
 2 files changed, 76 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..0f9d928e862 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,74 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+enum mode {
+	MODE_UNKNOWN,
+	MODE_TRIVIAL,
+	MODE_REAL,
+};
+
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    MODE_REAL),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	switch (o.mode) {
+	default:
+		BUG("unexpected command mode %d", o.mode);
+	case MODE_UNKNOWN:
+		switch (argc) {
+		default:
+			usage_with_options(merge_tree_usage, mt_options);
+		case 2:
+			o.mode = MODE_REAL;
+			break;
+		case 3:
+			o.mode = MODE_TRIVIAL;
+			break;
+		}
+		expected_remaining_argc = argc;
+		break;
+	case MODE_REAL:
+		expected_remaining_argc = 2;
+		break;
+	case MODE_TRIVIAL:
+		expected_remaining_argc = 3;
+		break;
+	}
+
+	if (argc != expected_remaining_argc)
+		usage_with_options(merge_tree_usage, mt_options);
+
+	/* Do the relevant type of merge */
+	if (o.mode == MODE_REAL)
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget

[PATCH v6 04/12] merge-tree: implement real merges

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability (meaning that NEWTREE would be a lot more than a tree in the
case of conflicts).

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  98 ++++++++++++++++++++++++----
 builtin/merge-tree.c             |  41 +++++++++++-
 t/t4301-merge-tree-write-tree.sh | 106 +++++++++++++++++++++++++++++++
 3 files changed, 232 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..2a9c91328de 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,100 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
+[[NEWMERGE]]
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+This command has a modern `--write-tree` mode and a deprecated
+`--trivial-merge` mode.  With the exception of the
+<<DEPMERGE,DEPRECATED DESCRIPTION>> section at the end, the rest of
+this documentation describes modern `--write-tree` mode.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The performed merge will use the same feature as the "real"
+linkgit:git-merge[1], including:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, a new toplevel tree object is created.  See
+`OUTPUT` below for details.
+
+[[OUTPUT]]
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1 (and the output is unspecified).
+
+USAGE NOTES
+-----------
+
+This command is intended as low-level plumbing, similar to
+linkgit:git-hash-object[1], linkgit:git-mktree[1],
+linkgit:git-commit-tree[1], linkgit:git-write-tree[1],
+linkgit:git-update-ref[1], and linkgit:git-mktag[1].  Thus, it can be
+used as a part of a series of steps such as:
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+[[DEPMERGE]]
+DEPRECATED DESCRIPTION
+----------------------
+
+Per the <<NEWMERGE,DESCRIPTION>> and unlike the rest of this
+documentation, this section describes the deprecated `--trivial-merge`
+mode.
+
+Other than the optional `--trivial-merge`, this mode accepts no
+options.
+
+This mode reads three tree-ish, and outputs trivial merge results and
+conflicting stages to the standard output in a semi-diff format.
+Since this was designed for higher level scripts to consume and merge
+the results back into the index, it omits entries that match
+<branch1>.  The result of this second form is similar to what
+three-way 'git read-tree -m' does, but instead of storing the results
+in the index, the command outputs the entries to the standard output.
+
+This form not only has limited applicability (a trivial merge cannot
+handle content merges of individual files, rename detection, proper
+directory/file conflict handling, etc.), the output format is also
+difficult to work with, and it will generally be less performant than
+the first form even on successful merges (especially if working in
+large repositories).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 0f9d928e862..2332525d8bd 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -398,7 +401,43 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *merge_bases = NULL;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	merge_bases = get_merge_bases(parent1, parent2);
+	if (!merge_bases)
+		die(_("refusing to merge unrelated histories"));
+	merge_bases = reverse_commit_list(merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..6d321652e21
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "$GIT_TEST_MERGE_ALGORITHM" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+	git branch side3 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications &&
+
+	git checkout side3 &&
+	git mv numbers sequence &&
+	test_tick &&
+	git commit -m rename-numbers
+'
+
+test_expect_success 'Clean merge' '
+	TREE_OID=$(git merge-tree --write-tree side1 side3) &&
+	q_to_tab <<-EOF >expect &&
+	100644 blob $(git rev-parse side1:greeting)Qgreeting
+	100644 blob $(git rev-parse side1:numbers)Qsequence
+	100644 blob $(git rev-parse side1:whatever)Qwhatever
+	EOF
+
+	git ls-tree $TREE_OID >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(git rev-parse AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	sed -e s/HEAD/side1/ tmp >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 invalid 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget

[PATCH v6 05/12] merge-ort: split out a separate display_update_messages() function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 78 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 37 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..ebaed98d53a 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4272,43 +4311,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		fclose(fp);
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
-
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v6 06/12] merge-tree: support including merge messages in output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 47 ++++++++++++++++++++++++++++----
 builtin/merge-tree.c             | 21 ++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 37 +++++++++++++++++++++++++
 3 files changed, 97 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 2a9c91328de..25b462be14e 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 [[NEWMERGE]]
@@ -37,18 +37,50 @@ linkgit:git-merge[1], including:
 After the merge completes, a new toplevel tree object is created.  See
 `OUTPUT` below for details.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 [[OUTPUT]]
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+For a successful merge, the output from git-merge-tree is simply one
+line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+[[OIDTLT]]
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+[[IM]]
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
+
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -72,6 +104,9 @@ used as a part of a series of steps such as:
        NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
        git update-ref $BRANCH1 $NEWCOMMIT
 
+Note that when the exit status is non-zero, `NEWTREE` in this sequence
+will contain a lot more output than just a tree.
+
 [[DEPMERGE]]
 DEPRECATED DESCRIPTION
 ----------------------
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 2332525d8bd..831d9c77583 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -396,6 +396,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -435,18 +436,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -456,10 +466,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    MODE_REAL),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc - 1; /* ignoring argv[0] */
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	switch (o.mode) {
@@ -483,8 +496,12 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 		break;
 	case MODE_TRIVIAL:
 		expected_remaining_argc = 3;
+		/* Removal of `--trivial-merge` is expected */
+		original_argc--;
 		break;
 	}
+	if (o.mode == MODE_TRIVIAL && argc < original_argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	if (argc != expected_remaining_argc)
 		usage_with_options(merge_tree_usage, mt_options);
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 6d321652e21..719d81e7173 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -103,4 +103,41 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+anonymize_hash() {
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" "$@"
+}
+
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
+for opt in $(git merge-tree --git-completion-helper-all)
+do
+	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
+	then
+		continue
+	fi
+
+	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
+		test_expect_code 128 git merge-tree --trivial-merge $opt side1 side2 side3
+	'
+done
+
 test_done
-- 
gitgitgadget

[PATCH v6 07/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index ebaed98d53a..e1b647b0a40 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4274,6 +4274,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..ddcc39d7270 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -88,6 +89,26 @@ void merge_switch_to_result(struct merge_options *opt,
 void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v6 08/12] merge-tree: provide a list of which files have conflicts

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  9 +++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 42 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 25b462be14e..68a51c82618 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -58,6 +58,7 @@ line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -70,6 +71,14 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+[[CFI]]
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 [[IM]]
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 831d9c77583..3653526cc0a 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -400,7 +403,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *merge_bases = NULL;
@@ -441,6 +445,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result);
@@ -508,7 +528,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == MODE_REAL)
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 719d81e7173..8e6dba44288 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -117,6 +117,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -140,4 +142,13 @@ do
 	'
 done
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v6 09/12] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide a --name-only option for users to exclude the mode, oid, and
stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 34 ++++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++--
 3 files changed, 62 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 68a51c82618..b89aabdb98e 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -40,6 +40,13 @@ After the merge completes, a new toplevel tree object is created.  See
 OPTIONS
 -------
 
+--name-only::
+	In the Conflicted file info section, instead of writing a list
+	of (mode, oid, stage, path) tuples to output for conflicted
+	files, just provide a list of filenames with conflicts (and
+	do not list filenames multiple times if they have multiple
+	conflicting stages).
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -58,7 +65,7 @@ line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -72,19 +79,24 @@ working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
 [[CFI]]
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--name-only` option is passed, the mode, object, and stage will
+be omitted.
 
 [[IM]]
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -116,6 +128,16 @@ used as a part of a series of steps such as:
 Note that when the exit status is non-zero, `NEWTREE` in this sequence
 will contain a lot more output than just a tree.
 
+For conflicts, the output includes the same information that you'd get
+with linkgit:git-merge[1]:
+
+  * what would be written to the working tree (the
+    <<OIDTLT,OID of toplevel tree>>)
+  * the higher order stages that would be written to the index (the
+    <<CFI,Conflicted file info>>)
+  * any messages that would have been printed to stdout (the
+    <<IM,Informational messages>>)
+
 [[DEPMERGE]]
 DEPRECATED DESCRIPTION
 ----------------------
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 3653526cc0a..4a79caf081a 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -400,6 +400,7 @@ enum mode {
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int name_only;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -453,7 +454,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->name_only)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -488,6 +493,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F(0, "name-only",
+			   &o.name_only,
+			   N_("list filenames without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 8e6dba44288..0ec5f0d3f7e 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -65,6 +65,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(git rev-parse AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -108,7 +109,7 @@ anonymize_hash() {
 }
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	# Expected results:
@@ -143,7 +144,7 @@ do
 done
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -151,4 +152,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v6 10/12] merge-tree: allow `ls-files -u` style info to be NUL terminated

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much as `git ls-files` has a -z option, let's add one to merge-tree so
that the conflict-info section can be NUL terminated (and avoid quoting
of unusual filenames).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 21 +++++++++++++----
 builtin/merge-tree.c             |  6 +++--
 t/t4301-merge-tree-write-tree.sh | 40 ++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index b89aabdb98e..75b57f8abab 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -40,6 +40,12 @@ After the merge completes, a new toplevel tree object is created.  See
 OPTIONS
 -------
 
+-z::
+	Do not quote filenames in the <Conflicted file info> section,
+	and end each filename with a NUL character rather than
+	newline.  Also begin the messages section with a NUL character
+	instead of a newline.  See <<OUTPUT>> below for more information.
+
 --name-only::
 	In the Conflicted file info section, instead of writing a list
 	of (mode, oid, stage, path) tuples to output for conflicted
@@ -76,7 +82,8 @@ OID of toplevel tree
 
 This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
-files within this tree may have embedded conflict markers.
+files within this tree may have embedded conflict markers.  This section
+is always followed by a newline (or NUL if `-z` is passed).
 
 [[CFI]]
 Conflicted file info
@@ -89,20 +96,26 @@ This is a sequence of lines with the format
 The filename will be quoted as explained for the configuration
 variable `core.quotePath` (see linkgit:git-config[1]).  However, if
 the `--name-only` option is passed, the mode, object, and stage will
-be omitted.
+be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
+character instead of a newline character.
 
 [[IM]]
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
-This always starts with a blank line to separate it from the previous
-sections, and then has free-form messages about the merge, such as:
+This always starts with a blank line (or NUL if `-z` is passed) to
+separate it from the previous sections, and then has free-form
+messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
   * "Failed to merge submodule <submodule> (<reason>)"
   * "Warning: cannot merge binary files: <filename>"
 
+Note that these free-form messages will never have a NUL character
+in or between them, even if -z is passed.  It is simply a large block
+of text taking up the remainder of the output.
+
 EXIT STATUS
 -----------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 4a79caf081a..767892006e3 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -445,7 +445,7 @@ static int real_merge(struct merge_tree_options *o,
 	if (o->show_messages == -1)
 		o->show_messages = !result.clean;
 
-	puts(oid_to_hex(&result.tree->object.oid));
+	printf("%s%c", oid_to_hex(&result.tree->object.oid), line_termination);
 	if (!result.clean) {
 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
 		const char *last = NULL;
@@ -467,7 +467,7 @@ static int real_merge(struct merge_tree_options *o,
 		string_list_clear(&conflicted_files, 1);
 	}
 	if (o->show_messages) {
-		printf("\n");
+		putchar(line_termination);
 		merge_display_update_messages(&opt, &result);
 	}
 	merge_finalize(&opt, &result);
@@ -493,6 +493,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_SET_INT('z', NULL, &line_termination,
+			    N_("separate paths with the NUL character"), '\0'),
 		OPT_BOOL_F(0, "name-only",
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 0ec5f0d3f7e..22e03f0939c 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -173,4 +173,44 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'NUL terminated conflicted file "lines"' '
+	git checkout -b tweak1 side1 &&
+	test_write_lines zero 1 2 3 4 5 6 >numbers &&
+	git add numbers &&
+	git mv numbers "Αυτά μου φαίνονται κινέζικα" &&
+	git commit -m "Renamed numbers" &&
+
+	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
+	echo HASH | lf_to_nul >expect &&
+
+	q_to_tab <<-EOF | lf_to_nul >>expect &&
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~tweak1
+	100644 HASH 2Qwhatever~tweak1
+	100644 HASH 1QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 2QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 3QΑυτά μου φαίνονται κινέζικα
+
+	EOF
+
+	cat <<-EOF >>expect &&
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
+	CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
+	Auto-merging Αυτά μου φαίνονται κινέζικα
+	CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v6 11/12] merge-tree: add a --allow-unrelated-histories flag

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 75b57f8abab..628324646d3 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -59,6 +59,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 [[OUTPUT]]
 OUTPUT
 ------
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 767892006e3..b4f5c7e0aab 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -399,6 +399,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int name_only;
 };
@@ -434,7 +435,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	merge_bases = get_merge_bases(parent1, parent2);
-	if (!merge_bases)
+	if (!merge_bases && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	merge_bases = reverse_commit_list(merge_bases);
 
@@ -499,6 +500,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 22e03f0939c..bd1769c624b 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -44,7 +44,13 @@ test_expect_success setup '
 	git checkout side3 &&
 	git mv numbers sequence &&
 	test_tick &&
-	git commit -m rename-numbers
+	git commit -m rename-numbers &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Clean merge' '
@@ -213,4 +219,20 @@ test_expect_success 'NUL terminated conflicted file "lines"' '
 	test_cmp expect actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v6 12/12] git-merge-tree.txt: add a section on potentional usage mistakes

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 628324646d3..ee8125810e6 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -156,6 +156,52 @@ with linkgit:git-merge[1]:
   * any messages that would have been printed to stdout (the
     <<IM,Informational messages>>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <<CFI,Conflicted file info>> section instead.
+Not only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <<CFI,Conflicted file info>> list as a clean
+merge; check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <<CFI,Conflicted file info>> list.  The information there is
+insufficient to do so.  For example: Rename/rename(1to2) conflicts (both
+sides renamed the same file differently) will result in three different
+file having higher order stages (but each only has one higher order
+stage), with no way (short of the <<IM,Informational messages>> section)
+to determine which three files are related.  File/directory conflicts
+also result in a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<<IM,Informational messages>> section has the necessary info, though it
+is not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <<IM,Informational messages>>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <<CFI,Conflicted file info>> and
+re-merging them to present the conflicts to the user.  This will lose
+information.  Instead, look up the version of the file found within the
+<<OIDTLT,OID of toplevel tree>> and show that instead.  In particular,
+the latter will have conflict markers annotated with the original
+branch/commit being merged and, if renames were involved, the original
+filename.  While you could include the original branch/commit in the
+conflict marker annotations when re-merging, the original filename is
+not available from the <<CFI,Conflicted file info>> and thus you would
+be losing information that might help the user resolve the conflict.
+
 [[DEPMERGE]]
 DEPRECATED DESCRIPTION
 ----------------------
-- 
gitgitgadget

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> The objection you are arguing against is not my position.  In fact,
> I'm not even objecting to having a single-step cherry-pick, I'm
> objecting to providing it _now_, which I thought would have been clear
> from the portion of my email you snipped ("...I'm happy to add [a
> single step cherry-pick primitive] along with the tool I submit
> later...").

The entry point into the in-core merge machinery of ort already
knows how to accept externally defined merge-base(s) to bypass the
"caller didn't give us the merge base, so let's figure them out by
using the two heads being merged" logic, so it just felt backwards
*not* to have a vanilla three-way merge that can take three trees
and be used for merge, cherry-pick or revert as the single primitive
in the very beginning before we talk about multi-step operations.

So, I guess I am still not getting where the "I'm happy to _add_"
part comes from.  If we start with a primitive (internally callable
from C) "here are three trees O A B, do the three-way merge", then
there is nothing to "add" at all to expose a single-step
cherry-pick.  In fact, to the users of merge-tree, the result does
not have to have any fixed meaning.  If they pass common ancestors
as the merge bases as Os and the current HEAD and the other branch
as A and B, they get a merge.  If they pass the commit to be picked
as B, current HEAD as A and B's parent as O, they get a cherry-pick.

Perhaps starting from "You are allowed to give me two commits A B,
and I do not let you specify the commit O to use as a common
'ancestor'" is the root cause of making this thing feel backwards.
I agree with the goal of having an all-incore machinery that can do
a merge.  I just do not see the reason why you have to build it in a
way that cannot be reused for other two directions of merge-y
operations.

Re: [PATCH v6 00/12] In-core git merge-tree ("Server side merges")

[Junio C Hamano]

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> == Updates Log ==
>
> Stuff NOT included that reviewers brought up in various rounds (and which
> might still be an open question):

Except for them, the differences since the last round all looked
sensible to me.

Will queue.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Elijah Newren]

On Wed, Feb 23, 2022 at 12:07 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > The objection you are arguing against is not my position.  In fact,
> > I'm not even objecting to having a single-step cherry-pick, I'm
> > objecting to providing it _now_, which I thought would have been clear
> > from the portion of my email you snipped ("...I'm happy to add [a
> > single step cherry-pick primitive] along with the tool I submit
> > later...").
>
> The entry point into the in-core merge machinery of ort already
> knows how to accept externally defined merge-base(s) to bypass the
> "caller didn't give us the merge base, so let's figure them out by
> using the two heads being merged" logic, so it just felt backwards
> *not* to have a vanilla three-way merge that can take three trees
> and be used for merge, cherry-pick or revert as the single primitive
> in the very beginning before we talk about multi-step operations.

"The entry point"?

There are two entry points: merge_incore_recursive(), and
merge_incore_nonrecursive().  The former is analogous to
merge-recursive's merge_recursive(), and the latter is analogous to
merge_trees().

"git merge" always uses merge_incore_recursive()/merge_recursive().
The other big merge-y operations (cherry-pick, rebase, revert), always
use merge_incore_nonrecursive()/merge_trees().

(Technically, merge-recursive has a third entry point used by am &
stash, but let's ignore it for a moment.)

> So, I guess I am still not getting where the "I'm happy to _add_"
> part comes from.  If we start with a primitive (internally callable
> from C) "here are three trees O A B, do the three-way merge", then
> there is nothing to "add" at all to expose a single-step
> cherry-pick.  In fact, to the users of merge-tree, the result does
> not have to have any fixed meaning.  If they pass common ancestors
> as the merge bases as Os and the current HEAD and the other branch
> as A and B, they get a merge.  If they pass the commit to be picked
> as B, current HEAD as A and B's parent as O, they get a cherry-pick.
>
> Perhaps starting from "You are allowed to give me two commits A B,
> and I do not let you specify the commit O to use as a common
> 'ancestor'" is the root cause of making this thing feel backwards.
> I agree with the goal of having an all-incore machinery that can do
> a merge.  I just do not see the reason why you have to build it in a
> way that cannot be reused for other two directions of merge-y
> operations.

So, am I correct to understand that what bugs you is actually
merge-recursive's and merge-ort's API?  That you don't want these two
types of merges to have different entry points, and that there should
in fact only be one?

That might be an interesting line of investigation to try to modify
the API to achieve that, but that feels like a bigger task that is
somewhat tangential to this series.

Without such a thing, handling both merging-of-divergent-branches and
three-way-merge-with-specified-merge-base would be separate codepaths
that call different functions.  I implemented one of those two
codepaths, and if we want the other then it needs to be added.  That's
where the "add" part comes from, in my view.


Does that help, or am I still missing what you're saying?

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> So, am I correct to understand that what bugs you is actually
> merge-recursive's and merge-ort's API?  That you don't want these two
> types of merges to have different entry points, and that there should
> in fact only be one?

It is more like

    It is more than OK that there are two, but the basic primitive
    is the "we have this and that tree objects to merge, and use
    this tree object as the ancestor" non-recursive thing, with the
    recursive one being just a thin wrapper around it to compute
    common ancestors, using the three-way primitive to reduce them
    into a single virtual ancestor, and finally using the three-way
    primitive to come up with the final result.

And making the composite "recursive" feature available long before
the underlying "non-recursive" primitive becomes easily accessible
to the scripters and system builders simply felt backwards.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> It is more like
> ...

Actually, I misspoke.  It is a bit different.

In my mind, the building block hierarchy would have been

 (1) Take three tree objects A, B, and O, and do the three-way
     merge.  No history relation is assumed among them.

 (2) Take two tree objects A and B, with one or more commit objects
     Os; use (2) recursively to reduce Os into a single O and then
     apply (1) on A, B and O.

 (3) Take two commit objects A and B.  Compute Os out of A and B and
     use (2) once to merge A and B.

I think the basic primitive that should be exposed to an external
world (read: plumbing) this year, after all years of experience with
merge-recursive, should be (2), not (1).  

If you have (2), then (3) is trivially possible (it is just a single
call to get_merge_base()).  "git merge-tree A B" without having to
spell out bases is so convenent and you do not have to write
"git merge-tree A B $(git merge-base --all A B)", so I am OK for it
to exist, but it is not essential.

If you have (2) and exposed (2) as the primitive plumbing,
cherry-pick and revert would be a narrow special case of passing
only one O to the machinery.

And coming from the above point of view, exposing (3) as the
primitive plumbing to scripters and system builders, and later
having to _add_ support to allow (2), felt backwards.  It should be
trivial for us to make (2) available before we can even offer (3),
but what is happening to this new plumbing command goes in the
opposite order.

It may be, as you said, the problem the underlying ort API has that
somehow makes it harder to expose (2), in which case, yes, I think
that is what bugs me.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Tue, 22 Feb 2022, Elijah Newren wrote:

> On Tue, Feb 22, 2022 at 8:54 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Mon, 21 Feb 2022, Johannes Schindelin wrote:
> >
> > > [...] since `merge-tree` is a low-level tool meant to be called by
> > > programs rather than humans, we need to make sure that those messages
> > > remain machine-parseable, even if they contain file names.
> > >
> > > [...]
> > >
> > > Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
> >
> > Or maybe much better: use NUL to separate those messages if `-z` is passed
> > to `merge-tree`? That would address the issue in one elegant diff.
> >
> > What do you think?
>
> Separating the combination of messages for a single target path from
> the combination of messages for a different target path by a NUL
> character may make sense.  Would we also want the messages for a path
> to be prepended by the pathname and a NUL character, in this case, to
> make it easier to determine which path the group of messages are for?
>
> I'm not sure if that does exactly what you are asking, though.

The most important thing I am asking for is a way for a program calling
`merge-tree` to figure out whether it knows how to handle all the types of
conflicts encountered in this merge.

So that a web UI could present e.g. simple content conflicts, and even
rename/rename conflicts, but would know that it cannot resolve the
conflicts e.g. when a submodule is affected.

So... I am fairly certain that we're not as close to addressing this as I
had hoped for.

> The thing that is stored (in opt->priv->output) is a strbuf per path,
> not an array of strbufs per path.  So, if we have a rename/delete and
> a rename/add and a mode conflict for the same "path" (A->B on one
> side, other side deletes A and adds a symlink B, resulting in three
> messages for path "B" that are all appended into a single strbuf),
> then we'll have a single "message" which has three newlines.  We can
> add a NUL character at the end of that, but not between the messages
> without restructuring things a bit.
>
> There's also at least one example, with submodules, where there are
> two path_msg() calls for the same individual conflict in order to
> split conflict info from resolution advice, and those really shouldn't
> be thought of as messages for different conflicts.  (I'm starting to
> wonder if the resolution advice should just be tossed; I kept it
> because merge-recursive had it, but it might not make sense with
> merge-ort being used by server side merges.  But even if we toss that
> one, I'm not sure I want to commit to one path_msg() call per "logical
> conflict".)
>
> But...maybe this would be good enough for some kind of use you have?
> Because if you only want to care about "simple" cases, you could
> potentially define those as ones with only one newline  in them.

We cannot rely on newline character parsing because that is a valid
filename character on Unix:

	$ echo a >'with
	> a newline'

	$ ls -la with*
	-rw-r--r-- 1 me me 2 Feb 25 17:10 'with'$'\n''a newline'

I guess we have to work harder on this and add more than just an `strbuf`
so that we can output `<path>NUL<conflict-type>NUL` pairs (where we
promise to keep the `<conflict-type>` strings constant) or something
similar.

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Tue, 22 Feb 2022, Elijah Newren wrote:

> Sidenote: Do you lump in binary merge conflicts with "non-semantic
> merge conflicts"?  You would by your definition, but I'm not sure it
> matches.
>
> I tend to call things either content-based conflicts or path-based
> conflicts, where content-based usually means textual-based but also
> includes merges of binaries.

I like "content-based conflicts".

And no, I had not even thought about binary merge conflicts yet...

> On Mon, Feb 21, 2022 at 2:46 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Concretely: while I am not currently aware of any web UI that allows
> > to resolve simple rename/rename conflicts, it is easily conceivable
> > how to implement such a thing. When that happens, we will need to be
> > able to teach the server-side code to discern between the cases that
> > can be handled in the web UI (trivial merge conflicts, trivial
> > rename/rename conflicts) as compared to scenarios where the conflicts
> > are just too complex.
>
> Um, I'm really worried about attempting to make the conflict notices
> machine parseable.  I don't like that idea at all, and I even tried to
> rule that out already with my wording:
> """
> In all cases, the
> <Informational messages> section has the necessary info, though it is
> not designed to be machine parseable.
> """
> though maybe I should have been even more explicit.  The restrictions
> that those messages be stable is too rigid, I think.  I also think
> they're a poor way to communicate information to a higher level tool.
> I would much rather us add some kind of additional return data
> structures from merge ort and use them if we want extra info.

Okay.

I thought that we could keep the `CONFLICT (<type>)` constant enough to
serve as such a machine-parseable thing. And then presenting
`<path>NUL<message>NUL` could have served my use case well...

> > Here's an excerpt from t4301:
> >
> > -- snip --
> > Auto-merging greeting
> > CONFLICT (content): Merge conflict in greeting
> > Auto-merging numbers
> > CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
> > CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
> > -- snap --
> >
> > This is the complete set of messages provided in the `test conflict
> > notices and such` test case.
> >
> > I immediately notice that every line contains at least one file name.
> > Looking at https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1899, it
> > does not seem as if the file names are quoted:
> >
> >                 path_msg(opt, path, 1, _("Auto-merging %s"), path);
> >
> > (where `path` is used verbatim in a call to `merge_3way()` before that,
> > i.e. it must not have been quoted)
> >
> > I would like to register a wish to ensure that file names with special
> > characters (such as most notably line-feed characters) are quoted in these
> > messages, so that a simple server-side parser can handle messages starting
> > with `Auto-merging` and with `CONFLICT (content): Merge conflict in `, and
> > "throw the hands up in the air" if any other message prefix is seen.
> >
> > Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
> > For the `Auto-merging` one, it would be trivial, but I fear that we will
> > have to work a bit on the `path_msg()` function
> > (https://github.com/git/git/blob/v2.35.1/merge-ort.c#L630-L649) because it
> > accepts a variable list of arguments without any clue whether the
> > arguments refer to paths or not. (And I would be loathe to switch _all_
> > callers to do the quoting themselves.)
> >
> > I see 28 calls to that function, and at least a couple that pass not only
> > a path but also an OID (e.g.
> > https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1611-L1613).
> >
> > We could of course be sloppy and pass even OIDs through
> > `sq_quote_buf_pretty()` in `path_msg()`, knowing that there won't be any
> > special characters in them, but it gets more complicated e.g. in
> > https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1648-L1651, where we
> > pass an `strbuf` that contains a somewhat free-form commit message.
> >
> > I guess we could still pass those through `sq_quote_buf_pretty()`, even if
> > they are not paths, to ensure that there are no special characters in the
> > machine-parseable lines.
> >
> > What do you think?
>
> Switching to single quoting paths as a matter of style might make
> sense, but only if we go through and change every caller to do so so
> that we can make sure it applies to all paths.  And only paths and not
> OIDs.

Yes, that sounds unappealing.

> But I'm going to reserve the right in merge-ort to modify, add, or
> delete any of those messages passed to path_msg(), which might wreak
> havoc on your attempts to parse those strings.  I think they're a bad
> form for communicating information to a script or program, and trying
> to transform them into such risks making them suboptimal at
> communicating info to humans.  These messages should optimize the
> latter, and if we want something for the former, it should probably be
> a new independent bit of info.

Makes sense.

So we need something in addition to those messages.

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I tend to call things either content-based conflicts or path-based
>> conflicts, where content-based usually means textual-based but also
>> includes merges of binaries.
>
> I like "content-based conflicts".

Yup, even before ort existed, we had clear distinction between tree
level merges (i.e. which path corresponds to which other path, which
is done in unpack_trees() and "read-tree O A B") and content level
merges (which is done with ll_merge()).

>> Switching to single quoting paths as a matter of style might make
>> sense, but only if we go through and change every caller to do so so
>> that we can make sure it applies to all paths.  And only paths and not
>> OIDs.
>
> Yes, that sounds unappealing.
>
>> But I'm going to reserve the right in merge-ort to modify, add, or
>> delete any of those messages passed to path_msg(), which might wreak
>> havoc on your attempts to parse those strings.  I think they're a bad
>> form for communicating information to a script or program, and trying
>> to transform them into such risks making them suboptimal at
>> communicating info to humans.  These messages should optimize the
>> latter, and if we want something for the former, it should probably be
>> a new independent bit of info.
>
> Makes sense.

As long as it is made clear that the path_msg() is not for machine
consumption (perhaps we can sprinkle <RED><RESET> at random places
to make it impossible for machines to handle), I think the direction
makes sense, too ;-)

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

Hi Dscho,

On Fri, Feb 25, 2022 at 8:31 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Tue, 22 Feb 2022, Elijah Newren wrote:
>
> > Sidenote: Do you lump in binary merge conflicts with "non-semantic
> > merge conflicts"?  You would by your definition, but I'm not sure it
> > matches.
> >
> > I tend to call things either content-based conflicts or path-based
> > conflicts, where content-based usually means textual-based but also
> > includes merges of binaries.
>
> I like "content-based conflicts".
>
> And no, I had not even thought about binary merge conflicts yet...
>
> > On Mon, Feb 21, 2022 at 2:46 AM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > Concretely: while I am not currently aware of any web UI that allows
> > > to resolve simple rename/rename conflicts, it is easily conceivable
> > > how to implement such a thing. When that happens, we will need to be
> > > able to teach the server-side code to discern between the cases that
> > > can be handled in the web UI (trivial merge conflicts, trivial
> > > rename/rename conflicts) as compared to scenarios where the conflicts
> > > are just too complex.
> >
> > Um, I'm really worried about attempting to make the conflict notices
> > machine parseable.  I don't like that idea at all, and I even tried to
> > rule that out already with my wording:
> > """
> > In all cases, the
> > <Informational messages> section has the necessary info, though it is
> > not designed to be machine parseable.
> > """
> > though maybe I should have been even more explicit.  The restrictions
> > that those messages be stable is too rigid, I think.  I also think
> > they're a poor way to communicate information to a higher level tool.
> > I would much rather us add some kind of additional return data
> > structures from merge ort and use them if we want extra info.
>
> Okay.
>
> I thought that we could keep the `CONFLICT (<type>)` constant enough to
> serve as such a machine-parseable thing.

That _probably_ is, but I thought you wanted to parse all N paths
embedded in the message after that part as well?

> And then presenting
> `<path>NUL<message>NUL` could have served my use case well...

Would it?  Wouldn't you need something more like

<number-of-paths>NUL<path1>NUL<path2>NUL<pathN>NUL<stable-short-type-description>NUL<message>NUL
 ?

I mean, if rename/rename is what you want to handle, there are three
paths in that message.  And you need to know all three paths in order
to combine the relevant parts of the <Conflicted File Info> section
together.

(Also, while we're at it, I decided to throw a stable short-type
description string (e.g. "CONFLICT (rename/rename)") in there, which
will _probably_ be the first part of the message string but still
allow us to change the message string later if we want.)


Also, we'd want those parsing this information to keep in mind that:
  * Any given conflict can affect multiple paths
  * Any path can be part of multiple conflicts
  * (The above two items imply a potentially many-to-many relationship
between paths and conflicts)
  * Paths listed in these logical conflicts may not correspond to a
file in the index (they could be a directory, or file that was in a
previous version)
  * Some of these "logical conflicts" are not actually conflicts but
just notices (e.g. "auto-merging" or "submodule updated" or "WARNING"
or "<submodules are weird>" messages)

and we'd have to do some work to make sure the paths in the given
messages lined up with the files actually recorded in the index (e.g.
with distinct types we rename both files to avoid the collision, but
print the conflict notice for the original path rather than the new
paths)

[...]
> > But I'm going to reserve the right in merge-ort to modify, add, or
> > delete any of those messages passed to path_msg(), which might wreak
> > havoc on your attempts to parse those strings.  I think they're a bad
> > form for communicating information to a script or program, and trying
> > to transform them into such risks making them suboptimal at
> > communicating info to humans.  These messages should optimize the
> > latter, and if we want something for the former, it should probably be
> > a new independent bit of info.
>
> Makes sense.
>
> So we need something in addition to those messages.

Yes.  Does the proposal above sound like it'd cover your needs?  If
so, we'd probably need to go through all the callers to path_msg() and
either add an immediate call to another function immediately
afterwards that stores this additional information or somehow change
the path_msg() call itself to somehow take an additional arbitrary
list of arguments representing the paths and short-desc we want to
store somewhere.

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Johannes Altmanninger]

On Tue, Feb 22, 2022 at 08:26:41AM -0800, Elijah Newren wrote:
> On Mon, Feb 21, 2022 at 10:55 AM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Elijah Newren <newren@gmail.com> writes:
> >
> > > Adding such an ability to merge-tree would be trivial -- it basically
> > > involves just two things: (1) accepting one extra argument, and (2)
> > > calling merge_incore_nonrecursive() instead of
> > > merge_incore_recursive().
> > >
> > > However, I think forking a subprocess for every merge of a series of
> > > commits is a completely unreasonable overhead, so even if we provide
> > > such an option to merge-tree, I still want a separate plumbing-ish
> > > tool that does non-worktree/non-index replaying of commits which is
> > > not written as a driver of merge-tree.  That other tool should just
> > > call merge_incore_nonrecursive() directly.  And such a tool, since it
> > > should handle an arbitrary number of commits, should certainly be able
> > > to handle just one commit.  From that angle, it feels like adding
> > > another mode to merge-tree would just be a partial duplication of the
> > > other tool.

I don't think "to avoid duplication" is a good argument for making this
plumbing command less flexible, because that's just chasing a local minimum
w.r.t. redundancy. More general APIs will lead to a global minimum.

> >
> > The above does not make much sense to me.
> >
> > I am hearing that "multi-step cherry-picks and reverts need to be
> > fast and we need something like sequencer that is all written in C,
> 
> Yes, I agree with that part so far.  jj is kicking our butt on rebase
> speed; I'm not sure if we can catch it, but it'd be nice to see us not
> be more than a hundred times slower.
> 
> > and single-step cherry-pick is merely a special case that does not
> > deserve a plumbing".
> 
> Well, apparently I failed at communication if that's what you heard.
> Perhaps I can step back and provide my high-level goals, and then
> mention how this series fits in.  My high-level goals:
> 
>   * new sequencer-like replay tool, including multiple abilities
> today's rebase/cherry-pick tools don't have
>   * enable folks to use merging machinery for server side operations
> (merge, rebase, cherry-pick, revert)
>   * do not repeat or encourage the rebase-as-shell-script mistakes of yesteryear
>   * somehow split this up into reviewable chunks
> 
> Now, in particular, the "merge divergent branches" piece seemed like a
> really simple portion of the problem space for which I could get some
> early feedback without having to address the whole problem space all
> at once, and which doesn't seem to have any downside risk.
> 
> And even with my attempt to narrow it in scope, and even despite lots
> of early feedback from the Git Virtual Contributor Summit six months
> ago, it's been nearly two months of active discussions including all
> kinds of intrinsic and tangential points about the UI and design.  Why
> try to prematurely widen the scope?  Can we just focus on merging
> divergent branches for now, and cover the rest later?
> 
> > But that argument leads to "and the same something-like-sequencer
> > that is all written in C would need '--rebase-merges' that can pick
> > multi-step merge sequences, and single-step merge does not deserve a
> > plumbing", which is an argument against this topic that is utterly
> > absurd.
> >
> > So why isn't your objection not equally absurd against having a
> > single step cherry-pick or revert primitive as a plumbing?
> 
> The objection you are arguing against is not my position.  In fact,
> I'm not even objecting to having a single-step cherry-pick, I'm
> objecting to providing it _now_, which I thought would have been clear
> from the portion of my email you snipped ("...I'm happy to add [a
> single step cherry-pick primitive] along with the tool I submit
> later...").  Since that wasn't clear, and since that wasn't my only
> communication failure here, let me attempt to be clearer about my
> objection(s):
> 
> 1. I'm really trying to pick off a small piece of the problem space
> and get feedback on it without unnecessarily complicating things with
> unrelated issues.  Thus, this series is _only_ about merging branches
> that have diverged, and leaves commit replaying for later.
> 
> 2. Two folks have chimed in about the single step cherry-pick, and the
> ONLY reason given for wanting such a thing was to create a
> rebasing/cherry-picking script which was driven by repeatedly invoking
> this low-level primitive command.  That's also the only usecase I can
> currently think of for such a primitive.  To me, that means providing
> such a low-level command now would be likely to result in the
> rebase-as-a-script mistake of yesteryear.  I think we can avoid that
> pitfall by first providing a tool that avoids the
> repeatedly-fork-git-subprocesses model.  (Also, providing a low-level
> single-step cherry-pick command also has the added negative of further
> distracting from the focus on merging divergent branches.)

I agree that it's not a good idea to call merge-tree in a loop for
cherry-picking commit sequences.

At the same time, it is weird for such a low-level tool to not allow
specifying merge bases.
Accepting merge bases is the more logical API, that might allow curious
users to figure out how revert/cherry-pick are implemented.

I intuitively prefer the version that accepts merge bases but I don't have
a good use case, so I think it's okay to add that later if we ever find use
for it.

> 
> 3. The merge primitive in this series is useful completely independent
> of any rebasing script (it would not be used solely for rebasing
> merges, if it's used for that purpose at all, as evidenced by the fact
> that dscho is already trying to use it for doing new real merges).
> 
> 4. Once we have a git-replay tool that can replay a sequence of
> commits, there _might_ not be a need for a single commit replaying
> primitive.  If we provided one as you and Johannes Altimanninger were
> asking for, and it turned out to be deemed useless because the later
> tool I provide can do everything it can and more, haven't we just
> wasted time in providing it?  And perhaps also wasted future time as
> we then have work to do to deprecate and remove the new command or
> mode? (NOTE: I did *not* say there was "no need" for a single-commit
> replaying primitive -- I said there "might not" be a need.)

If we get a tool that can do multiple cherry-picks, I think there is no
technical reason against having an equivalent tool that can do multiple merges.
In that future, merge-tree might be mostly obsolete.

In general, this is a difficult discussion.  It's really hard to judge
this series without a bigger picture of how our future UI will look like.
(Thanks for sharing the replay code BTW, there are some nice features in
there.)  Though I agree that integrating this (minimal) series first makes
a ton of sense, because it already supports a valid use case.

I feel like the output format is a bit experimental because it doesn't give
much of the conflict information in a machine-parseable format.  Of course
it's good enough for many uses (so I don't think this should block this
topic) but I think we should have a plan on how to change the output format
in future without adding ugly compatibility hacks. Marking merge-tree as
"experimental" (like git-switch/git-restore) comes to mind.  That would work
although it's not the most user-friendly way.

(OK I just saw that you are still looking into the output format in
CABPp-BG++YqesTxp+JL3XzwrogfMag1NscoMpCOExmV9z6Py9A@mail.gmail.com )

I wanted to implement some (cherry-picking) scripts using merge-tree but I
don't have enough time or need, so I don't have much feedback on the output
format today. I can imagine that it would be nice to have a clear distinction
between content conflicts and non-content conflicts, but let's worry about
that later..

> 
> Also, since you bring up --rebase-merges, there's an additional point
> about it that might be relevant:
> 
> 5. While you could implement a naive --rebase-merges in terms of a
> primitive for merging divergent branches (or vice-versa, i.e.
> implement merging divergent branches from a naive --rebase-merges
> implementation), I think replaying merges more intelligently[*] is
> actually a distinct operation from doing a new merge of divergent
> branches and that you probably can't implement one in terms of the
> other.  (I'm not certain on this, and definitely don't want to argue
> the finer points on it while my implementation is still half-baked,
> but I really do think they are different things right now.)
> 
> [*] https://lore.kernel.org/git/CABPp-BHp+d62dCyAaJfh1cZ8xVpGyb97mZryd02aCOX=Qn=Ltw@mail.gmail.com/

Re: [PATCH v3 04/15] merge-tree: implement real merges

[Johannes Altmanninger]

On Thu, Feb 24, 2022 at 03:36:55PM -0800, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > It is more like
> > ...
> 
> Actually, I misspoke.  It is a bit different.
> 
> In my mind, the building block hierarchy would have been
> 
>  (1) Take three tree objects A, B, and O, and do the three-way
>      merge.  No history relation is assumed among them.
> 
>  (2) Take two tree objects A and B, with one or more commit objects
>      Os; use (2) recursively to reduce Os into a single O and then
>      apply (1) on A, B and O.

Accepting multiple bases is nice (because it frees users of having
to recursively merge their merge-bases),

Let's say we take this series in its current form:

	git merge-tree [--write-tree] [<options>] <branch1> <branch2>
	git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)

and later discover we want to add (2), one possible syntax would be

	git merge-tree --write-tree [<options>] <branch1> <branch2> <base>...

(or put the bases in the middle like merge-file).
Though the mandatory --write-tree leaves a bad taste.

A separate option is a better alternative:

	git merge-tree [--write-tree] [<options>] --base=<base1>,<base2>,... <branch1> <branch2>

Anyway, no need to worry about that now, especially since the root cause of
the ugliness is the legacy --trivial-merge, and there is no way avoid that,
even if we add this now.

> 
>  (3) Take two commit objects A and B.  Compute Os out of A and B and
>      use (2) once to merge A and B.
> 
> I think the basic primitive that should be exposed to an external
> world (read: plumbing) this year, after all years of experience with
> merge-recursive, should be (2), not (1).  
> 
> If you have (2), then (3) is trivially possible (it is just a single
> call to get_merge_base()).  "git merge-tree A B" without having to
> spell out bases is so convenent and you do not have to write
> "git merge-tree A B $(git merge-base --all A B)", so I am OK for it
> to exist, but it is not essential.
> 
> If you have (2) and exposed (2) as the primitive plumbing,
> cherry-pick and revert would be a narrow special case of passing
> only one O to the machinery.
> 
> And coming from the above point of view, exposing (3) as the
> primitive plumbing to scripters and system builders, and later
> having to _add_ support to allow (2), felt backwards.  It should be
> trivial for us to make (2) available before we can even offer (3),
> but what is happening to this new plumbing command goes in the
> opposite order.
> 
> It may be, as you said, the problem the underlying ort API has that
> somehow makes it harder to expose (2), in which case, yes, I think
> that is what bugs me.
> 
> 
> 

Re: machine-parsable git-merge-tree messages (was: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function)

[Ævar Arnfjörð Bjarmason]

On Tue, Feb 22 2022, Elijah Newren wrote:

> On Mon, Feb 21, 2022 at 6:37 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>>
>> On Mon, Feb 21 2022, Johannes Schindelin wrote:
>>
>> [I sent out an empty reply to this earlier by mistake, sorry about that]
>>
>> > [...]
>> > Which brings me to the next concern: since `merge-tree` is a low-level
>> > tool meant to be called by programs rather than humans, we need to make
>> > sure that those messages remain machine-parseable, even if they contain
>> > file names.
>> >
>> > Concretely: while I am not currently aware of any web UI that allows to
>> > resolve simple rename/rename conflicts, it is easily conceivable how to
>> > implement such a thing. When that happens, we will need to be able to
>> > teach the server-side code to discern between the cases that can be
>> > handled in the web UI (trivial merge conflicts, trivial rename/rename
>> > conflicts) as compared to scenarios where the conflicts are just too
>> > complex.
>> >
>> > Here's an excerpt from t4301:
>> >
>> > -- snip --
>> > Auto-merging greeting
>> > CONFLICT (content): Merge conflict in greeting
>> > Auto-merging numbers
>> > CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
>> > CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
>> > -- snap --
>> >
>> > This is the complete set of messages provided in the `test conflict
>> > notices and such` test case.
>> >
>> > I immediately notice that every line contains at least one file name.
>> > Looking at https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1899, it
>> > does not seem as if the file names are quoted:
>> >
>> >               path_msg(opt, path, 1, _("Auto-merging %s"), path);
>> >
>> > (where `path` is used verbatim in a call to `merge_3way()` before that,
>> > i.e. it must not have been quoted)
>> >
>> > I would like to register a wish to ensure that file names with special
>> > characters (such as most notably line-feed characters) are quoted in these
>> > messages, so that a simple server-side parser can handle messages starting
>> > with `Auto-merging` and with `CONFLICT (content): Merge conflict in `, and
>> > "throw the hands up in the air" if any other message prefix is seen.
>> >
>> > Do you think we can switch to `sq_quote_buf_pretty()` for these messages?
>> > For the `Auto-merging` one, it would be trivial, but I fear that we will
>> > have to work a bit on the `path_msg()` function
>> > (https://github.com/git/git/blob/v2.35.1/merge-ort.c#L630-L649) because it
>> > accepts a variable list of arguments without any clue whether the
>> > arguments refer to paths or not. (And I would be loathe to switch _all_
>> > callers to do the quoting themselves.)
>> >
>> > I see 28 calls to that function, and at least a couple that pass not only
>> > a path but also an OID (e.g.
>> > https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1611-L1613).
>> >
>> > We could of course be sloppy and pass even OIDs through
>> > `sq_quote_buf_pretty()` in `path_msg()`, knowing that there won't be any
>> > special characters in them, but it gets more complicated e.g. in
>> > https://github.com/git/git/blob/v2.35.1/merge-ort.c#L1648-L1651, where we
>> > pass an `strbuf` that contains a somewhat free-form commit message.
>> >
>> > I guess we could still pass those through `sq_quote_buf_pretty()`, even if
>> > they are not paths, to ensure that there are no special characters in the
>> > machine-parseable lines.
>> >
>> > What do you think?
>>
>> That sounds like a rather nasty hack, this is too, but demonstrates that
>> we can pretty easily extract this in a machine-readable format with just
>> a few lines now):
>>
>> diff --git a/merge-ort.c b/merge-ort.c
>> index 8a5f201d190..a906881f9b3 100644
>> --- a/merge-ort.c
>> +++ b/merge-ort.c
>> @@ -633,7 +633,7 @@ static void path_msg(struct merge_options *opt,
>>                      int omittable_hint, /* skippable under --remerge-diff */
>>                      const char *fmt, ...)
>>  {
>> -       va_list ap;
>> +       va_list ap, cp;
>>         struct strbuf *sb, *dest;
>>         struct strbuf tmp = STRBUF_INIT;
>>
>> @@ -650,7 +650,9 @@ static void path_msg(struct merge_options *opt,
>>
>>         dest = (opt->record_conflict_msgs_as_headers ? &tmp : sb);
>>
>> +       va_copy(cp, ap);
>>         va_start(ap, fmt);
>> +
>>         if (opt->priv->call_depth) {
>>                 strbuf_addchars(dest, ' ', 2);
>>                 strbuf_addstr(dest, "From inner merge:");
>> @@ -659,6 +661,15 @@ static void path_msg(struct merge_options *opt,
>>         strbuf_vaddf(dest, fmt, ap);
>>         va_end(ap);
>>
>> +       va_start(cp, fmt);
>> +       trace2_region_enter_printf("merge", "conflict/path", opt->repo, "%s", path);
>> +       trace2_region_leave("merge", "conflict/path", opt->repo);
>> +       trace2_region_enter_printf("merge", "conflict/fmt", opt->repo, "%s", fmt);
>> +       trace2_region_leave("merge", "conflict/fmt", opt->repo);
>> +       trace2_region_enter_printf_va("merge", "conflict/msg", opt->repo, fmt, cp);
>> +       trace2_region_leave("merge", "conflict/msg", opt->repo);
>> +       va_end(cp);
>> +
>>         if (opt->record_conflict_msgs_as_headers) {
>>                 int i_sb = 0, i_tmp = 0;
>>
>> You can run that with one of the tests added in this series to get the
>> output as JSON, e.g.:
>>
>>      GIT_TRACE2_EVENT=/dev/stderr GIT_TRACE2_EVENT_NESTING=10 ~/g/git/git merge-tree --write-tree --no-messages --name-only --messages side1 side2 2>&1|jq -r .| grep '"msg"'
>>       "msg": "whatever~side1"
>>       "msg": "CONFLICT (file/directory): directory in the way of %s from %s; moving it to %s instead."
>>       "msg": "CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead."
>>       "msg": "whatever~side1"
>>       "msg": "CONFLICT (modify/delete): %s deleted in %s and modified in %s.  Version %s of %s left in tree."
>>       "msg": "CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree."
>>       "msg": "numbers"
>>       "msg": "Auto-merging %s"
>>       "msg": "Auto-merging numbers"
>>       "msg": "greeting"
>>       "msg": "Auto-merging %s"
>>       "msg": "Auto-merging greeting"
>>       "msg": "greeting"
>>       "msg": "CONFLICT (%s): Merge conflict in %s"
>>       "msg": "CONFLICT (content): Merge conflict in greeting"
>>
>> A "proper" fix for this doesn't sound too hard, we'd just instrument the
>> path_msg() function to pass along some "message category", see
>> e.g. unpack_plumbing_errors in unpack-trees.c for one example of such a
>> thing, or the "enum fsck_msg_id".
>>
>> Then we'd just allow you to emit any of the sprintf() format itself, or
>> the expanded version, the path, or an id like "CONFLICT:file/directory"
>> or "auto-merging" etc.
>
> I don't see how this helps solve the problem Dscho was bringing up at
> all.  Your reference to "the path" means you've missed his whole
> complaint -- that with more complex conflicts (renames, directory/file
> conflicts resolved via moving the file out of the way, mode conflicts
> resolved by moving both files out of the way, etc) there are multiple
> paths involved and he's trying to determine what those paths are.
> He's particularly focusing on rename/rename cases where a single path
> was renamed differently by the two sides of history (which results in
> a conflict message only being associated with the path from the merge
> base in order to avoid repeating the same message 2-3 times, but that
> one message has three distinct paths embedded in the string).
>
> Also, the additional paths is not part of the API to path_msg; it's
> merely embedded in a string.  (And, in case it bears repeating: as
> mentioned elsewhere, we cannot assume there will only be one
> path_msg() call per path, and we at least currently can't assume that
> each path_msg() call is for a separate logical conflict; there might
> be two for a single "conflict".)
>
> I agree that parsing these meant-for-human-consumption (and not
> promised to be stable) messages is not a good way to go, but
> pretending the current API has enough info to answer his questions
> isn't right either.

The intent here wasn't to present a complete solution, but to reply to
the part of Johannes's E-Mail that e.g. mention "and I would be loathe
to switch _all_ callers to do the quoting themselves.".

I.e. it's a POC for passing this data further up the stack. The issue
you mention with the renaming case could/should be handled by having
whatever handles the vargs accept those N arguments, the POC doesn't
handle it.

But in any case, needing to convert "28 calls to [path_msg()]" doesn't
seem like it's required.

But obviously we wouldn't want to use trace2 as a plumbing layer for
message passing, but could format the same data in a similar way,
especially in the context of a discussion about filenames with odd
characters in them (some of which JSON is inherently incapable of
encoding).

>> I think that would be particularly useful in conjuction with the
>> --format changes I was proposing for this (and hacked up a WIP patch
>> for). You could just have a similar --format-messages or whatever.
>>
>> Then you could pick \0\0 as a delimiter for your "main" --format, and
>> "\0" as the delimiter for your --format-messages, and thus be able to
>> parse N-nested \0-delimited content.
>
> To be honest, the --format stuff is sounding a little bit like a
> solution in search of a problem.

Opinions on this obviously differ, and I'm not going to pick this as my
particular hill to die on :)

But I do think it's the other way around, in that hardcoded output
formats are a problem requiring solutions.

Which e.g. is seen (to be fair, in rather small inconveniences) in this
series where you need to grep out the first OID line etc. in the tests,
instead of the command just being able to be asked for the exact thing
the user wants.

Now, perfect shouldn't be the enemy of the good, and I think this series
is definitely in good enough shape as it is. But as
e.g. "builtin/ls-tree.c" in "seen" currently shows using the
strbuf_expand() callback mechanism to emit output will result in less
code/special cases.

Now, in this case the problem is mostly orthagonal, i.e. that the
merge-ort.c API understandably isn't returning structured data in this
case, as the goal so far has been replacing the in-tree
merge-recursive.c use-case.

But if & when that happens I think e.g. returning those as a "struct
string_list" with some custom struct in the "util" member might be a
good idea, followed by a strbuf_expand() knowing how to format fields
from that struct.

The whole thing could then become e.g. (whitespace added for
convenience):

	--format="\
		%(objectname)
		\0\0
		%(conflict:%(
			%(conflictmode)
                        \0
                        %(conflictobject)
                        \0
                        %(conflictstage)
                        \0
                        %(conflictfilename)
		)
	"

And it seems to me like the whole of --show-messages etc. could be
generalized as some (the "if" syntax would need to be extracted and
generalized from ref-filter.c:

    %(if:notequal=0)%(mergeresult)%(then)...%(end)

So again, I'm not saying it's needed now, just that I think the pattern
of preparing structured data and throwing it at a formatting engine is
worth considering sooner than later.

It we didn't have that for "git-for-each-ref" I think it would have N
number of "--only-this-but-not-that"-type options, which this new
command already requires.

And with a formatting engine things like "I want my output quoted for
consumption by programming language X" become much harder, but which
"git-for-each-ref" already supports using that formatting mechanism.

Re: machine-parsable git-merge-tree messages (was: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function)

[Elijah Newren]

On Mon, Feb 28, 2022 at 1:27 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Tue, Feb 22 2022, Elijah Newren wrote:
>
[...]
> > I don't see how this helps solve the problem Dscho was bringing up at
> > all.  Your reference to "the path" means you've missed his whole
> > complaint -- that with more complex conflicts (renames, directory/file
> > conflicts resolved via moving the file out of the way, mode conflicts
> > resolved by moving both files out of the way, etc) there are multiple
> > paths involved and he's trying to determine what those paths are.
> > He's particularly focusing on rename/rename cases where a single path
> > was renamed differently by the two sides of history (which results in
> > a conflict message only being associated with the path from the merge
> > base in order to avoid repeating the same message 2-3 times, but that
> > one message has three distinct paths embedded in the string).
> >
> > Also, the additional paths is not part of the API to path_msg; it's
> > merely embedded in a string.  (And, in case it bears repeating: as
> > mentioned elsewhere, we cannot assume there will only be one
> > path_msg() call per path, and we at least currently can't assume that
> > each path_msg() call is for a separate logical conflict; there might
> > be two for a single "conflict".)
> >
> > I agree that parsing these meant-for-human-consumption (and not
> > promised to be stable) messages is not a good way to go, but
> > pretending the current API has enough info to answer his questions
> > isn't right either.
>
> The intent here wasn't to present a complete solution, but to reply to
> the part of Johannes's E-Mail that e.g. mention "and I would be loathe
> to switch _all_ callers to do the quoting themselves.".
>
> I.e. it's a POC for passing this data further up the stack. The issue
> you mention with the renaming case could/should be handled by having
> whatever handles the vargs accept those N arguments, the POC doesn't
> handle it.
>
> But in any case, needing to convert "28 calls to [path_msg()]" doesn't
> seem like it's required.

The problem isn't that it's an incomplete solution, it's that AFAICT,
the user's stated problem is not aided at all by this POC.  Passing
existing data further up the stack cannot solve the problem if the
data being passed is insufficient for the problem at hand.

Perhaps you have some clever solution to get the extra information,
though?  Could you elaborate on how path_msg() could handle its
varargs differently to deduce which of those correspond to paths?  The
only way I can see how to do that, short of modifying all 28 callers
of path_msg() to pass those paths as additional information, is hacks
like parsing the (non-stable, not-meant-for-machine-parseability)
format string.

(Getting the paths would get us most of the way to a solution, though
it's still incomplete.  But it's the relevant bit under discussion
here.)

> But obviously we wouldn't want to use trace2 as a plumbing layer for
> message passing, but could format the same data in a similar way,
> especially in the context of a discussion about filenames with odd
> characters in them (some of which JSON is inherently incapable of
> encoding).
>
> >> I think that would be particularly useful in conjuction with the
> >> --format changes I was proposing for this (and hacked up a WIP patch
> >> for). You could just have a similar --format-messages or whatever.
> >>
> >> Then you could pick \0\0 as a delimiter for your "main" --format, and
> >> "\0" as the delimiter for your --format-messages, and thus be able to
> >> parse N-nested \0-delimited content.
> >
> > To be honest, the --format stuff is sounding a little bit like a
> > solution in search of a problem.
>
> Opinions on this obviously differ, and I'm not going to pick this as my
> particular hill to die on :)
>
> But I do think it's the other way around, in that hardcoded output
> formats are a problem requiring solutions.

I might be more convinced if folks tried to address how to output
things _after_ we had determined *what* things should be output.  If
we don't have sufficient information to solve what users want,
discussing how we format the information we do have cannot help solve
the actual problems.  It might be useful as an add-on later, but
discussing it first is putting the cart before the horse.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Fri, 25 Feb 2022, Elijah Newren wrote:

> On Fri, Feb 25, 2022 at 8:31 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Tue, 22 Feb 2022, Elijah Newren wrote:
> >
> > > On Mon, Feb 21, 2022 at 2:46 AM Johannes Schindelin
> > > <Johannes.Schindelin@gmx.de> wrote:
> > > >
> > > > Concretely: while I am not currently aware of any web UI that allows
> > > > to resolve simple rename/rename conflicts, it is easily conceivable
> > > > how to implement such a thing. When that happens, we will need to be
> > > > able to teach the server-side code to discern between the cases that
> > > > can be handled in the web UI (trivial merge conflicts, trivial
> > > > rename/rename conflicts) as compared to scenarios where the conflicts
> > > > are just too complex.
> > >
> > > Um, I'm really worried about attempting to make the conflict notices
> > > machine parseable.  I don't like that idea at all, and I even tried to
> > > rule that out already with my wording:
> > > """
> > > In all cases, the
> > > <Informational messages> section has the necessary info, though it is
> > > not designed to be machine parseable.
> > > """
> > > though maybe I should have been even more explicit.  The restrictions
> > > that those messages be stable is too rigid, I think.  I also think
> > > they're a poor way to communicate information to a higher level tool.
> > > I would much rather us add some kind of additional return data
> > > structures from merge ort and use them if we want extra info.
> >
> > Okay.
> >
> > I thought that we could keep the `CONFLICT (<type>)` constant enough to
> > serve as such a machine-parseable thing.
>
> That _probably_ is, but I thought you wanted to parse all N paths
> embedded in the message after that part as well?

Actually, no, sorry for being unclear. My main aim with the
machine-parseable messages was to detect whether a given failed merge
contains _only_ conflicts of the sort that a particular UI can handle.

In that respect, I would not even need to parse the file names, I'd just
require them not to contain line feed characters ;-)

> > And then presenting
> > `<path>NUL<message>NUL` could have served my use case well...
>
> Would it?  Wouldn't you need something more like
>
> <number-of-paths>NUL<path1>NUL<path2>NUL<pathN>NUL<stable-short-type-description>NUL<message>NUL
>  ?

Probably, you're right.

> I mean, if rename/rename is what you want to handle, there are three
> paths in that message.  And you need to know all three paths in order
> to combine the relevant parts of the <Conflicted File Info> section
> together.
>
> (Also, while we're at it, I decided to throw a stable short-type
> description string (e.g. "CONFLICT (rename/rename)") in there, which
> will _probably_ be the first part of the message string but still
> allow us to change the message string later if we want.)

Yes, I very much like that idea to keep the prefix in a format that we can
guarantee to be stable enough for applications (or server backends) to
rely on.

> Also, we'd want those parsing this information to keep in mind that:
>   * Any given conflict can affect multiple paths
>   * Any path can be part of multiple conflicts
>   * (The above two items imply a potentially many-to-many relationship
> between paths and conflicts)
>   * Paths listed in these logical conflicts may not correspond to a
> file in the index (they could be a directory, or file that was in a
> previous version)
>   * Some of these "logical conflicts" are not actually conflicts but
> just notices (e.g. "auto-merging" or "submodule updated" or "WARNING"
> or "<submodules are weird>" messages)
>
> and we'd have to do some work to make sure the paths in the given
> messages lined up with the files actually recorded in the index (e.g.
> with distinct types we rename both files to avoid the collision, but
> print the conflict notice for the original path rather than the new
> paths)
>
> [...]
> > > But I'm going to reserve the right in merge-ort to modify, add, or
> > > delete any of those messages passed to path_msg(), which might wreak
> > > havoc on your attempts to parse those strings.  I think they're a bad
> > > form for communicating information to a script or program, and trying
> > > to transform them into such risks making them suboptimal at
> > > communicating info to humans.  These messages should optimize the
> > > latter, and if we want something for the former, it should probably be
> > > a new independent bit of info.
> >
> > Makes sense.
> >
> > So we need something in addition to those messages.
>
> Yes.  Does the proposal above sound like it'd cover your needs?  If
> so, we'd probably need to go through all the callers to path_msg() and
> either add an immediate call to another function immediately
> afterwards that stores this additional information or somehow change
> the path_msg() call itself to somehow take an additional arbitrary
> list of arguments representing the paths and short-desc we want to
> store somewhere.

Yes, you're right, the proper thing to do is to go through the callers to
`path_msg()` so that we can provide that stable output you proposed. A
couple of thoughts about this:

* These are not informal messages, i.e. I think we would need another flag
  that would then trigger another double-`NUL`-separated section to be
  shown, probably between the conflicted file info and the informational
  messages.

* Instead of _adding_ the calls, we could look into refactoring the
  existing `path_msg()` calls, introducing yet another function that would
  call `path_msg()` but also optionally generate that machine-parseable
  conflict info.

* None of this needs to hold up `en/merge-tree`. I am sorry that I am the
  blocker (unintentionally so, I guarantee you that!).

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Mon, Mar 7, 2022 at 8:27 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Fri, 25 Feb 2022, Elijah Newren wrote:
>
> > On Fri, Feb 25, 2022 at 8:31 AM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > On Tue, 22 Feb 2022, Elijah Newren wrote:
> > >
> > > > On Mon, Feb 21, 2022 at 2:46 AM Johannes Schindelin
> > > > <Johannes.Schindelin@gmx.de> wrote:
> > > > >
> > > > > Concretely: while I am not currently aware of any web UI that allows
> > > > > to resolve simple rename/rename conflicts, it is easily conceivable
> > > > > how to implement such a thing. When that happens, we will need to be
> > > > > able to teach the server-side code to discern between the cases that
> > > > > can be handled in the web UI (trivial merge conflicts, trivial
> > > > > rename/rename conflicts) as compared to scenarios where the conflicts
> > > > > are just too complex.
> > > >
> > > > Um, I'm really worried about attempting to make the conflict notices
> > > > machine parseable.  I don't like that idea at all, and I even tried to
> > > > rule that out already with my wording:
> > > > """
> > > > In all cases, the
> > > > <Informational messages> section has the necessary info, though it is
> > > > not designed to be machine parseable.
> > > > """
> > > > though maybe I should have been even more explicit.  The restrictions
> > > > that those messages be stable is too rigid, I think.  I also think
> > > > they're a poor way to communicate information to a higher level tool.
> > > > I would much rather us add some kind of additional return data
> > > > structures from merge ort and use them if we want extra info.
> > >
> > > Okay.
> > >
> > > I thought that we could keep the `CONFLICT (<type>)` constant enough to
> > > serve as such a machine-parseable thing.
> >
> > That _probably_ is, but I thought you wanted to parse all N paths
> > embedded in the message after that part as well?
>
> Actually, no, sorry for being unclear. My main aim with the
> machine-parseable messages was to detect whether a given failed merge
> contains _only_ conflicts of the sort that a particular UI can handle.

...and in order to do that, you'd need to parse all the filenames to
determine whether files were involved in multiple conflict types,
since you earlier suggested you would just bail on such conflicts (at
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2202211059430.26495@tvgsbejvaqbjf.bet/).
In particular, I think the "mod6" and "rrdd" and "rad" cases kind of
triggered those comments, and rename/rename is involved in two of
those.

Further, even if you could assume there'd only be simple conflicts
such as the simple rename/rename case, you'd still need the files so
that you know _which_ of the modes/oids in the <Conflicted file info>
section relate to the message you are looking at.  (As mentioned
previously, sorting the <Conflicted file info> to attempt to put them
together is not feasible and is a no-go.)

> > > And then presenting
> > > `<path>NUL<message>NUL` could have served my use case well...
> >
> > Would it?  Wouldn't you need something more like
> >
> > <number-of-paths>NUL<path1>NUL<path2>NUL<pathN>NUL<stable-short-type-description>NUL<message>NUL
> >  ?
>
> Probably, you're right.
>
> > I mean, if rename/rename is what you want to handle, there are three
> > paths in that message.  And you need to know all three paths in order
> > to combine the relevant parts of the <Conflicted File Info> section
> > together.
> >
> > (Also, while we're at it, I decided to throw a stable short-type
> > description string (e.g. "CONFLICT (rename/rename)") in there, which
> > will _probably_ be the first part of the message string but still
> > allow us to change the message string later if we want.)
>
> Yes, I very much like that idea to keep the prefix in a format that we can
> guarantee to be stable enough for applications (or server backends) to
> rely on.

Um, I explicitly avoided saying the prefix would be stable by
introducing a separate short string just so that we could change the
prefix later if wanted.  The short string is _probably_ the current
prefix or something close to it, but that stable string wouldn't
necessarily remain the prefix of the message string, since the entire
message string would be allowed to change.

> > Also, we'd want those parsing this information to keep in mind that:
> >   * Any given conflict can affect multiple paths
> >   * Any path can be part of multiple conflicts
> >   * (The above two items imply a potentially many-to-many relationship
> > between paths and conflicts)
> >   * Paths listed in these logical conflicts may not correspond to a
> > file in the index (they could be a directory, or file that was in a
> > previous version)
> >   * Some of these "logical conflicts" are not actually conflicts but
> > just notices (e.g. "auto-merging" or "submodule updated" or "WARNING"
> > or "<submodules are weird>" messages)
> >
> > and we'd have to do some work to make sure the paths in the given
> > messages lined up with the files actually recorded in the index (e.g.
> > with distinct types we rename both files to avoid the collision, but
> > print the conflict notice for the original path rather than the new
> > paths)
> >
> > [...]
> > > > But I'm going to reserve the right in merge-ort to modify, add, or
> > > > delete any of those messages passed to path_msg(), which might wreak
> > > > havoc on your attempts to parse those strings.  I think they're a bad
> > > > form for communicating information to a script or program, and trying
> > > > to transform them into such risks making them suboptimal at
> > > > communicating info to humans.  These messages should optimize the
> > > > latter, and if we want something for the former, it should probably be
> > > > a new independent bit of info.
> > >
> > > Makes sense.
> > >
> > > So we need something in addition to those messages.
> >
> > Yes.  Does the proposal above sound like it'd cover your needs?  If
> > so, we'd probably need to go through all the callers to path_msg() and
> > either add an immediate call to another function immediately
> > afterwards that stores this additional information or somehow change
> > the path_msg() call itself to somehow take an additional arbitrary
> > list of arguments representing the paths and short-desc we want to
> > store somewhere.
>
> Yes, you're right, the proper thing to do is to go through the callers to
> `path_msg()` so that we can provide that stable output you proposed. A
> couple of thoughts about this:
>
> * These are not informal messages, i.e. I think we would need another flag
>   that would then trigger another double-`NUL`-separated section to be
>   shown, probably between the conflicted file info and the informational
>   messages.

Why would that be a separate section, though?  While we don't want
machines parsing the informal messages and trying to determine the
components of that message, they definitely should be able to tell
which informal messages are associated with which paths (otherwise how
can you group the <Conflict message info> as per your needs and how do
you know which message to show the user when dealing with that
particular conflict?).  That requirement suggests the informal
messages should be part of the same section.  Thus my suggestion to
just make it be the <message> part at the end of my earlier suggestion
of

<number-of-paths>NUL<path1>NUL<path2>NUL<pathN>NUL<stable-short-type-description>NUL<message>NUL

> * Instead of _adding_ the calls, we could look into refactoring the
>   existing `path_msg()` calls, introducing yet another function that would
>   call `path_msg()` but also optionally generate that machine-parseable
>   conflict info.

So of the two alternatives I suggested above, it sounds like you're in
favor of the "or somehow change the path_msg() call itself to somehow
take an additional arbitrary list of arguments representing the paths
and short-desc we want to store somewhere" option I suggested.  I
think I prefer that one too.

> * None of this needs to hold up `en/merge-tree`. I am sorry that I am the
>   blocker (unintentionally so, I guarantee you that!).

Sometimes I find it frustrating that changes don't merge down for
months, particularly when the topic was already finished weeks and
weeks (if not months) ago and/or when there's a follow-on series
depending on it.  But in this case I have no follow-on series ready to
send, and this topic has been under (very!) active discussion
essentially the whole time.  And, perhaps even more importantly, I'd
like to avoid solidifying the "machine parseable output format"
prematurely and making it harder to work around later.  I think
addressing this issue you bring up requires fundamentally redoing the
informational messages section as per my proposal, and it makes me
wonder what should even be shown without the -z option (thoughts
welcome).

So, this is one series where even if everyone else says to merge it
already, I'd like to wait a bit longer on it until I feel confident we
have a solution that handles at least the current usecases.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Tue, 8 Mar 2022, Elijah Newren wrote:

> On Mon, Mar 7, 2022 at 8:27 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Fri, 25 Feb 2022, Elijah Newren wrote:
> >
> > > On Fri, Feb 25, 2022 at 8:31 AM Johannes Schindelin
> > > <Johannes.Schindelin@gmx.de> wrote:
> > > >
> > > > On Tue, 22 Feb 2022, Elijah Newren wrote:
> > > >
> > > > > On Mon, Feb 21, 2022 at 2:46 AM Johannes Schindelin
> > > > > <Johannes.Schindelin@gmx.de> wrote:
> > > > > >
> > > > > > Concretely: while I am not currently aware of any web UI that
> > > > > > allows to resolve simple rename/rename conflicts, it is easily
> > > > > > conceivable how to implement such a thing. When that happens,
> > > > > > we will need to be able to teach the server-side code to
> > > > > > discern between the cases that can be handled in the web UI
> > > > > > (trivial merge conflicts, trivial rename/rename conflicts) as
> > > > > > compared to scenarios where the conflicts are just too
> > > > > > complex.
> > > > >
> > > > > Um, I'm really worried about attempting to make the conflict
> > > > > notices machine parseable.  I don't like that idea at all, and I
> > > > > even tried to rule that out already with my wording: """ In all
> > > > > cases, the <Informational messages> section has the necessary
> > > > > info, though it is not designed to be machine parseable. """
> > > > > though maybe I should have been even more explicit.  The
> > > > > restrictions that those messages be stable is too rigid, I
> > > > > think.  I also think they're a poor way to communicate
> > > > > information to a higher level tool. I would much rather us add
> > > > > some kind of additional return data structures from merge ort
> > > > > and use them if we want extra info.
> > > >
> > > > Okay.
> > > >
> > > > I thought that we could keep the `CONFLICT (<type>)` constant
> > > > enough to serve as such a machine-parseable thing.
> > >
> > > That _probably_ is, but I thought you wanted to parse all N paths
> > > embedded in the message after that part as well?
> >
> > Actually, no, sorry for being unclear. My main aim with the
> > machine-parseable messages was to detect whether a given failed merge
> > contains _only_ conflicts of the sort that a particular UI can handle.
>
> ...and in order to do that, you'd need to parse all the filenames to
> determine whether files were involved in multiple conflict types, since
> you earlier suggested you would just bail on such conflicts (at
> https://lore.kernel.org/git/nycvar.QRO.7.76.6.2202211059430.26495@tvgsbejvaqbjf.bet/).

That's a good point. I had somehow still in my mind that the `CONFLICT
(<type>)` prefix would already give me enough data to decide whether to
bail out or not, but that prefix would only tell me about a single
`rename/rename` conflict, not about multiple such conflicts involving the
_same_ file.

> In particular, I think the "mod6" and "rrdd" and "rad" cases kind of
> triggered those comments, and rename/rename is involved in two of those.
>
> Further, even if you could assume there'd only be simple conflicts such
> as the simple rename/rename case, you'd still need the files so that you
> know _which_ of the modes/oids in the <Conflicted file info> section
> relate to the message you are looking at.  (As mentioned previously,
> sorting the <Conflicted file info> to attempt to put them together is
> not feasible and is a no-go.)

True.

> > > > And then presenting `<path>NUL<message>NUL` could have served my
> > > > use case well...
> > >
> > > Would it?  Wouldn't you need something more like
> > >
> > > <number-of-paths>NUL<path1>NUL<path2>NUL<pathN>NUL<stable-short-type-description>NUL<message>NUL
> > > ?
> >
> > Probably, you're right.
> >
> > > I mean, if rename/rename is what you want to handle, there are three
> > > paths in that message.  And you need to know all three paths in
> > > order to combine the relevant parts of the <Conflicted File Info>
> > > section together.
> > >
> > > (Also, while we're at it, I decided to throw a stable short-type
> > > description string (e.g. "CONFLICT (rename/rename)") in there, which
> > > will _probably_ be the first part of the message string but still
> > > allow us to change the message string later if we want.)
> >
> > Yes, I very much like that idea to keep the prefix in a format that we
> > can guarantee to be stable enough for applications (or server
> > backends) to rely on.
>
> Um, I explicitly avoided saying the prefix would be stable by
> introducing a separate short string just so that we could change the
> prefix later if wanted.  The short string is _probably_ the current
> prefix or something close to it, but that stable string wouldn't
> necessarily remain the prefix of the message string, since the entire
> message string would be allowed to change.

Okay.

> > [T]he proper thing to do is to go through the callers to `path_msg()`
> > so that we can provide that stable output you proposed. A couple of
> > thoughts about this:
> >
> > * These are not informal messages, i.e. I think we would need another flag
> >   that would then trigger another double-`NUL`-separated section to be
> >   shown, probably between the conflicted file info and the informational
> >   messages.
>
> Why would that be a separate section, though?

I thought you wanted to keep the informal messages human-readable, so
naturally I thought that the machine-readable part should be in a separate
section.

> While we don't want machines parsing the informal messages and trying to
> determine the components of that message, they definitely should be able
> to tell which informal messages are associated with which paths
> (otherwise how can you group the <Conflict message info> as per your
> needs and how do you know which message to show the user when dealing
> with that particular conflict?).  That requirement suggests the informal
> messages should be part of the same section.  Thus my suggestion to just
> make it be the <message> part at the end of my earlier suggestion of
>
> <number-of-paths>NUL<path1>NUL<path2>NUL<pathN>NUL<stable-short-type-description>NUL<message>NUL

Oh, okay.

Should this format be the one with `-z`, and the current format remain the
one being shown without `-z`?

> > * Instead of _adding_ the calls, we could look into refactoring the
> >   existing `path_msg()` calls, introducing yet another function that would
> >   call `path_msg()` but also optionally generate that machine-parseable
> >   conflict info.
>
> So of the two alternatives I suggested above, it sounds like you're in
> favor of the "or somehow change the path_msg() call itself to somehow
> take an additional arbitrary list of arguments representing the paths
> and short-desc we want to store somewhere" option I suggested.  I
> think I prefer that one too.

Yes!

> > * None of this needs to hold up `en/merge-tree`. I am sorry that I am the
> >   blocker (unintentionally so, I guarantee you that!).
>
> Sometimes I find it frustrating that changes don't merge down for
> months, particularly when the topic was already finished weeks and
> weeks (if not months) ago and/or when there's a follow-on series
> depending on it.  But in this case I have no follow-on series ready to
> send, and this topic has been under (very!) active discussion
> essentially the whole time.  And, perhaps even more importantly, I'd
> like to avoid solidifying the "machine parseable output format"
> prematurely and making it harder to work around later.  I think
> addressing this issue you bring up requires fundamentally redoing the
> informational messages section as per my proposal, and it makes me
> wonder what should even be shown without the -z option (thoughts
> welcome).

My suggestion is above: without `-z`, the messages should be identical to
what is shown now.

> So, this is one series where even if everyone else says to merge it
> already, I'd like to wait a bit longer on it until I feel confident we
> have a solution that handles at least the current usecases.

Fair enough, you're in charge of this series, and I really like what you
came up with.

My thinking was driven more by the users' side, as I am relatively eager
to integrate this into production, but am loathe to do that with an early
iteration of `en/merge-tree` that might be substantially revamped, still.

It can bog me down quite well if I have to adjust local (or not-so-local)
patch series to refactorings in core Git. For example, the other day, I
was blocked from more interesting work until I could resolve all kinds of
conflicts where core Git renaming `hash_object_file_literally()` to
`write_object_file_literally()` completely intersected with
actually-interesting `unsigned long` to `size_t` changes in Git for
Windows. And let me tell you, once you have to deal with such conflicts a
couple of times, being prevented from doing work that is actually fun, it
is relatively easy to become frustrated and think about those refactorings
as pointless and as useless churn, no matter how well-intended they may
be.

So my prodding is kind of selfish, as an `en/merge-tree` cherry-picked
from `next` would provide somewhat of a promise that I do _not_ have to
spend _so_ much time adjusting my code in case a new iteration with
substantial changes shows up ;-)

But you're right, let's tread wisely and only call this patch series done
only when we are reasonably certain that the machine-parseable output is
stable.

Speaking of which: Would you like me to give that `path_msg()` refactoring
a try, or are you already busy with it? (I cannot make any promises, but I
also do not want to lean on you, to leave you alone with that work, I
would feel really bad about that.)

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Thu, 10 Mar 2022, Johannes Schindelin wrote:

> On Tue, 8 Mar 2022, Elijah Newren wrote:
>
> > So, this is one series where even if everyone else says to merge it
> > already, I'd like to wait a bit longer on it until I feel confident we
> > have a solution that handles at least the current usecases.
>
> Fair enough, you're in charge of this series, and I really like what you
> came up with.
>
> My thinking was driven more by the users' side, as I am relatively eager
> to integrate this into production, but am loathe to do that with an early
> iteration of `en/merge-tree` that might be substantially revamped, still.

I've been bogged down with things elsewhere, but should now have time to
help on this end.

Elijah, _is_ there anything I can help with?

Thanks,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

Hi Johannes,

On Fri, May 13, 2022 at 3:21 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Thu, 10 Mar 2022, Johannes Schindelin wrote:
>
> > On Tue, 8 Mar 2022, Elijah Newren wrote:
> >
> > > So, this is one series where even if everyone else says to merge it
> > > already, I'd like to wait a bit longer on it until I feel confident we
> > > have a solution that handles at least the current usecases.
> >
> > Fair enough, you're in charge of this series, and I really like what you
> > came up with.
> >
> > My thinking was driven more by the users' side, as I am relatively eager
> > to integrate this into production, but am loathe to do that with an early
> > iteration of `en/merge-tree` that might be substantially revamped, still.
>
> I've been bogged down with things elsewhere, but should now have time to
> help on this end.
>
> Elijah, _is_ there anything I can help with?

Yeah, I've been bogged down with other things too; the little Git time
I've had has been spent responding to review requests or other things
folks manually were asking for my input on.

I think I got a fair amount of this implemented about a month or so
ago.  I just pushed up what I have to the wip-for-in-core-merge-tree
branch of newren/git.  Some notes:

  * A big "WIP" commit that needs to be broken up
  * The previous "output" member of merge_result, containing a strmap
of conflict and informational messages (basically a mapping of
filename -> strbuf) now needs to be replaced by a strmap "conflicts",
which is now a mapping of primary_filename -> logical_conflicts, and
logical_conflicts is an array of logical_conflict, and
logical_conflict has a type, array of paths, and message.
  * Since "output" is no longer part of merge_result, the new
remerge-diff functionality is going to need to be modified since it
used that field, and instead iterate on "conflicts" to get the same
information
  * I have some FIXME comments in a couple places where I need to
figure out how I want to pass the variable number of arguments (in a
function already accepting a variable number of arguments for other
reasons, making the function in a way have to variable length lists of
arguments)
  * The new enums and structs I added to merge-ort.c really have to be
added to merge-ort.h and become part of the API.  Feels a little
unfortunate since it'll make the API _much_ more involved, but I don't
see any other way to solve your usecase.

If you want to take a stab at the above, or even see if my changes
make sense (sorry for it all being squashed into one big commit and
not having good commit messages, but, you know...you did ask), that'd
be great.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Tue, 17 May 2022, Elijah Newren wrote:

> On Fri, May 13, 2022 at 3:21 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Thu, 10 Mar 2022, Johannes Schindelin wrote:
> >
> > > On Tue, 8 Mar 2022, Elijah Newren wrote:
> > >
> > > > So, this is one series where even if everyone else says to merge it
> > > > already, I'd like to wait a bit longer on it until I feel confident we
> > > > have a solution that handles at least the current usecases.
> > >
> > > Fair enough, you're in charge of this series, and I really like what you
> > > came up with.
> > >
> > > My thinking was driven more by the users' side, as I am relatively eager
> > > to integrate this into production, but am loathe to do that with an early
> > > iteration of `en/merge-tree` that might be substantially revamped, still.
> >
> > I've been bogged down with things elsewhere, but should now have time to
> > help on this end.
> >
> > Elijah, _is_ there anything I can help with?
>
> Yeah, I've been bogged down with other things too; the little Git time
> I've had has been spent responding to review requests or other things
> folks manually were asking for my input on.
>
> I think I got a fair amount of this implemented about a month or so
> ago.  I just pushed up what I have to the wip-for-in-core-merge-tree
> branch of newren/git.

Thank you so much!

I worked a few hours on this and pushed up my changes under the same
branch name to dscho/git.

> Some notes:
>
>   * A big "WIP" commit that needs to be broken up

I did not yet start on that.

>   * The previous "output" member of merge_result, containing a strmap
> of conflict and informational messages (basically a mapping of
> filename -> strbuf) now needs to be replaced by a strmap "conflicts",
> which is now a mapping of primary_filename -> logical_conflicts, and
> logical_conflicts is an array of logical_conflict, and
> logical_conflict has a type, array of paths, and message.
>   * Since "output" is no longer part of merge_result, the new
> remerge-diff functionality is going to need to be modified since it
> used that field, and instead iterate on "conflicts" to get the same
> information

I punted on that for now, recreating an `output`-style strmap and storing
it as `path_messages` attribute.

>   * I have some FIXME comments in a couple places where I need to
> figure out how I want to pass the variable number of arguments (in a
> function already accepting a variable number of arguments for other
> reasons, making the function in a way have to variable length lists of
> arguments)

In my WIP fixups, I refactored this into a version that takes varargs and
another version that takes a string_list.

However, after getting all this to compile and t4301 to pass, I think we
actually only need a version that takes up to two "other" paths, and a
version that takes a string_list with those "other" paths, where the
former constructs a temporary string_list and then calls the latter.

>   * The new enums and structs I added to merge-ort.c really have to be
> added to merge-ort.h and become part of the API.  Feels a little
> unfortunate since it'll make the API _much_ more involved, but I don't
> see any other way to solve your usecase.

I agree, but I did not do that yet ;-)

Another thing I noticed is that we can probably ensure consistency between
the `conflict_and_info_types` enum and the `type_short_descriptions` array
by using the same C99 construct we're already using in the
`advice_setting` array in advice.c:

	static const char *type_short_descriptions[NB_CONFLICT_TYPES] = {
		/*** "Simple" conflicts and informational messages ***/
		[INFO_AUTO_MERGING] = "Auto-merging",
		[CONFLICT_CONTENTS] = "CONFLICT (contents)",
	[...]

> If you want to take a stab at the above, or even see if my changes
> make sense (sorry for it all being squashed into one big commit and
> not having good commit messages, but, you know...you did ask), that'd
> be great.

Yes, I did ask, and I did receive ;-)

Thank you so much! It would be great if you could have a quick look over
the commits I added on top of your branch, to see whether things make more
or less sense to you. But if you're too busy elsewhere, I am one of the
best persons to understand that, too.

Thanks!
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

On Sat, 4 Jun 2022, Johannes Schindelin wrote:

> On Tue, 17 May 2022, Elijah Newren wrote:
>
> > I think I got a fair amount of this implemented about a month or so
> > ago.  I just pushed up what I have to the wip-for-in-core-merge-tree
> > branch of newren/git.
>
> Thank you so much!
>
> I worked a few hours on this and pushed up my changes under the same
> branch name to dscho/git.

And I worked on it some more, and pushed up the result (which passes the
CI build except for some problem caused by changes outside of Git's source
code) ;-)

> > Some notes:
> >
> >   * A big "WIP" commit that needs to be broken up
>
> I did not yet start on that.

Still no start on that yet.

> >   * The previous "output" member of merge_result, containing a strmap
> > of conflict and informational messages (basically a mapping of
> > filename -> strbuf) now needs to be replaced by a strmap "conflicts",
> > which is now a mapping of primary_filename -> logical_conflicts, and
> > logical_conflicts is an array of logical_conflict, and
> > logical_conflict has a type, array of paths, and message.
> >   * Since "output" is no longer part of merge_result, the new
> > remerge-diff functionality is going to need to be modified since it
> > used that field, and instead iterate on "conflicts" to get the same
> > information
>
> I punted on that for now, recreating an `output`-style strmap and storing
> it as `path_messages` attribute.

I still punted on that because I wanted to see whether I could address the
test suite failures first (narrator's voice: he could).

> >   * I have some FIXME comments in a couple places where I need to
> > figure out how I want to pass the variable number of arguments (in a
> > function already accepting a variable number of arguments for other
> > reasons, making the function in a way have to variable length lists of
> > arguments)
>
> In my WIP fixups, I refactored this into a version that takes varargs and
> another version that takes a string_list.
>
> However, after getting all this to compile and t4301 to pass, I think we
> actually only need a version that takes up to two "other" paths, and a
> version that takes a string_list with those "other" paths, where the
> former constructs a temporary string_list and then calls the latter.

I ended up refactoring the refactor. The `path_msg()` function now takes
the "other paths" in two different forms: it accepts two (optional) `const
char*` parameters and an (also optional) `struct string_list *`. Either of
them, if non-NULL, will be added to the `struct strvec` field.

This looks _slightly_ ugly to me, but from an implementation point is the
cleanest solution.

> >   * The new enums and structs I added to merge-ort.c really have to be
> > added to merge-ort.h and become part of the API.  Feels a little
> > unfortunate since it'll make the API _much_ more involved, but I don't
> > see any other way to solve your usecase.
>
> I agree, but I did not do that yet ;-)
>
> Another thing I noticed is that we can probably ensure consistency between
> the `conflict_and_info_types` enum and the `type_short_descriptions` array
> by using the same C99 construct we're already using in the
> `advice_setting` array in advice.c:
>
> 	static const char *type_short_descriptions[NB_CONFLICT_TYPES] = {
> 		/*** "Simple" conflicts and informational messages ***/
> 		[INFO_AUTO_MERGING] = "Auto-merging",
> 		[CONFLICT_CONTENTS] = "CONFLICT (contents)",
> 	[...]

Still haven't done that, either, as it is merely syntactic sugar, really,
and not really an interesting change. I think I'll leave that to a time
after I managed to modify the remerge-diff machinery to accept the
new-style `conflicts` map (instead of recreating the old-style `output`
map, as I am doing for now).

> It would be great if you could have a quick look over the commits I
> added on top of your branch, to see whether things make more or less
> sense to you. But if you're too busy elsewhere, I am one of the best
> persons to understand that, too.

Hopefully I will get this into a reviewable shape before you get to
looking at it, so that your time is spent more wisely than what I asked
for ;-)

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

Hi Elijah,

[sorry for sending this flurry of mails, I just wasn't sure how
consecutively I could work on the `merge-tree` patches, and therefore sent
mails whenever I had to take a break in case I wouldn't be able to get
back to this project for a couple of days.]

On Sun, 5 Jun 2022, Johannes Schindelin wrote:

> On Sat, 4 Jun 2022, Johannes Schindelin wrote:
>
> > On Tue, 17 May 2022, Elijah Newren wrote:
> >
> > > * The previous "output" member of merge_result, containing a strmap
> > > of conflict and informational messages (basically a mapping of
> > > filename -> strbuf) now needs to be replaced by a strmap
> > > "conflicts", which is now a mapping of primary_filename ->
> > > logical_conflicts, and logical_conflicts is an array of
> > > logical_conflict, and logical_conflict has a type, array of paths,
> > > and message.

I massaged this a bit further: since we no longer actually need a `strbuf`
there (we no longer append to the `strbuf` but instead to the list of
logical conflicts), I replaced `struct logical_conflicts` with a
`string_list` where each item contains the conflict message and its `util`
points to a `struct logical_merge_info` that contains the `type` and the
involved paths.

This lets me...

> > >   * Since "output" is no longer part of merge_result, the new
> > > remerge-diff functionality is going to need to be modified since it
> > > used that field, and instead iterate on "conflicts" to get the same
> > > information
> >
> > I punted on that for now, recreating an `output`-style strmap and storing
> > it as `path_messages` attribute.
>
> I still punted on that because I wanted to see whether I could address the
> test suite failures first (narrator's voice: he could).

... address this issue without resorting to declaring the `logical_merge`
struct in `merge-ort.h` (which would get a bit messy, not only because we
would have to `#include <strvec.h>` in that header because that struct
contains a `struct strvec paths` and therefore must know the storage size
of `strvec`, but also because `merge-ort.h` is not included in `diff.c`,
and it would be a bit iffy to do that).

Since the per-path conflicts are now stored as a `string_list`, and since
we only need the messages in remerge, we can continue to simply pass the
pointer to `strmap conflicts` to the remerge machinery (in the common
case, if pathspecs are involved, we continue to lightly copy-filter that
`strmap`).

> > >   * The new enums and structs I added to merge-ort.c really have to be
> > > added to merge-ort.h and become part of the API.  Feels a little
> > > unfortunate since it'll make the API _much_ more involved, but I don't
> > > see any other way to solve your usecase.
> >
> > I agree, but I did not do that yet ;-)

As mentioned above, I think the better way is to _not_ declare that enum
and structs in `merge-ort.h` and instead store the per-path conflicts as a
`string_list` whose strings contain the conflict message and whose `util`
contains the type and the list of involved paths.

This simplifies the API rather dramatically, since the current user
(remerge) is only interested in the conflict message, but not in the type
nor in the full list of involved paths.

Should we ever need to access the type or the paths outside of
`merge-ort.c`, it is easy enough to add a simple function to access that
information via the `string_list`'s `util`.

> > Another thing I noticed is that we can probably ensure consistency between
> > the `conflict_and_info_types` enum and the `type_short_descriptions` array
> > by using the same C99 construct we're already using in the
> > `advice_setting` array in advice.c:
> >
> > 	static const char *type_short_descriptions[NB_CONFLICT_TYPES] = {
> > 		/*** "Simple" conflicts and informational messages ***/
> > 		[INFO_AUTO_MERGING] = "Auto-merging",
> > 		[CONFLICT_CONTENTS] = "CONFLICT (contents)",
> > 	[...]
>
> Still haven't done that, either, as it is merely syntactic sugar, really,
> and not really an interesting change. I think I'll leave that to a time
> after I managed to modify the remerge-diff machinery to accept the
> new-style `conflicts` map (instead of recreating the old-style `output`
> map, as I am doing for now).

Since I addressed that `output` issue, I now also C99-ified the
`type_short_descriptions` array.

> > It would be great if you could have a quick look over the commits I
> > added on top of your branch, to see whether things make more or less
> > sense to you. But if you're too busy elsewhere, I am one of the best
> > persons to understand that, too.
>
> Hopefully I will get this into a reviewable shape before you get to
> looking at it, so that your time is spent more wisely than what I asked
> for ;-)

I hope to find some time to work on this more tomorrow; If not, I will get
back to the project on Wednesday and push it further.

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 8966 bytes --]

Hi Elijah,

[after this update, I will shut up until you chime in, promise!]

On Mon, 6 Jun 2022, Johannes Schindelin wrote:

> On Sun, 5 Jun 2022, Johannes Schindelin wrote:
>
> > On Sat, 4 Jun 2022, Johannes Schindelin wrote:
> >
> > > It would be great if you could have a quick look over the commits I
> > > added on top of your branch, to see whether things make more or less
> > > sense to you. But if you're too busy elsewhere, I am one of the best
> > > persons to understand that, too.
> >
> > Hopefully I will get this into a reviewable shape before you get to
> > looking at it, so that your time is spent more wisely than what I
> > asked for ;-)
>
> I hope to find some time to work on this more tomorrow; If not, I will
> get back to the project on Wednesday and push it further.

I did get a chance, and polished the patch series a bit. I also rebased it
onto the current tip of Git's main branch, mainly to address some merge
conflicts preemptively. The result is pushed up to
https://github.com/dscho/git/tree/js/in-core-merge-tree. This is the
range-diff relative to `newren/wip-for-in-core-merge-tree`:

-- snip --
 1:  cccb3888070 <  -:  ----------- tmp-objdir: new API for creating temporary writable databases
 2:  4e44121c2d7 <  -:  ----------- tmp-objdir: disable ref updates when replacing the primary odb
 3:  0b94724311d <  -:  ----------- show, log: provide a --remerge-diff capability
 4:  f06de6c1b2f <  -:  ----------- log: clean unneeded objects during `log --remerge-diff`
 5:  8d6c3d48f0e <  -:  ----------- ll-merge: make callers responsible for showing warnings
 6:  de8e8f88fa4 <  -:  ----------- merge-ort: capture and print ll-merge warnings in our preferred fashion
 7:  6b535a4d55a <  -:  ----------- merge-ort: mark a few more conflict messages as omittable
 8:  e2441608c63 <  -:  ----------- merge-ort: format messages slightly different for use in headers
 9:  62734beb693 <  -:  ----------- diff: add ability to insert additional headers for paths
10:  17eccf7e0d6 <  -:  ----------- show, log: include conflict/warning messages in --remerge-diff headers
11:  b3e7656cfc6 <  -:  ----------- merge-ort: mark conflict/warning messages from inner merges as omittable
12:  ea5df61cf35 <  -:  ----------- diff-merges: avoid history simplifications when diffing merges
13:  4a7cd5542bb =  1:  8fb51817ed4 merge-tree: rename merge_trees() to trivial_merge_trees()
14:  4780ff6784d =  2:  8e0a79fa1ad merge-tree: move logic for existing merge into new function
15:  60253745f5c =  3:  baf0950bcb6 merge-tree: add option parsing and initial shell for real merge function
16:  f8266d39c1b =  4:  697470e50ae merge-tree: implement real merges
17:  6629af14919 =  5:  069af1ecc30 merge-ort: split out a separate display_update_messages() function
18:  17b57efb714 =  6:  53c92a5d8d9 merge-tree: support including merge messages in output
19:  4c8f42372dd =  7:  67a728d35f0 merge-ort: provide a merge_get_conflicted_files() helper function
25:  8fe5be07cd0 !  8:  6419487e26b merge-ort: remove command-line-centric submodule message from merge-ort
    @@ merge-ort.c: static int merge_submodule(struct merge_options *opt,
      		strbuf_release(&sb);
      		break;
      	default:
    +
    + ## t/t6437-submodule-merge.sh ##
    +@@ t/t6437-submodule-merge.sh: test_expect_success 'merging should conflict for non fast-forward' '
    + 	(cd merge-search &&
    + 	 git checkout -b test-nonforward b &&
    + 	 (cd sub &&
    +-	  git rev-parse sub-d > ../expect) &&
    ++	  git rev-parse --short sub-d > ../expect) &&
    + 	  if test "$GIT_TEST_MERGE_ALGORITHM" = ort
    + 	  then
    + 		test_must_fail git merge c >actual
20:  7b1ee417f3d !  9:  c92b81e7366 merge-tree: provide a list of which files have conflicts
    @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
     +		string_list_clear(&conflicted_files, 1);
     +	}
      	if (o->show_messages) {
    - 		printf("\n");
    +-		printf("\n");
    ++		putchar(line_termination);
      		merge_display_update_messages(&opt, &result);
    + 	}
    + 	merge_finalize(&opt, &result);
     @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)

      	/* Do the relevant type of merge */
21:  f1231a8fbc8 = 10:  d7360f58f16 merge-tree: provide easy access to `ls-files -u` style info
 -:  ----------- > 11:  b53ef9c2116 merge-ort: store messages in a list, not in a single strbuf
 -:  ----------- > 12:  b16d570d248 merge-ort: make `path_messages` a strmap to a string_list
 -:  ----------- > 13:  b575a6b5f8a merge-ort: store more specific conflict information
 -:  ----------- > 14:  4f245cc28ae merge-ort: optionally produce machine-readable output
22:  22297e6ce75 ! 15:  6a369f837be merge-tree: allow `ls-files -u` style info to be NUL terminated
    @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
      	if (!result.clean) {
      		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
      		const char *last = NULL;
    -@@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
    - 		string_list_clear(&conflicted_files, 1);
    - 	}
    - 	if (o->show_messages) {
    --		printf("\n");
    -+		putchar(line_termination);
    - 		merge_display_update_messages(&opt, &result);
    - 	}
    - 	merge_finalize(&opt, &result);
     @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
      			    N_("do a trivial merge only"), MODE_TRIVIAL),
      		OPT_BOOL(0, "messages", &o.show_messages,
    @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
     +
     +	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
     +	anonymize_hash out >actual &&
    ++	printf "\\n" >>actual &&
     +
     +	# Expected results:
     +	#   "greeting" should merge with conflicts
    @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
     +
     +	EOF
     +
    -+	cat <<-EOF >>expect &&
    -+	Auto-merging greeting
    -+	CONFLICT (content): Merge conflict in greeting
    -+	CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
    -+	CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
    -+	Auto-merging Αυτά μου φαίνονται κινέζικα
    -+	CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
    ++	q_to_nul <<-EOF >>expect &&
    ++	1QgreetingQAuto-mergingQAuto-merging greeting
    ++	Q1QgreetingQCONFLICT (contents)QCONFLICT (content): Merge conflict in greeting
    ++	Q2Qwhatever~tweak1QwhateverQCONFLICT (file/directory)QCONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
    ++	Q1Qwhatever~tweak1QCONFLICT (modify/delete)QCONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
    ++	Q1QΑυτά μου φαίνονται κινέζικαQAuto-mergingQAuto-merging Αυτά μου φαίνονται κινέζικα
    ++	Q1QΑυτά μου φαίνονται κινέζικαQCONFLICT (contents)QCONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
    ++	Q
     +	EOF
     +
     +	test_cmp expect actual
23:  db73c6dd823 = 16:  47146dd59dd merge-tree: add a --allow-unrelated-histories flag
24:  d58a7c7a9f6 ! 17:  3ce28f6fd97 git-merge-tree.txt: add a section on potentional usage mistakes
    @@ Documentation/git-merge-tree.txt: with linkgit:git-merge[1]:
     +<<IM,Informational messages>> section has the necessary info, though it
     +is not designed to be machine parseable.
     +
    ++Do NOT assume that each paths from <<CFI,Conflicted file info>>, and
    ++the logical conflicts in the <<IM,Informational messages>> have a
    ++one-to-one mapping, nor that there is a one-to-many mapping, nor a
    ++many-to-one mapping.  Many-to-many mappings exist, meaning that each
    ++path can have many logical conflict types in a single merge, and each
    ++logical conflict type can affect many paths.
    ++
     +Do NOT assume all filenames listed in the <<IM,Informational messages>>
     +section had conflicts.  Messages can be included for files that have no
     +conflicts, such as "Auto-merging <file>".
26:  78e1243eca1 <  -:  ----------- WIP
-- snap --

I am pretty happy with the current state of the patches, and hope that we
can push this patch series over the finish line.

If you can think of anything I can do to help with this, please do let me
know, I am _very_ interested in getting this done, and finally am in a
position to help.

Ciao,
Dscho

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

Hi Dscho,

On Mon, Jun 6, 2022 at 2:37 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> [after this update, I will shut up until you chime in, promise!]
>
> On Mon, 6 Jun 2022, Johannes Schindelin wrote:
>
> > On Sun, 5 Jun 2022, Johannes Schindelin wrote:
> >
> > > On Sat, 4 Jun 2022, Johannes Schindelin wrote:
> > >
> > > > It would be great if you could have a quick look over the commits I
> > > > added on top of your branch, to see whether things make more or less
> > > > sense to you. But if you're too busy elsewhere, I am one of the best
> > > > persons to understand that, too.
> > >
> > > Hopefully I will get this into a reviewable shape before you get to
> > > looking at it, so that your time is spent more wisely than what I
> > > asked for ;-)
> >
> > I hope to find some time to work on this more tomorrow; If not, I will
> > get back to the project on Wednesday and push it further.
>
> I did get a chance, and polished the patch series a bit. I also rebased it
> onto the current tip of Git's main branch, mainly to address some merge
> conflicts preemptively. The result is pushed up to
> https://github.com/dscho/git/tree/js/in-core-merge-tree. This is the
> range-diff relative to `newren/wip-for-in-core-merge-tree`:

This is so awesome; thanks for working on this.  Sorry I haven't had
time to review yet, but I'm hoping to be able to near the end of this
week.  I'm excited to see how it looks.  :-)

> -- snip --
>  1:  cccb3888070 <  -:  ----------- tmp-objdir: new API for creating temporary writable databases
>  2:  4e44121c2d7 <  -:  ----------- tmp-objdir: disable ref updates when replacing the primary odb
>  3:  0b94724311d <  -:  ----------- show, log: provide a --remerge-diff capability
>  4:  f06de6c1b2f <  -:  ----------- log: clean unneeded objects during `log --remerge-diff`
>  5:  8d6c3d48f0e <  -:  ----------- ll-merge: make callers responsible for showing warnings
>  6:  de8e8f88fa4 <  -:  ----------- merge-ort: capture and print ll-merge warnings in our preferred fashion
>  7:  6b535a4d55a <  -:  ----------- merge-ort: mark a few more conflict messages as omittable
>  8:  e2441608c63 <  -:  ----------- merge-ort: format messages slightly different for use in headers
>  9:  62734beb693 <  -:  ----------- diff: add ability to insert additional headers for paths
> 10:  17eccf7e0d6 <  -:  ----------- show, log: include conflict/warning messages in --remerge-diff headers
> 11:  b3e7656cfc6 <  -:  ----------- merge-ort: mark conflict/warning messages from inner merges as omittable
> 12:  ea5df61cf35 <  -:  ----------- diff-merges: avoid history simplifications when diffing merges
> 13:  4a7cd5542bb =  1:  8fb51817ed4 merge-tree: rename merge_trees() to trivial_merge_trees()
> 14:  4780ff6784d =  2:  8e0a79fa1ad merge-tree: move logic for existing merge into new function
> 15:  60253745f5c =  3:  baf0950bcb6 merge-tree: add option parsing and initial shell for real merge function
> 16:  f8266d39c1b =  4:  697470e50ae merge-tree: implement real merges
> 17:  6629af14919 =  5:  069af1ecc30 merge-ort: split out a separate display_update_messages() function
> 18:  17b57efb714 =  6:  53c92a5d8d9 merge-tree: support including merge messages in output
> 19:  4c8f42372dd =  7:  67a728d35f0 merge-ort: provide a merge_get_conflicted_files() helper function
> 25:  8fe5be07cd0 !  8:  6419487e26b merge-ort: remove command-line-centric submodule message from merge-ort
>     @@ merge-ort.c: static int merge_submodule(struct merge_options *opt,
>                 strbuf_release(&sb);
>                 break;
>         default:
>     +
>     + ## t/t6437-submodule-merge.sh ##
>     +@@ t/t6437-submodule-merge.sh: test_expect_success 'merging should conflict for non fast-forward' '
>     +   (cd merge-search &&
>     +    git checkout -b test-nonforward b &&
>     +    (cd sub &&
>     +-    git rev-parse sub-d > ../expect) &&
>     ++    git rev-parse --short sub-d > ../expect) &&
>     +     if test "$GIT_TEST_MERGE_ALGORITHM" = ort
>     +     then
>     +           test_must_fail git merge c >actual
> 20:  7b1ee417f3d !  9:  c92b81e7366 merge-tree: provide a list of which files have conflicts
>     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
>      +          string_list_clear(&conflicted_files, 1);
>      +  }
>         if (o->show_messages) {
>     -           printf("\n");
>     +-          printf("\n");
>     ++          putchar(line_termination);
>                 merge_display_update_messages(&opt, &result);
>     +   }
>     +   merge_finalize(&opt, &result);
>      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>
>         /* Do the relevant type of merge */
> 21:  f1231a8fbc8 = 10:  d7360f58f16 merge-tree: provide easy access to `ls-files -u` style info
>  -:  ----------- > 11:  b53ef9c2116 merge-ort: store messages in a list, not in a single strbuf
>  -:  ----------- > 12:  b16d570d248 merge-ort: make `path_messages` a strmap to a string_list
>  -:  ----------- > 13:  b575a6b5f8a merge-ort: store more specific conflict information
>  -:  ----------- > 14:  4f245cc28ae merge-ort: optionally produce machine-readable output
> 22:  22297e6ce75 ! 15:  6a369f837be merge-tree: allow `ls-files -u` style info to be NUL terminated
>     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
>         if (!result.clean) {
>                 struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
>                 const char *last = NULL;
>     -@@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
>     -           string_list_clear(&conflicted_files, 1);
>     -   }
>     -   if (o->show_messages) {
>     --          printf("\n");
>     -+          putchar(line_termination);
>     -           merge_display_update_messages(&opt, &result);
>     -   }
>     -   merge_finalize(&opt, &result);
>      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>                             N_("do a trivial merge only"), MODE_TRIVIAL),
>                 OPT_BOOL(0, "messages", &o.show_messages,
>     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
>      +
>      +  test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
>      +  anonymize_hash out >actual &&
>     ++  printf "\\n" >>actual &&
>      +
>      +  # Expected results:
>      +  #   "greeting" should merge with conflicts
>     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
>      +
>      +  EOF
>      +
>     -+  cat <<-EOF >>expect &&
>     -+  Auto-merging greeting
>     -+  CONFLICT (content): Merge conflict in greeting
>     -+  CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
>     -+  CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
>     -+  Auto-merging Αυτά μου φαίνονται κινέζικα
>     -+  CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
>     ++  q_to_nul <<-EOF >>expect &&
>     ++  1QgreetingQAuto-mergingQAuto-merging greeting
>     ++  Q1QgreetingQCONFLICT (contents)QCONFLICT (content): Merge conflict in greeting
>     ++  Q2Qwhatever~tweak1QwhateverQCONFLICT (file/directory)QCONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
>     ++  Q1Qwhatever~tweak1QCONFLICT (modify/delete)QCONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
>     ++  Q1QΑυτά μου φαίνονται κινέζικαQAuto-mergingQAuto-merging Αυτά μου φαίνονται κινέζικα
>     ++  Q1QΑυτά μου φαίνονται κινέζικαQCONFLICT (contents)QCONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
>     ++  Q
>      +  EOF
>      +
>      +  test_cmp expect actual
> 23:  db73c6dd823 = 16:  47146dd59dd merge-tree: add a --allow-unrelated-histories flag
> 24:  d58a7c7a9f6 ! 17:  3ce28f6fd97 git-merge-tree.txt: add a section on potentional usage mistakes
>     @@ Documentation/git-merge-tree.txt: with linkgit:git-merge[1]:
>      +<<IM,Informational messages>> section has the necessary info, though it
>      +is not designed to be machine parseable.
>      +
>     ++Do NOT assume that each paths from <<CFI,Conflicted file info>>, and
>     ++the logical conflicts in the <<IM,Informational messages>> have a
>     ++one-to-one mapping, nor that there is a one-to-many mapping, nor a
>     ++many-to-one mapping.  Many-to-many mappings exist, meaning that each
>     ++path can have many logical conflict types in a single merge, and each
>     ++logical conflict type can affect many paths.
>     ++
>      +Do NOT assume all filenames listed in the <<IM,Informational messages>>
>      +section had conflicts.  Messages can be included for files that have no
>      +conflicts, such as "Auto-merging <file>".
> 26:  78e1243eca1 <  -:  ----------- WIP
> -- snap --
>
> I am pretty happy with the current state of the patches, and hope that we
> can push this patch series over the finish line.
>
> If you can think of anything I can do to help with this, please do let me
> know, I am _very_ interested in getting this done, and finally am in a
> position to help.

Very much appreciated.  Looks like you're now blocking on my review,
so I'll try to make some time by end of week to look over things.

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren]

On Tue, Jun 7, 2022 at 12:38 AM Elijah Newren <newren@gmail.com> wrote:
>
> Hi Dscho,
>
> On Mon, Jun 6, 2022 at 2:37 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Hi Elijah,
> >
> > [after this update, I will shut up until you chime in, promise!]
> >
> > On Mon, 6 Jun 2022, Johannes Schindelin wrote:
> >
> > > On Sun, 5 Jun 2022, Johannes Schindelin wrote:
> > >
> > > > On Sat, 4 Jun 2022, Johannes Schindelin wrote:
> > > >
> > > > > It would be great if you could have a quick look over the commits I
> > > > > added on top of your branch, to see whether things make more or less
> > > > > sense to you. But if you're too busy elsewhere, I am one of the best
> > > > > persons to understand that, too.
> > > >
> > > > Hopefully I will get this into a reviewable shape before you get to
> > > > looking at it, so that your time is spent more wisely than what I
> > > > asked for ;-)
> > >
> > > I hope to find some time to work on this more tomorrow; If not, I will
> > > get back to the project on Wednesday and push it further.
> >
> > I did get a chance, and polished the patch series a bit. I also rebased it
> > onto the current tip of Git's main branch, mainly to address some merge
> > conflicts preemptively. The result is pushed up to
> > https://github.com/dscho/git/tree/js/in-core-merge-tree. This is the
> > range-diff relative to `newren/wip-for-in-core-merge-tree`:
>
> This is so awesome; thanks for working on this.  Sorry I haven't had
> time to review yet, but I'm hoping to be able to near the end of this
> week.  I'm excited to see how it looks.  :-)
>
> > -- snip --
> >  1:  cccb3888070 <  -:  ----------- tmp-objdir: new API for creating temporary writable databases
> >  2:  4e44121c2d7 <  -:  ----------- tmp-objdir: disable ref updates when replacing the primary odb
> >  3:  0b94724311d <  -:  ----------- show, log: provide a --remerge-diff capability
> >  4:  f06de6c1b2f <  -:  ----------- log: clean unneeded objects during `log --remerge-diff`
> >  5:  8d6c3d48f0e <  -:  ----------- ll-merge: make callers responsible for showing warnings
> >  6:  de8e8f88fa4 <  -:  ----------- merge-ort: capture and print ll-merge warnings in our preferred fashion
> >  7:  6b535a4d55a <  -:  ----------- merge-ort: mark a few more conflict messages as omittable
> >  8:  e2441608c63 <  -:  ----------- merge-ort: format messages slightly different for use in headers
> >  9:  62734beb693 <  -:  ----------- diff: add ability to insert additional headers for paths
> > 10:  17eccf7e0d6 <  -:  ----------- show, log: include conflict/warning messages in --remerge-diff headers
> > 11:  b3e7656cfc6 <  -:  ----------- merge-ort: mark conflict/warning messages from inner merges as omittable
> > 12:  ea5df61cf35 <  -:  ----------- diff-merges: avoid history simplifications when diffing merges
> > 13:  4a7cd5542bb =  1:  8fb51817ed4 merge-tree: rename merge_trees() to trivial_merge_trees()
> > 14:  4780ff6784d =  2:  8e0a79fa1ad merge-tree: move logic for existing merge into new function
> > 15:  60253745f5c =  3:  baf0950bcb6 merge-tree: add option parsing and initial shell for real merge function
> > 16:  f8266d39c1b =  4:  697470e50ae merge-tree: implement real merges
> > 17:  6629af14919 =  5:  069af1ecc30 merge-ort: split out a separate display_update_messages() function
> > 18:  17b57efb714 =  6:  53c92a5d8d9 merge-tree: support including merge messages in output
> > 19:  4c8f42372dd =  7:  67a728d35f0 merge-ort: provide a merge_get_conflicted_files() helper function
> > 25:  8fe5be07cd0 !  8:  6419487e26b merge-ort: remove command-line-centric submodule message from merge-ort
> >     @@ merge-ort.c: static int merge_submodule(struct merge_options *opt,
> >                 strbuf_release(&sb);
> >                 break;
> >         default:
> >     +
> >     + ## t/t6437-submodule-merge.sh ##
> >     +@@ t/t6437-submodule-merge.sh: test_expect_success 'merging should conflict for non fast-forward' '
> >     +   (cd merge-search &&
> >     +    git checkout -b test-nonforward b &&
> >     +    (cd sub &&
> >     +-    git rev-parse sub-d > ../expect) &&
> >     ++    git rev-parse --short sub-d > ../expect) &&
> >     +     if test "$GIT_TEST_MERGE_ALGORITHM" = ort
> >     +     then
> >     +           test_must_fail git merge c >actual
> > 20:  7b1ee417f3d !  9:  c92b81e7366 merge-tree: provide a list of which files have conflicts
> >     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
> >      +          string_list_clear(&conflicted_files, 1);
> >      +  }
> >         if (o->show_messages) {
> >     -           printf("\n");
> >     +-          printf("\n");
> >     ++          putchar(line_termination);
> >                 merge_display_update_messages(&opt, &result);
> >     +   }
> >     +   merge_finalize(&opt, &result);
> >      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >
> >         /* Do the relevant type of merge */
> > 21:  f1231a8fbc8 = 10:  d7360f58f16 merge-tree: provide easy access to `ls-files -u` style info
> >  -:  ----------- > 11:  b53ef9c2116 merge-ort: store messages in a list, not in a single strbuf
> >  -:  ----------- > 12:  b16d570d248 merge-ort: make `path_messages` a strmap to a string_list
> >  -:  ----------- > 13:  b575a6b5f8a merge-ort: store more specific conflict information
> >  -:  ----------- > 14:  4f245cc28ae merge-ort: optionally produce machine-readable output
> > 22:  22297e6ce75 ! 15:  6a369f837be merge-tree: allow `ls-files -u` style info to be NUL terminated
> >     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
> >         if (!result.clean) {
> >                 struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> >                 const char *last = NULL;
> >     -@@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
> >     -           string_list_clear(&conflicted_files, 1);
> >     -   }
> >     -   if (o->show_messages) {
> >     --          printf("\n");
> >     -+          putchar(line_termination);
> >     -           merge_display_update_messages(&opt, &result);
> >     -   }
> >     -   merge_finalize(&opt, &result);
> >      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >                             N_("do a trivial merge only"), MODE_TRIVIAL),
> >                 OPT_BOOL(0, "messages", &o.show_messages,
> >     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
> >      +
> >      +  test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
> >      +  anonymize_hash out >actual &&
> >     ++  printf "\\n" >>actual &&
> >      +
> >      +  # Expected results:
> >      +  #   "greeting" should merge with conflicts
> >     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
> >      +
> >      +  EOF
> >      +
> >     -+  cat <<-EOF >>expect &&
> >     -+  Auto-merging greeting
> >     -+  CONFLICT (content): Merge conflict in greeting
> >     -+  CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
> >     -+  CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
> >     -+  Auto-merging Αυτά μου φαίνονται κινέζικα
> >     -+  CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
> >     ++  q_to_nul <<-EOF >>expect &&
> >     ++  1QgreetingQAuto-mergingQAuto-merging greeting
> >     ++  Q1QgreetingQCONFLICT (contents)QCONFLICT (content): Merge conflict in greeting
> >     ++  Q2Qwhatever~tweak1QwhateverQCONFLICT (file/directory)QCONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
> >     ++  Q1Qwhatever~tweak1QCONFLICT (modify/delete)QCONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
> >     ++  Q1QΑυτά μου φαίνονται κινέζικαQAuto-mergingQAuto-merging Αυτά μου φαίνονται κινέζικα
> >     ++  Q1QΑυτά μου φαίνονται κινέζικαQCONFLICT (contents)QCONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
> >     ++  Q
> >      +  EOF
> >      +
> >      +  test_cmp expect actual
> > 23:  db73c6dd823 = 16:  47146dd59dd merge-tree: add a --allow-unrelated-histories flag
> > 24:  d58a7c7a9f6 ! 17:  3ce28f6fd97 git-merge-tree.txt: add a section on potentional usage mistakes
> >     @@ Documentation/git-merge-tree.txt: with linkgit:git-merge[1]:
> >      +<<IM,Informational messages>> section has the necessary info, though it
> >      +is not designed to be machine parseable.
> >      +
> >     ++Do NOT assume that each paths from <<CFI,Conflicted file info>>, and
> >     ++the logical conflicts in the <<IM,Informational messages>> have a
> >     ++one-to-one mapping, nor that there is a one-to-many mapping, nor a
> >     ++many-to-one mapping.  Many-to-many mappings exist, meaning that each
> >     ++path can have many logical conflict types in a single merge, and each
> >     ++logical conflict type can affect many paths.
> >     ++
> >      +Do NOT assume all filenames listed in the <<IM,Informational messages>>
> >      +section had conflicts.  Messages can be included for files that have no
> >      +conflicts, such as "Auto-merging <file>".
> > 26:  78e1243eca1 <  -:  ----------- WIP
> > -- snap --
> >
> > I am pretty happy with the current state of the patches, and hope that we
> > can push this patch series over the finish line.
> >
> > If you can think of anything I can do to help with this, please do let me
> > know, I am _very_ interested in getting this done, and finally am in a
> > position to help.
>
> Very much appreciated.  Looks like you're now blocking on my review,
> so I'll try to make some time by end of week to look over things.

Sorry for the delay.  However, I have looked over the changes in
detail.  Well done!  I very much like this version.  Thanks for taking
the time to break up my WIP patches, fix my bugs, address all the
FIXMEs, and then going and adding some nice cleanups and improvements
on top!  I'll add my Signed-off-by on a couple patches and have
gitgitgadget submit this round.

[PATCH v7 00/17] In-core git merge-tree ("Server side merges")

[Elijah Newren via GitGitGadget]

Sorry for the long delay. Much thanks to Johannes for tying up the loose
ends and making several improvements seen in this round. Note: rebased on
main (it's been 4 months since v6, and there were a few conflicts...).

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs. It could
be used to do merges when in a bare repository (thus potentially making it
of interest to Git hosting sites, i.e. "Server side merges"), or for doing a
merge of branches that aren't checked out.

It does not handle similar functionality for cherry-picks, rebases, or
reverts; that is also of interest, but is being deferred for a future
series.

== Quick Overview ==

 * Patches 1-2: preparatory cleanups
 * Patches 3-4: implement basic real merges
 * Patches 5-6: include informational messages ("CONFLICT" messages and
   such) in output
 * Patches 7-15: add ability to include ls-files -u style of info in the
   output
 * Patch 16: support --allow-unrelated-histories
 * Patch 17: augment the manual with potential usage mistakes

== Updates Log ==

Stuff NOT included that reviewers brought up in various rounds (and which
might still be an open question):

 * Very generic (mode, oid, stage, filename) printing formatting[3]
 * Providing similar functionality for doing cherry-picks/rebases/reverts,
   i.e. a scheme for three-way merges with a specified merge-base[4]. That's
   being deferred to a future series. [3]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [4]
   https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/

Updates since v6:

 * Significant merge-ort changes to support machine-parseable information
   about individual conflicts and the paths involved in each, rather than
   just a group of free-form output messages that may change over time. This
   includes 5 new patches, 2 written by Johannes (and with breaking out the
   other 3 from some work-in-progress changes I had including some fixes and
   improvements he added).

Updates since v5:

 * Used reverse_commit_list() to reverse a commit_list without extra
   allocations
 * Several documentation updates

Updates since v4:

 * Fixed double "is" in documentation.
 * Fixed a few small items with testcases

Updates since v3:

 * Dropped previous patches 5, 6, and 8 of the old series; they weren't
   being used and opened a can of worms[1]
 * [Patch 3] Restructured argument checking, including using an enum
 * [Patch 4] Restored the extended paragraph about the deprecated form of
   git-merge-tree, mentioned write-tree in plumbing commands, and a few
   other small fixups to the documentation
 * [Patch 4] Also provide an example of a clean merge rather than just a
   conflicted one
 * [Patch 6] Fix the incompatible arguments check and add some tests for it
 * [Patch 6] Introduce an anonymize_hash() shell function to make tests
   easier to read (less repeated sed)
 * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
   option for now
 * [Patch 10] When -z passed, the tree in the first section should have a
   trailing NUL rather than trailing newline [1]
   https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/

Updates since v2:

 * Improved patches from Dscho for the diff_warn_rename_limit() handling
 * Add a -z option for NUL-terminated conflict info lines (so that filenames
   do not have to be quoted)

Updates since v1 (or v3 depending on how you count; thanks to René, Ævar,
Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

[There were also two submissions of a previous series; see
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/]
[https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/%5D]

Updates since original submission v2 (thanks to Christian, Dscho, Ramsay,
and René for suggestions and comments):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since original submission v1 (thanks to Johannes Altmanninger and
Fabian for suggestions):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (15):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-ort: remove command-line-centric submodule message from
    merge-ort
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-ort: store more specific conflict information
  merge-ort: optionally produce machine-readable output
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

Johannes Schindelin (2):
  merge-ort: store messages in a list, not in a single strbuf
  merge-ort: make `path_messages` a strmap to a string_list

 Documentation/git-merge-tree.txt | 235 ++++++++++++++-
 builtin/merge-tree.c             | 187 +++++++++++-
 diff.c                           |  27 +-
 git.c                            |   2 +-
 merge-ort.c                      | 474 ++++++++++++++++++++++---------
 merge-ort.h                      |  32 ++-
 t/t4301-merge-tree-write-tree.sh | 240 ++++++++++++++++
 t/t6437-submodule-merge.sh       |   2 +-
 8 files changed, 1032 insertions(+), 167 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ab336e8f1c8009c8b1aab8deb592148e69217085
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v7
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v6:

  1:  4a7cd5542bb =  1:  8fb51817ed4 merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  8e0a79fa1ad merge-tree: move logic for existing merge into new function
  3:  60253745f5c =  3:  baf0950bcb6 merge-tree: add option parsing and initial shell for real merge function
  4:  f8266d39c1b =  4:  697470e50ae merge-tree: implement real merges
  5:  6629af14919 =  5:  069af1ecc30 merge-ort: split out a separate display_update_messages() function
  6:  17b57efb714 =  6:  53c92a5d8d9 merge-tree: support including merge messages in output
  7:  4c8f42372dd =  7:  67a728d35f0 merge-ort: provide a merge_get_conflicted_files() helper function
  -:  ----------- >  8:  6419487e26b merge-ort: remove command-line-centric submodule message from merge-ort
  8:  7b1ee417f3d !  9:  c92b81e7366 merge-tree: provide a list of which files have conflicts
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
      +		string_list_clear(&conflicted_files, 1);
      +	}
       	if (o->show_messages) {
     - 		printf("\n");
     +-		printf("\n");
     ++		putchar(line_termination);
       		merge_display_update_messages(&opt, &result);
     + 	}
     + 	merge_finalize(&opt, &result);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
       
       	/* Do the relevant type of merge */
  9:  f1231a8fbc8 = 10:  d7360f58f16 merge-tree: provide easy access to `ls-files -u` style info
  -:  ----------- > 11:  f523b08ab5a merge-ort: store messages in a list, not in a single strbuf
  -:  ----------- > 12:  6b47c0fdbd7 merge-ort: make `path_messages` a strmap to a string_list
  -:  ----------- > 13:  7eb70f77c81 merge-ort: store more specific conflict information
  -:  ----------- > 14:  662e97f2ed4 merge-ort: optionally produce machine-readable output
 10:  22297e6ce75 ! 15:  b314aa9c436 merge-tree: allow `ls-files -u` style info to be NUL terminated
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	if (!result.clean) {
       		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
       		const char *last = NULL;
     -@@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
     - 		string_list_clear(&conflicted_files, 1);
     - 	}
     - 	if (o->show_messages) {
     --		printf("\n");
     -+		putchar(line_termination);
     - 		merge_display_update_messages(&opt, &result);
     - 	}
     - 	merge_finalize(&opt, &result);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
       			    N_("do a trivial merge only"), MODE_TRIVIAL),
       		OPT_BOOL(0, "messages", &o.show_messages,
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
      +
      +	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
      +	anonymize_hash out >actual &&
     ++	printf "\\n" >>actual &&
      +
      +	# Expected results:
      +	#   "greeting" should merge with conflicts
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
      +
      +	EOF
      +
     -+	cat <<-EOF >>expect &&
     -+	Auto-merging greeting
     -+	CONFLICT (content): Merge conflict in greeting
     -+	CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
     -+	CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
     -+	Auto-merging Αυτά μου φαίνονται κινέζικα
     -+	CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
     ++	q_to_nul <<-EOF >>expect &&
     ++	1QgreetingQAuto-mergingQAuto-merging greeting
     ++	Q1QgreetingQCONFLICT (contents)QCONFLICT (content): Merge conflict in greeting
     ++	Q2Qwhatever~tweak1QwhateverQCONFLICT (file/directory)QCONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
     ++	Q1Qwhatever~tweak1QCONFLICT (modify/delete)QCONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
     ++	Q1QΑυτά μου φαίνονται κινέζικαQAuto-mergingQAuto-merging Αυτά μου φαίνονται κινέζικα
     ++	Q1QΑυτά μου φαίνονται κινέζικαQCONFLICT (contents)QCONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
     ++	Q
      +	EOF
      +
      +	test_cmp expect actual
 11:  db73c6dd823 = 16:  66df0c2e837 merge-tree: add a --allow-unrelated-histories flag
 12:  d58a7c7a9f6 ! 17:  70ea8281952 git-merge-tree.txt: add a section on potentional usage mistakes
     @@ Documentation/git-merge-tree.txt: with linkgit:git-merge[1]:
      +<<IM,Informational messages>> section has the necessary info, though it
      +is not designed to be machine parseable.
      +
     ++Do NOT assume that each paths from <<CFI,Conflicted file info>>, and
     ++the logical conflicts in the <<IM,Informational messages>> have a
     ++one-to-one mapping, nor that there is a one-to-many mapping, nor a
     ++many-to-one mapping.  Many-to-many mappings exist, meaning that each
     ++path can have many logical conflict types in a single merge, and each
     ++logical conflict type can affect many paths.
     ++
      +Do NOT assume all filenames listed in the <<IM,Informational messages>>
      +section had conflicts.  Messages can be included for files that have no
      +conflicts, such as "Auto-merging <file>".

-- 
gitgitgadget

[PATCH v7 01/17] merge-tree: rename merge_trees() to trivial_merge_trees()

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget

[PATCH v7 02/17] merge-tree: move logic for existing merge into new function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget

[PATCH v7 03/17] merge-tree: add option parsing and initial shell for real merge function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 84 +++++++++++++++++++++++++++++++++++++++-----
 git.c                |  2 +-
 2 files changed, 76 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..0f9d928e862 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,74 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+enum mode {
+	MODE_UNKNOWN,
+	MODE_TRIVIAL,
+	MODE_REAL,
+};
+
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    MODE_REAL),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	switch (o.mode) {
+	default:
+		BUG("unexpected command mode %d", o.mode);
+	case MODE_UNKNOWN:
+		switch (argc) {
+		default:
+			usage_with_options(merge_tree_usage, mt_options);
+		case 2:
+			o.mode = MODE_REAL;
+			break;
+		case 3:
+			o.mode = MODE_TRIVIAL;
+			break;
+		}
+		expected_remaining_argc = argc;
+		break;
+	case MODE_REAL:
+		expected_remaining_argc = 2;
+		break;
+	case MODE_TRIVIAL:
+		expected_remaining_argc = 3;
+		break;
+	}
+
+	if (argc != expected_remaining_argc)
+		usage_with_options(merge_tree_usage, mt_options);
+
+	/* Do the relevant type of merge */
+	if (o.mode == MODE_REAL)
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff4f3e25b7..861d966c374 100644
--- a/git.c
+++ b/git.c
@@ -565,7 +565,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget

[PATCH v7 04/17] merge-tree: implement real merges

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability (meaning that NEWTREE would be a lot more than a tree in the
case of conflicts).

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  98 ++++++++++++++++++++++++----
 builtin/merge-tree.c             |  41 +++++++++++-
 t/t4301-merge-tree-write-tree.sh | 106 +++++++++++++++++++++++++++++++
 3 files changed, 232 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..2a9c91328de 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,100 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
+[[NEWMERGE]]
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+This command has a modern `--write-tree` mode and a deprecated
+`--trivial-merge` mode.  With the exception of the
+<<DEPMERGE,DEPRECATED DESCRIPTION>> section at the end, the rest of
+this documentation describes modern `--write-tree` mode.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The performed merge will use the same feature as the "real"
+linkgit:git-merge[1], including:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, a new toplevel tree object is created.  See
+`OUTPUT` below for details.
+
+[[OUTPUT]]
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1 (and the output is unspecified).
+
+USAGE NOTES
+-----------
+
+This command is intended as low-level plumbing, similar to
+linkgit:git-hash-object[1], linkgit:git-mktree[1],
+linkgit:git-commit-tree[1], linkgit:git-write-tree[1],
+linkgit:git-update-ref[1], and linkgit:git-mktag[1].  Thus, it can be
+used as a part of a series of steps such as:
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+[[DEPMERGE]]
+DEPRECATED DESCRIPTION
+----------------------
+
+Per the <<NEWMERGE,DESCRIPTION>> and unlike the rest of this
+documentation, this section describes the deprecated `--trivial-merge`
+mode.
+
+Other than the optional `--trivial-merge`, this mode accepts no
+options.
+
+This mode reads three tree-ish, and outputs trivial merge results and
+conflicting stages to the standard output in a semi-diff format.
+Since this was designed for higher level scripts to consume and merge
+the results back into the index, it omits entries that match
+<branch1>.  The result of this second form is similar to what
+three-way 'git read-tree -m' does, but instead of storing the results
+in the index, the command outputs the entries to the standard output.
+
+This form not only has limited applicability (a trivial merge cannot
+handle content merges of individual files, rename detection, proper
+directory/file conflict handling, etc.), the output format is also
+difficult to work with, and it will generally be less performant than
+the first form even on successful merges (especially if working in
+large repositories).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 0f9d928e862..2332525d8bd 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -398,7 +401,43 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *merge_bases = NULL;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	merge_bases = get_merge_bases(parent1, parent2);
+	if (!merge_bases)
+		die(_("refusing to merge unrelated histories"));
+	merge_bases = reverse_commit_list(merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..6d321652e21
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,106 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "$GIT_TEST_MERGE_ALGORITHM" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+	git branch side3 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications &&
+
+	git checkout side3 &&
+	git mv numbers sequence &&
+	test_tick &&
+	git commit -m rename-numbers
+'
+
+test_expect_success 'Clean merge' '
+	TREE_OID=$(git merge-tree --write-tree side1 side3) &&
+	q_to_tab <<-EOF >expect &&
+	100644 blob $(git rev-parse side1:greeting)Qgreeting
+	100644 blob $(git rev-parse side1:numbers)Qsequence
+	100644 blob $(git rev-parse side1:whatever)Qwhatever
+	EOF
+
+	git ls-tree $TREE_OID >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(git rev-parse AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	sed -e s/HEAD/side1/ tmp >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 invalid 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget

[PATCH v7 05/17] merge-ort: split out a separate display_update_messages() function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 78 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 37 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 0d3f42592fb..b9c9e906e94 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4256,6 +4256,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4293,43 +4332,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		fclose(fp);
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
-
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v7 06/17] merge-tree: support including merge messages in output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 47 ++++++++++++++++++++++++++++----
 builtin/merge-tree.c             | 21 ++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 37 +++++++++++++++++++++++++
 3 files changed, 97 insertions(+), 8 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 2a9c91328de..25b462be14e 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 [[NEWMERGE]]
@@ -37,18 +37,50 @@ linkgit:git-merge[1], including:
 After the merge completes, a new toplevel tree object is created.  See
 `OUTPUT` below for details.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 [[OUTPUT]]
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+For a successful merge, the output from git-merge-tree is simply one
+line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+[[OIDTLT]]
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+[[IM]]
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
+
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -72,6 +104,9 @@ used as a part of a series of steps such as:
        NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
        git update-ref $BRANCH1 $NEWCOMMIT
 
+Note that when the exit status is non-zero, `NEWTREE` in this sequence
+will contain a lot more output than just a tree.
+
 [[DEPMERGE]]
 DEPRECATED DESCRIPTION
 ----------------------
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 2332525d8bd..831d9c77583 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -396,6 +396,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -435,18 +436,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -456,10 +466,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    MODE_REAL),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc - 1; /* ignoring argv[0] */
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	switch (o.mode) {
@@ -483,8 +496,12 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 		break;
 	case MODE_TRIVIAL:
 		expected_remaining_argc = 3;
+		/* Removal of `--trivial-merge` is expected */
+		original_argc--;
 		break;
 	}
+	if (o.mode == MODE_TRIVIAL && argc < original_argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	if (argc != expected_remaining_argc)
 		usage_with_options(merge_tree_usage, mt_options);
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 6d321652e21..719d81e7173 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -103,4 +103,41 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+anonymize_hash() {
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" "$@"
+}
+
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
+for opt in $(git merge-tree --git-completion-helper-all)
+do
+	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
+	then
+		continue
+	fi
+
+	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
+		test_expect_code 128 git merge-tree --trivial-merge $opt side1 side2 side3
+	'
+done
+
 test_done
-- 
gitgitgadget

[PATCH v7 07/17] merge-ort: provide a merge_get_conflicted_files() helper function

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index b9c9e906e94..1635d215c0b 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4295,6 +4295,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..ddcc39d7270 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -88,6 +89,26 @@ void merge_switch_to_result(struct merge_options *opt,
 void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget

[PATCH v7 08/17] merge-ort: remove command-line-centric submodule message from merge-ort

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

There was one case in merge-ort that would call path_msg() multiple
times for the same logical conflict, and it was in order to give advice
about how to resolve a conflict.  This advice does not make as much
sense with remerge-diff, or with merge-tree being invoked by a GitHub
GUI for resolution of messages, and is making it hard to provide
which-logical-conflict-affects-which-paths information in a machine
parseable way to a higher level caller of merge-tree.  Let's simply
remove this informational message.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c                | 9 +--------
 t/t6437-submodule-merge.sh | 2 +-
 2 files changed, 2 insertions(+), 9 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 1635d215c0b..7e8b9cd6ea7 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -1693,15 +1693,8 @@ static int merge_submodule(struct merge_options *opt,
 			      (struct commit *)merges.objects[0].item);
 		path_msg(opt, path, 0,
 			 _("Failed to merge submodule %s, but a possible merge "
-			   "resolution exists:\n%s\n"),
+			   "resolution exists: %s"),
 			 path, sb.buf);
-		path_msg(opt, path, 1,
-			 _("If this is correct simply add it to the index "
-			   "for example\n"
-			   "by using:\n\n"
-			   "  git update-index --cacheinfo 160000 %s \"%s\"\n\n"
-			   "which will accept this suggestion.\n"),
-			 oid_to_hex(&merges.objects[0].item->oid), path);
 		strbuf_release(&sb);
 		break;
 	default:
diff --git a/t/t6437-submodule-merge.sh b/t/t6437-submodule-merge.sh
index 178413c22f0..c253bf759ab 100755
--- a/t/t6437-submodule-merge.sh
+++ b/t/t6437-submodule-merge.sh
@@ -133,7 +133,7 @@ test_expect_success 'merging should conflict for non fast-forward' '
 	(cd merge-search &&
 	 git checkout -b test-nonforward b &&
 	 (cd sub &&
-	  git rev-parse sub-d > ../expect) &&
+	  git rev-parse --short sub-d > ../expect) &&
 	  if test "$GIT_TEST_MERGE_ALGORITHM" = ort
 	  then
 		test_must_fail git merge c >actual
-- 
gitgitgadget

[PATCH v7 09/17] merge-tree: provide a list of which files have conflicts

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  9 +++++++++
 builtin/merge-tree.c             | 26 +++++++++++++++++++++++---
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 43 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 25b462be14e..68a51c82618 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -58,6 +58,7 @@ line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -70,6 +71,14 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+[[CFI]]
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 [[IM]]
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 831d9c77583..13a9536f7c1 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -400,7 +403,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *merge_bases = NULL;
@@ -441,8 +445,24 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
-		printf("\n");
+		putchar(line_termination);
 		merge_display_update_messages(&opt, &result);
 	}
 	merge_finalize(&opt, &result);
@@ -508,7 +528,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == MODE_REAL)
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 719d81e7173..8e6dba44288 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -117,6 +117,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -140,4 +142,13 @@ do
 	'
 done
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v7 10/17] merge-tree: provide easy access to `ls-files -u` style info

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide a --name-only option for users to exclude the mode, oid, and
stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 34 ++++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++--
 3 files changed, 62 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 68a51c82618..b89aabdb98e 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -40,6 +40,13 @@ After the merge completes, a new toplevel tree object is created.  See
 OPTIONS
 -------
 
+--name-only::
+	In the Conflicted file info section, instead of writing a list
+	of (mode, oid, stage, path) tuples to output for conflicted
+	files, just provide a list of filenames with conflicts (and
+	do not list filenames multiple times if they have multiple
+	conflicting stages).
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -58,7 +65,7 @@ line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -72,19 +79,24 @@ working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
 [[CFI]]
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--name-only` option is passed, the mode, object, and stage will
+be omitted.
 
 [[IM]]
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -116,6 +128,16 @@ used as a part of a series of steps such as:
 Note that when the exit status is non-zero, `NEWTREE` in this sequence
 will contain a lot more output than just a tree.
 
+For conflicts, the output includes the same information that you'd get
+with linkgit:git-merge[1]:
+
+  * what would be written to the working tree (the
+    <<OIDTLT,OID of toplevel tree>>)
+  * the higher order stages that would be written to the index (the
+    <<CFI,Conflicted file info>>)
+  * any messages that would have been printed to stdout (the
+    <<IM,Informational messages>>)
+
 [[DEPMERGE]]
 DEPRECATED DESCRIPTION
 ----------------------
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 13a9536f7c1..c61b5b4a10d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -400,6 +400,7 @@ enum mode {
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int name_only;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -453,7 +454,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->name_only)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -488,6 +493,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F(0, "name-only",
+			   &o.name_only,
+			   N_("list filenames without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 8e6dba44288..0ec5f0d3f7e 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -65,6 +65,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(git rev-parse AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -108,7 +109,7 @@ anonymize_hash() {
 }
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	# Expected results:
@@ -143,7 +144,7 @@ do
 done
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --name-only side1 side2 >out &&
 	anonymize_hash out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -151,4 +152,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	anonymize_hash out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v7 11/17] merge-ort: store messages in a list, not in a single strbuf

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

To prepare for using the `merge-ort` machinery in server operations, we
cannot simply produce a free-form string that combines a variable-length
list of messages.

Instead, we need to list them one by one. The natural fit for this is a
`string_list`.

We will subsequently add even more information in the `util` attribute
of the string list items.

Based-on-a-patch-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 123 ++++++++++++++++++++++++++++++++++------------------
 merge-ort.h |   2 +-
 2 files changed, 81 insertions(+), 44 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 7e8b9cd6ea7..668aec64f13 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -349,13 +349,15 @@ struct merge_options_internal {
 	struct mem_pool pool;
 
 	/*
-	 * output: special messages and conflict notices for various paths
+	 * conflicts: logical conflicts and messages stored by _primary_ path
 	 *
 	 * This is a map of pathnames (a subset of the keys in "paths" above)
-	 * to strbufs.  It gathers various warning/conflict/notice messages
-	 * for later processing.
+	 * to struct string_list, with each item's `util` containing a
+	 * `struct logical_conflict_info`. Note, though, that for each path,
+	 * it only stores the logical conflicts for which that path is the
+	 * primary path; the path might be part of additional conflicts.
 	 */
-	struct strmap output;
+	struct strmap conflicts;
 
 	/*
 	 * renames: various data relating to rename detection
@@ -567,20 +569,20 @@ static void clear_or_reinit_internal_opts(struct merge_options_internal *opti,
 		struct strmap_entry *e;
 
 		/* Release and free each strbuf found in output */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			struct strbuf *sb = e->value;
-			strbuf_release(sb);
+		strmap_for_each_entry(&opti->conflicts, &iter, e) {
+			struct string_list *list = e->value;
 			/*
-			 * While strictly speaking we don't need to free(sb)
-			 * here because we could pass free_values=1 when
-			 * calling strmap_clear() on opti->output, that would
-			 * require strmap_clear to do another
-			 * strmap_for_each_entry() loop, so we just free it
-			 * while we're iterating anyway.
+			 * While strictly speaking we don't need to
+			 * free(conflicts) here because we could pass
+			 * free_values=1 when calling strmap_clear() on
+			 * opti->conflicts, that would require strmap_clear
+			 * to do another strmap_for_each_entry() loop, so we
+			 * just free it while we're iterating anyway.
 			 */
-			free(sb);
+			string_list_clear(list, 1);
+			free(list);
 		}
-		strmap_clear(&opti->output, 0);
+		strmap_clear(&opti->conflicts, 0);
 	}
 
 	mem_pool_discard(&opti->pool, 0);
@@ -634,7 +636,9 @@ static void path_msg(struct merge_options *opt,
 		     const char *fmt, ...)
 {
 	va_list ap;
-	struct strbuf *sb, *dest;
+	struct string_list *path_conflicts;
+	struct strbuf buf = STRBUF_INIT;
+	struct strbuf *dest;
 	struct strbuf tmp = STRBUF_INIT;
 
 	if (opt->record_conflict_msgs_as_headers && omittable_hint)
@@ -642,14 +646,16 @@ static void path_msg(struct merge_options *opt,
 	if (opt->priv->call_depth && opt->verbosity < 5)
 		return; /* Ignore messages from inner merges */
 
-	sb = strmap_get(&opt->priv->output, path);
-	if (!sb) {
-		sb = xmalloc(sizeof(*sb));
-		strbuf_init(sb, 0);
-		strmap_put(&opt->priv->output, path, sb);
+	/* Ensure path_conflicts (ptr to array of logical_conflict) allocated */
+	path_conflicts = strmap_get(&opt->priv->conflicts, path);
+	if (!path_conflicts) {
+		path_conflicts = xmalloc(sizeof(*path_conflicts));
+		string_list_init_dup(path_conflicts);
+		strmap_put(&opt->priv->conflicts, path, path_conflicts);
 	}
 
-	dest = (opt->record_conflict_msgs_as_headers ? &tmp : sb);
+	/* Handle message and its format, in normal case */
+	dest = (opt->record_conflict_msgs_as_headers ? &tmp : &buf);
 
 	va_start(ap, fmt);
 	if (opt->priv->call_depth) {
@@ -660,32 +666,31 @@ static void path_msg(struct merge_options *opt,
 	strbuf_vaddf(dest, fmt, ap);
 	va_end(ap);
 
+	/* Handle specialized formatting of message under --remerge-diff */
 	if (opt->record_conflict_msgs_as_headers) {
 		int i_sb = 0, i_tmp = 0;
 
 		/* Start with the specified prefix */
 		if (opt->msg_header_prefix)
-			strbuf_addf(sb, "%s ", opt->msg_header_prefix);
+			strbuf_addf(&buf, "%s ", opt->msg_header_prefix);
 
 		/* Copy tmp to sb, adding spaces after newlines */
-		strbuf_grow(sb, sb->len + 2*tmp.len); /* more than sufficient */
+		strbuf_grow(&buf, buf.len + 2*tmp.len); /* more than sufficient */
 		for (; i_tmp < tmp.len; i_tmp++, i_sb++) {
 			/* Copy next character from tmp to sb */
-			sb->buf[sb->len + i_sb] = tmp.buf[i_tmp];
+			buf.buf[buf.len + i_sb] = tmp.buf[i_tmp];
 
 			/* If we copied a newline, add a space */
 			if (tmp.buf[i_tmp] == '\n')
-				sb->buf[++i_sb] = ' ';
+				buf.buf[++i_sb] = ' ';
 		}
 		/* Update length and ensure it's NUL-terminated */
-		sb->len += i_sb;
-		sb->buf[sb->len] = '\0';
+		buf.len += i_sb;
+		buf.buf[buf.len] = '\0';
 
 		strbuf_release(&tmp);
 	}
-
-	/* Add final newline character to sb */
-	strbuf_addch(sb, '\n');
+	string_list_append_nodup(path_conflicts, strbuf_detach(&buf, NULL));
 }
 
 static struct diff_filespec *pool_alloc_filespec(struct mem_pool *pool,
@@ -4256,7 +4261,6 @@ void merge_display_update_messages(struct merge_options *opt,
 	struct hashmap_iter iter;
 	struct strmap_entry *e;
 	struct string_list olist = STRING_LIST_INIT_NODUP;
-	int i;
 
 	if (opt->record_conflict_msgs_as_headers)
 		BUG("Either display conflict messages or record them as headers, not both");
@@ -4264,20 +4268,20 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_enter("merge", "display messages", opt->repo);
 
 	/* Hack to pre-allocate olist to the desired size */
-	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->conflicts),
 		   olist.alloc);
 
 	/* Put every entry from output into olist, then sort */
-	strmap_for_each_entry(&opti->output, &iter, e) {
+	strmap_for_each_entry(&opti->conflicts, &iter, e) {
 		string_list_append(&olist, e->key)->util = e->value;
 	}
 	string_list_sort(&olist);
 
 	/* Iterate over the items, printing them */
-	for (i = 0; i < olist.nr; ++i) {
-		struct strbuf *sb = olist.items[i].util;
-
-		printf("%s", sb->buf);
+	for (int path_nr = 0; path_nr < olist.nr; ++path_nr) {
+		struct string_list *conflicts = olist.items[path_nr].util;
+		for (int i = 0; i < conflicts->nr; i++)
+			puts(conflicts->items[i].string);
 	}
 	string_list_clear(&olist, 0);
 
@@ -4366,6 +4370,8 @@ void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result)
 {
 	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
 
 	if (opt->renormalize)
 		git_attr_set_direction(GIT_ATTR_CHECKIN);
@@ -4373,6 +4379,15 @@ void merge_finalize(struct merge_options *opt,
 
 	clear_or_reinit_internal_opts(opti, 0);
 	FREE_AND_NULL(opti);
+
+	/* Release and free each strbuf found in path_messages */
+	strmap_for_each_entry(result->path_messages, &iter, e) {
+		struct strbuf *buf = e->value;
+
+		strbuf_release(buf);
+	}
+	strmap_clear(result->path_messages, 1);
+	FREE_AND_NULL(result->path_messages);
 }
 
 /*** Function Grouping: helper functions for merge_incore_*() ***/
@@ -4531,11 +4546,11 @@ static void merge_start(struct merge_options *opt, struct merge_result *result)
 	strmap_init_with_options(&opt->priv->conflicted, pool, 0);
 
 	/*
-	 * keys & strbufs in output will sometimes need to outlive "paths",
-	 * so it will have a copy of relevant keys.  It's probably a small
-	 * subset of the overall paths that have special output.
+	 * keys & string_lists in conflicts will sometimes need to outlive
+	 * "paths", so it will have a copy of relevant keys.  It's probably
+	 * a small subset of the overall paths that have special output.
 	 */
-	strmap_init(&opt->priv->output);
+	strmap_init(&opt->priv->conflicts);
 
 	trace2_region_leave("merge", "allocate/init", opt->repo);
 }
@@ -4596,6 +4611,8 @@ static void merge_ort_nonrecursive_internal(struct merge_options *opt,
 					    struct merge_result *result)
 {
 	struct object_id working_tree_oid;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
 
 	if (opt->subtree_shift) {
 		side2 = shift_tree_object(opt->repo, side1, side2,
@@ -4636,7 +4653,27 @@ redo:
 	trace2_region_leave("merge", "process_entries", opt->repo);
 
 	/* Set return values */
-	result->path_messages = &opt->priv->output;
+	result->path_messages = xcalloc(1, sizeof(*result->path_messages));
+	strmap_init_with_options(result->path_messages, NULL, 0);
+	strmap_for_each_entry(&opt->priv->conflicts, &iter, e) {
+		const char *path = e->key;
+		struct strbuf *buf = strmap_get(result->path_messages, path);
+		struct string_list *conflicts = e->value;
+
+		if (!buf) {
+			buf = xcalloc(1, sizeof(*buf));
+			strbuf_init(buf, 0);
+			strmap_put(result->path_messages, path, buf);
+		}
+
+		for (int i = 0; i < conflicts->nr; i++) {
+			if (buf->len)
+				strbuf_addch(buf, '\n');
+			strbuf_addstr(buf, conflicts->items[i].string);
+			strbuf_trim_trailing_newline(buf);
+		}
+	}
+
 	result->tree = parse_tree_indirect(&working_tree_oid);
 	/* existence of conflicted entries implies unclean */
 	result->clean &= strmap_empty(&opt->priv->conflicted);
diff --git a/merge-ort.h b/merge-ort.h
index ddcc39d7270..f9c536ed8c4 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -28,7 +28,7 @@ struct merge_result {
 	/*
 	 * Special messages and conflict notices for various paths
 	 *
-	 * This is a map of pathnames to strbufs.  It contains various
+	 * This is a map of pathnames to strbufs. It contains various
 	 * warning/conflict/notice messages (possibly multiple per path)
 	 * that callers may want to use.
 	 */
-- 
gitgitgadget

[PATCH v7 12/17] merge-ort: make `path_messages` a strmap to a string_list

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

This allows us once again to get away with less data copying.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 diff.c      | 27 ++++++++++++++++++++-------
 merge-ort.c | 34 +---------------------------------
 merge-ort.h |  2 +-
 3 files changed, 22 insertions(+), 41 deletions(-)

diff --git a/diff.c b/diff.c
index e71cf758861..2214ae49e4b 100644
--- a/diff.c
+++ b/diff.c
@@ -3362,23 +3362,23 @@ struct userdiff_driver *get_textconv(struct repository *r,
 	return userdiff_get_textconv(r, one->driver);
 }
 
-static struct strbuf *additional_headers(struct diff_options *o,
-					 const char *path)
+static struct string_list *additional_headers(struct diff_options *o,
+					      const char *path)
 {
 	if (!o->additional_path_headers)
 		return NULL;
 	return strmap_get(o->additional_path_headers, path);
 }
 
-static void add_formatted_headers(struct strbuf *msg,
-				  struct strbuf *more_headers,
+static void add_formatted_header(struct strbuf *msg,
+				  const char *header,
 				  const char *line_prefix,
 				  const char *meta,
 				  const char *reset)
 {
-	char *next, *newline;
+	const char *next, *newline;
 
-	for (next = more_headers->buf; *next; next = newline) {
+	for (next = header; *next; next = newline) {
 		newline = strchrnul(next, '\n');
 		strbuf_addf(msg, "%s%s%.*s%s\n", line_prefix, meta,
 			    (int)(newline - next), next, reset);
@@ -3387,6 +3387,19 @@ static void add_formatted_headers(struct strbuf *msg,
 	}
 }
 
+static void add_formatted_headers(struct strbuf *msg,
+				  struct string_list *more_headers,
+				  const char *line_prefix,
+				  const char *meta,
+				  const char *reset)
+{
+	int i;
+
+	for (i = 0; i < more_headers->nr; i++)
+		add_formatted_header(msg, more_headers->items[i].string,
+				     line_prefix, meta, reset);
+}
+
 static void builtin_diff(const char *name_a,
 			 const char *name_b,
 			 struct diff_filespec *one,
@@ -4314,7 +4327,7 @@ static void fill_metainfo(struct strbuf *msg,
 	const char *set = diff_get_color(use_color, DIFF_METAINFO);
 	const char *reset = diff_get_color(use_color, DIFF_RESET);
 	const char *line_prefix = diff_line_prefix(o);
-	struct strbuf *more_headers = NULL;
+	struct string_list *more_headers = NULL;
 
 	*must_show_header = 1;
 	strbuf_init(msg, PATH_MAX * 2 + 300);
diff --git a/merge-ort.c b/merge-ort.c
index 668aec64f13..dfec08c88be 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4370,8 +4370,6 @@ void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result)
 {
 	struct merge_options_internal *opti = result->priv;
-	struct hashmap_iter iter;
-	struct strmap_entry *e;
 
 	if (opt->renormalize)
 		git_attr_set_direction(GIT_ATTR_CHECKIN);
@@ -4379,15 +4377,6 @@ void merge_finalize(struct merge_options *opt,
 
 	clear_or_reinit_internal_opts(opti, 0);
 	FREE_AND_NULL(opti);
-
-	/* Release and free each strbuf found in path_messages */
-	strmap_for_each_entry(result->path_messages, &iter, e) {
-		struct strbuf *buf = e->value;
-
-		strbuf_release(buf);
-	}
-	strmap_clear(result->path_messages, 1);
-	FREE_AND_NULL(result->path_messages);
 }
 
 /*** Function Grouping: helper functions for merge_incore_*() ***/
@@ -4611,8 +4600,6 @@ static void merge_ort_nonrecursive_internal(struct merge_options *opt,
 					    struct merge_result *result)
 {
 	struct object_id working_tree_oid;
-	struct hashmap_iter iter;
-	struct strmap_entry *e;
 
 	if (opt->subtree_shift) {
 		side2 = shift_tree_object(opt->repo, side1, side2,
@@ -4653,26 +4640,7 @@ redo:
 	trace2_region_leave("merge", "process_entries", opt->repo);
 
 	/* Set return values */
-	result->path_messages = xcalloc(1, sizeof(*result->path_messages));
-	strmap_init_with_options(result->path_messages, NULL, 0);
-	strmap_for_each_entry(&opt->priv->conflicts, &iter, e) {
-		const char *path = e->key;
-		struct strbuf *buf = strmap_get(result->path_messages, path);
-		struct string_list *conflicts = e->value;
-
-		if (!buf) {
-			buf = xcalloc(1, sizeof(*buf));
-			strbuf_init(buf, 0);
-			strmap_put(result->path_messages, path, buf);
-		}
-
-		for (int i = 0; i < conflicts->nr; i++) {
-			if (buf->len)
-				strbuf_addch(buf, '\n');
-			strbuf_addstr(buf, conflicts->items[i].string);
-			strbuf_trim_trailing_newline(buf);
-		}
-	}
+	result->path_messages = &opt->priv->conflicts;
 
 	result->tree = parse_tree_indirect(&working_tree_oid);
 	/* existence of conflicted entries implies unclean */
diff --git a/merge-ort.h b/merge-ort.h
index f9c536ed8c4..c4909bcbf96 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -28,7 +28,7 @@ struct merge_result {
 	/*
 	 * Special messages and conflict notices for various paths
 	 *
-	 * This is a map of pathnames to strbufs. It contains various
+	 * This is a map of pathnames to a string_list. It contains various
 	 * warning/conflict/notice messages (possibly multiple per path)
 	 * that callers may want to use.
 	 */
-- 
gitgitgadget

[PATCH v7 13/17] merge-ort: store more specific conflict information

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

It is all fine and dandy for a regular Git command that is intended to
be run interactively to produce a bunch of messages upon an error.

However, in `merge-ort`'s case, we want to call the command e.g. in
server-side software, where the actual error messages are not quite as
interesting as machine-readable, immutable terms that describe the exact
nature of any given conflict.

With this patch, the `merge-ort` machinery records the exact type (as
specified via an `enum` value) as well as the involved path(s) together
with the conflict's message.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 merge-ort.c | 267 +++++++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 212 insertions(+), 55 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index dfec08c88be..432937255f6 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -483,6 +483,100 @@ struct conflict_info {
 	unsigned match_mask:3;
 };
 
+enum conflict_and_info_types {
+	/* "Simple" conflicts and informational messages */
+	INFO_AUTO_MERGING = 0,
+	CONFLICT_CONTENTS,       /* text file that failed to merge */
+	CONFLICT_BINARY,
+	CONFLICT_FILE_DIRECTORY,
+	CONFLICT_DISTINCT_MODES,
+	CONFLICT_MODIFY_DELETE,
+	CONFLICT_PRESENT_DESPITE_SKIPPED,
+
+	/* Regular rename */
+	CONFLICT_RENAME_RENAME,   /* same file renamed differently */
+	CONFLICT_RENAME_COLLIDES, /* rename/add or two files renamed to 1 */
+	CONFLICT_RENAME_DELETE,
+
+	/* Basic directory rename */
+	CONFLICT_DIR_RENAME_SUGGESTED,
+	INFO_DIR_RENAME_APPLIED,
+
+	/* Special directory rename cases */
+	INFO_DIR_RENAME_SKIPPED_DUE_TO_RERENAME,
+	CONFLICT_DIR_RENAME_FILE_IN_WAY,
+	CONFLICT_DIR_RENAME_COLLISION,
+	CONFLICT_DIR_RENAME_SPLIT,
+
+	/* Basic submodule */
+	INFO_SUBMODULE_FAST_FORWARDING,
+	CONFLICT_SUBMODULE_FAILED_TO_MERGE,
+
+	/* Special submodule cases broken out from FAILED_TO_MERGE */
+	CONFLICT_SUBMODULE_FAILED_TO_MERGE_BUT_POSSIBLE_RESOLUTION,
+	CONFLICT_SUBMODULE_NOT_INITIALIZED,
+	CONFLICT_SUBMODULE_HISTORY_NOT_AVAILABLE,
+	CONFLICT_SUBMODULE_MAY_HAVE_REWINDS,
+
+	/* Keep this entry _last_ in the list */
+	NB_CONFLICT_TYPES,
+};
+
+/*
+ * Short description of conflict type, relied upon by external tools.
+ *
+ * We can add more entries, but DO NOT change any of these strings.  Also,
+ * Order MUST match conflict_info_and_types.
+ */
+static const char *type_short_descriptions[] = {
+	/*** "Simple" conflicts and informational messages ***/
+	[INFO_AUTO_MERGING] = "Auto-merging",
+	[CONFLICT_CONTENTS] = "CONFLICT (contents)",
+	[CONFLICT_BINARY] = "CONFLICT (binary)",
+	[CONFLICT_FILE_DIRECTORY] = "CONFLICT (file/directory)",
+	[CONFLICT_DISTINCT_MODES] = "CONFLICT (distinct modes)",
+	[CONFLICT_MODIFY_DELETE] = "CONFLICT (modify/delete)",
+	[CONFLICT_PRESENT_DESPITE_SKIPPED] =
+		"CONFLICT (upgrade your version of git)",
+
+	/*** Regular rename ***/
+	[CONFLICT_RENAME_RENAME] = "CONFLICT (rename/rename)",
+	[CONFLICT_RENAME_COLLIDES] = "CONFLICT (rename involved in collision)",
+	[CONFLICT_RENAME_DELETE] = "CONFLICT (rename/delete)",
+
+	/*** Basic directory rename ***/
+	[CONFLICT_DIR_RENAME_SUGGESTED] =
+		"CONFLICT (directory rename suggested)",
+	[INFO_DIR_RENAME_APPLIED] = "Path updated due to directory rename",
+
+	/*** Special directory rename cases ***/
+	[INFO_DIR_RENAME_SKIPPED_DUE_TO_RERENAME] =
+		"Directory rename skipped since directory was renamed on both sides",
+	[CONFLICT_DIR_RENAME_FILE_IN_WAY] =
+		"CONFLICT (file in way of directory rename)",
+	[CONFLICT_DIR_RENAME_COLLISION] = "CONFLICT(directory rename collision)",
+	[CONFLICT_DIR_RENAME_SPLIT] = "CONFLICT(directory rename unclear split)",
+
+	/*** Basic submodule ***/
+	[INFO_SUBMODULE_FAST_FORWARDING] = "Fast forwarding submodule",
+	[CONFLICT_SUBMODULE_FAILED_TO_MERGE] = "CONFLICT (submodule)",
+
+	/*** Special submodule cases broken out from FAILED_TO_MERGE ***/
+	[CONFLICT_SUBMODULE_FAILED_TO_MERGE_BUT_POSSIBLE_RESOLUTION] =
+		"CONFLICT (submodule with possible resolution)",
+	[CONFLICT_SUBMODULE_NOT_INITIALIZED] =
+		"CONFLICT (submodule not initialized)",
+	[CONFLICT_SUBMODULE_HISTORY_NOT_AVAILABLE] =
+		"CONFLICT (submodule history not available)",
+	[CONFLICT_SUBMODULE_MAY_HAVE_REWINDS] =
+		"CONFLICT (submodule may have rewinds)",
+};
+
+struct logical_conflict_info {
+	enum conflict_and_info_types type;
+	struct strvec paths;
+};
+
 /*** Function Grouping: various utility functions ***/
 
 /*
@@ -571,6 +665,11 @@ static void clear_or_reinit_internal_opts(struct merge_options_internal *opti,
 		/* Release and free each strbuf found in output */
 		strmap_for_each_entry(&opti->conflicts, &iter, e) {
 			struct string_list *list = e->value;
+			for (int i = 0; i < list->nr; i++) {
+				struct logical_conflict_info *info =
+					list->items[i].util;
+				strvec_clear(&info->paths);
+			}
 			/*
 			 * While strictly speaking we don't need to
 			 * free(conflicts) here because we could pass
@@ -629,31 +728,56 @@ static void format_commit(struct strbuf *sb,
 	strbuf_addch(sb, '\n');
 }
 
-__attribute__((format (printf, 4, 5)))
+__attribute__((format (printf, 8, 9)))
 static void path_msg(struct merge_options *opt,
-		     const char *path,
+		     enum conflict_and_info_types type,
 		     int omittable_hint, /* skippable under --remerge-diff */
+		     const char *primary_path,
+		     const char *other_path_1, /* may be NULL */
+		     const char *other_path_2, /* may be NULL */
+		     struct string_list *other_paths, /* may be NULL */
 		     const char *fmt, ...)
 {
 	va_list ap;
 	struct string_list *path_conflicts;
+	struct logical_conflict_info *info;
 	struct strbuf buf = STRBUF_INIT;
 	struct strbuf *dest;
 	struct strbuf tmp = STRBUF_INIT;
 
+	/* Sanity checks */
+	assert(omittable_hint ==
+	       !starts_with(type_short_descriptions[type], "CONFLICT") ||
+	       type == CONFLICT_DIR_RENAME_SUGGESTED ||
+	       type == CONFLICT_PRESENT_DESPITE_SKIPPED);
 	if (opt->record_conflict_msgs_as_headers && omittable_hint)
 		return; /* Do not record mere hints in headers */
 	if (opt->priv->call_depth && opt->verbosity < 5)
 		return; /* Ignore messages from inner merges */
 
 	/* Ensure path_conflicts (ptr to array of logical_conflict) allocated */
-	path_conflicts = strmap_get(&opt->priv->conflicts, path);
+	path_conflicts = strmap_get(&opt->priv->conflicts, primary_path);
 	if (!path_conflicts) {
 		path_conflicts = xmalloc(sizeof(*path_conflicts));
 		string_list_init_dup(path_conflicts);
-		strmap_put(&opt->priv->conflicts, path, path_conflicts);
+		strmap_put(&opt->priv->conflicts, primary_path, path_conflicts);
 	}
 
+	/* Add a logical_conflict at the end to store info from this call */
+	info = xcalloc(1, sizeof(*info));
+	info->type = type;
+	strvec_init(&info->paths);
+
+	/* Handle the list of paths */
+	strvec_push(&info->paths, primary_path);
+	if (other_path_1)
+		strvec_push(&info->paths, other_path_1);
+	if (other_path_2)
+		strvec_push(&info->paths, other_path_2);
+	if (other_paths)
+		for (int i = 0; i < other_paths->nr; i++)
+		strvec_push(&info->paths, other_paths->items[i].string);
+
 	/* Handle message and its format, in normal case */
 	dest = (opt->record_conflict_msgs_as_headers ? &tmp : &buf);
 
@@ -690,7 +814,8 @@ static void path_msg(struct merge_options *opt,
 
 		strbuf_release(&tmp);
 	}
-	string_list_append_nodup(path_conflicts, strbuf_detach(&buf, NULL));
+	string_list_append_nodup(path_conflicts, strbuf_detach(&buf, NULL))
+		->util = info;
 }
 
 static struct diff_filespec *pool_alloc_filespec(struct mem_pool *pool,
@@ -1631,16 +1756,18 @@ static int merge_submodule(struct merge_options *opt,
 		return 0;
 
 	if (repo_submodule_init(&subrepo, opt->repo, path, null_oid())) {
-		path_msg(opt, path, 0,
-				_("Failed to merge submodule %s (not checked out)"),
-				path);
+		path_msg(opt, CONFLICT_SUBMODULE_NOT_INITIALIZED, 0,
+			 path, NULL, NULL, NULL,
+			 _("Failed to merge submodule %s (not checked out)"),
+			 path);
 		return 0;
 	}
 
 	if (!(commit_o = lookup_commit_reference(&subrepo, o)) ||
 	    !(commit_a = lookup_commit_reference(&subrepo, a)) ||
 	    !(commit_b = lookup_commit_reference(&subrepo, b))) {
-		path_msg(opt, path, 0,
+		path_msg(opt, CONFLICT_SUBMODULE_HISTORY_NOT_AVAILABLE, 0,
+			 path, NULL, NULL, NULL,
 			 _("Failed to merge submodule %s (commits not present)"),
 			 path);
 		goto cleanup;
@@ -1649,7 +1776,8 @@ static int merge_submodule(struct merge_options *opt,
 	/* check whether both changes are forward */
 	if (!repo_in_merge_bases(&subrepo, commit_o, commit_a) ||
 	    !repo_in_merge_bases(&subrepo, commit_o, commit_b)) {
-		path_msg(opt, path, 0,
+		path_msg(opt, CONFLICT_SUBMODULE_MAY_HAVE_REWINDS, 0,
+			 path, NULL, NULL, NULL,
 			 _("Failed to merge submodule %s "
 			   "(commits don't follow merge-base)"),
 			 path);
@@ -1659,7 +1787,8 @@ static int merge_submodule(struct merge_options *opt,
 	/* Case #1: a is contained in b or vice versa */
 	if (repo_in_merge_bases(&subrepo, commit_a, commit_b)) {
 		oidcpy(result, b);
-		path_msg(opt, path, 1,
+		path_msg(opt, INFO_SUBMODULE_FAST_FORWARDING, 1,
+			 path, NULL, NULL, NULL,
 			 _("Note: Fast-forwarding submodule %s to %s"),
 			 path, oid_to_hex(b));
 		ret = 1;
@@ -1667,7 +1796,8 @@ static int merge_submodule(struct merge_options *opt,
 	}
 	if (repo_in_merge_bases(&subrepo, commit_b, commit_a)) {
 		oidcpy(result, a);
-		path_msg(opt, path, 1,
+		path_msg(opt, INFO_SUBMODULE_FAST_FORWARDING, 1,
+			 path, NULL, NULL, NULL,
 			 _("Note: Fast-forwarding submodule %s to %s"),
 			 path, oid_to_hex(a));
 		ret = 1;
@@ -1690,13 +1820,16 @@ static int merge_submodule(struct merge_options *opt,
 					 &merges);
 	switch (parent_count) {
 	case 0:
-		path_msg(opt, path, 0, _("Failed to merge submodule %s"), path);
+		path_msg(opt, CONFLICT_SUBMODULE_FAILED_TO_MERGE, 0,
+			 path, NULL, NULL, NULL,
+			 _("Failed to merge submodule %s"), path);
 		break;
 
 	case 1:
 		format_commit(&sb, 4, &subrepo,
 			      (struct commit *)merges.objects[0].item);
-		path_msg(opt, path, 0,
+		path_msg(opt, CONFLICT_SUBMODULE_FAILED_TO_MERGE_BUT_POSSIBLE_RESOLUTION, 0,
+			 path, NULL, NULL, NULL,
 			 _("Failed to merge submodule %s, but a possible merge "
 			   "resolution exists: %s"),
 			 path, sb.buf);
@@ -1706,7 +1839,8 @@ static int merge_submodule(struct merge_options *opt,
 		for (i = 0; i < merges.nr; i++)
 			format_commit(&sb, 4, &subrepo,
 				      (struct commit *)merges.objects[i].item);
-		path_msg(opt, path, 0,
+		path_msg(opt, CONFLICT_SUBMODULE_FAILED_TO_MERGE_BUT_POSSIBLE_RESOLUTION, 0,
+			 path, NULL, NULL, NULL,
 			 _("Failed to merge submodule %s, but multiple "
 			   "possible merges exist:\n%s"), path, sb.buf);
 		strbuf_release(&sb);
@@ -1832,7 +1966,8 @@ static int merge_3way(struct merge_options *opt,
 				&src1, name1, &src2, name2,
 				&opt->priv->attr_index, &ll_opts);
 	if (merge_status == LL_MERGE_BINARY_CONFLICT)
-		path_msg(opt, path, 0,
+		path_msg(opt, CONFLICT_BINARY, 0,
+			 path, NULL, NULL, NULL,
 			 "warning: Cannot merge binary files: %s (%s vs. %s)",
 			 path, name1, name2);
 
@@ -1944,7 +2079,8 @@ static int handle_content_merge(struct merge_options *opt,
 		if (ret)
 			return -1;
 		clean &= (merge_status == 0);
-		path_msg(opt, path, 1, _("Auto-merging %s"), path);
+		path_msg(opt, INFO_AUTO_MERGING, 1, path, NULL, NULL, NULL,
+			 _("Auto-merging %s"), path);
 	} else if (S_ISGITLINK(a->mode)) {
 		int two_way = ((S_IFMT & o->mode) != (S_IFMT & a->mode));
 		clean = merge_submodule(opt, pathnames[0],
@@ -2082,21 +2218,24 @@ static char *handle_path_level_conflicts(struct merge_options *opt,
 		c_info->reported_already = 1;
 		strbuf_add_separated_string_list(&collision_paths, ", ",
 						 &c_info->source_files);
-		path_msg(opt, new_path, 0,
-			 _("CONFLICT (implicit dir rename): Existing file/dir "
-			   "at %s in the way of implicit directory rename(s) "
-			   "putting the following path(s) there: %s."),
-		       new_path, collision_paths.buf);
+		path_msg(opt, CONFLICT_DIR_RENAME_FILE_IN_WAY, 0,
+			 new_path, NULL, NULL, &c_info->source_files,
+			 _("CONFLICT (implicit dir rename): Existing "
+			   "file/dir at %s in the way of implicit "
+			   "directory rename(s) putting the following "
+			   "path(s) there: %s."),
+			 new_path, collision_paths.buf);
 		clean = 0;
 	} else if (c_info->source_files.nr > 1) {
 		c_info->reported_already = 1;
 		strbuf_add_separated_string_list(&collision_paths, ", ",
 						 &c_info->source_files);
-		path_msg(opt, new_path, 0,
-			 _("CONFLICT (implicit dir rename): Cannot map more "
-			   "than one path to %s; implicit directory renames "
-			   "tried to put these paths there: %s"),
-		       new_path, collision_paths.buf);
+		path_msg(opt, CONFLICT_DIR_RENAME_COLLISION, 0,
+			 new_path, NULL, NULL, &c_info->source_files,
+			 _("CONFLICT (implicit dir rename): Cannot map "
+			   "more than one path to %s; implicit directory "
+			   "renames tried to put these paths there: %s"),
+			 new_path, collision_paths.buf);
 		clean = 0;
 	}
 
@@ -2150,13 +2289,14 @@ static void get_provisional_directory_renames(struct merge_options *opt,
 			continue;
 
 		if (bad_max == max) {
-			path_msg(opt, source_dir, 0,
-			       _("CONFLICT (directory rename split): "
-				 "Unclear where to rename %s to; it was "
-				 "renamed to multiple other directories, with "
-				 "no destination getting a majority of the "
-				 "files."),
-			       source_dir);
+			path_msg(opt, CONFLICT_DIR_RENAME_SPLIT, 0,
+				 source_dir, NULL, NULL, NULL,
+				 _("CONFLICT (directory rename split): "
+				   "Unclear where to rename %s to; it was "
+				   "renamed to multiple other directories, "
+				   "with no destination getting a majority of "
+				   "the files."),
+				 source_dir);
 			*clean = 0;
 		} else {
 			strmap_put(&renames->dir_renames[side],
@@ -2304,7 +2444,8 @@ static char *check_for_directory_rename(struct merge_options *opt,
 	 */
 	otherinfo = strmap_get_entry(dir_rename_exclusions, new_dir);
 	if (otherinfo) {
-		path_msg(opt, rename_info->key, 1,
+		path_msg(opt, INFO_DIR_RENAME_SKIPPED_DUE_TO_RERENAME, 1,
+			 rename_info->key, path, new_dir, NULL,
 			 _("WARNING: Avoiding applying %s -> %s rename "
 			   "to %s, because %s itself was renamed."),
 			 rename_info->key, new_dir, path, new_dir);
@@ -2444,14 +2585,16 @@ static void apply_directory_rename_modifications(struct merge_options *opt,
 	if (opt->detect_directory_renames == MERGE_DIRECTORY_RENAMES_TRUE) {
 		/* Notify user of updated path */
 		if (pair->status == 'A')
-			path_msg(opt, new_path, 1,
+			path_msg(opt, INFO_DIR_RENAME_APPLIED, 1,
+				 new_path, old_path, NULL, NULL,
 				 _("Path updated: %s added in %s inside a "
 				   "directory that was renamed in %s; moving "
 				   "it to %s."),
 				 old_path, branch_with_new_path,
 				 branch_with_dir_rename, new_path);
 		else
-			path_msg(opt, new_path, 1,
+			path_msg(opt, INFO_DIR_RENAME_APPLIED, 1,
+				 new_path, old_path, NULL, NULL,
 				 _("Path updated: %s renamed to %s in %s, "
 				   "inside a directory that was renamed in %s; "
 				   "moving it to %s."),
@@ -2464,7 +2607,8 @@ static void apply_directory_rename_modifications(struct merge_options *opt,
 		 */
 		ci->path_conflict = 1;
 		if (pair->status == 'A')
-			path_msg(opt, new_path, 1,
+			path_msg(opt, CONFLICT_DIR_RENAME_SUGGESTED, 1,
+				 new_path, old_path, NULL, NULL,
 				 _("CONFLICT (file location): %s added in %s "
 				   "inside a directory that was renamed in %s, "
 				   "suggesting it should perhaps be moved to "
@@ -2472,7 +2616,8 @@ static void apply_directory_rename_modifications(struct merge_options *opt,
 				 old_path, branch_with_new_path,
 				 branch_with_dir_rename, new_path);
 		else
-			path_msg(opt, new_path, 1,
+			path_msg(opt, CONFLICT_DIR_RENAME_SUGGESTED, 1,
+				 new_path, old_path, NULL, NULL,
 				 _("CONFLICT (file location): %s renamed to %s "
 				   "in %s, inside a directory that was renamed "
 				   "in %s, suggesting it should perhaps be "
@@ -2628,7 +2773,8 @@ static int process_renames(struct merge_options *opt,
 			 * and remove the setting of base->path_conflict to 1.
 			 */
 			base->path_conflict = 1;
-			path_msg(opt, oldpath, 0,
+			path_msg(opt, CONFLICT_RENAME_RENAME, 0,
+				 pathnames[0], pathnames[1], pathnames[2], NULL,
 				 _("CONFLICT (rename/rename): %s renamed to "
 				   "%s in %s and to %s in %s."),
 				 pathnames[0],
@@ -2723,7 +2869,8 @@ static int process_renames(struct merge_options *opt,
 			memcpy(&newinfo->stages[target_index], &merged,
 			       sizeof(merged));
 			if (!clean) {
-				path_msg(opt, newpath, 0,
+				path_msg(opt, CONFLICT_RENAME_COLLIDES, 0,
+					 newpath, oldpath, NULL, NULL,
 					 _("CONFLICT (rename involved in "
 					   "collision): rename of %s -> %s has "
 					   "content conflicts AND collides "
@@ -2742,7 +2889,8 @@ static int process_renames(struct merge_options *opt,
 			 */
 
 			newinfo->path_conflict = 1;
-			path_msg(opt, newpath, 0,
+			path_msg(opt, CONFLICT_RENAME_DELETE, 0,
+				 newpath, oldpath, NULL, NULL,
 				 _("CONFLICT (rename/delete): %s renamed "
 				   "to %s in %s, but deleted in %s."),
 				 oldpath, newpath, rename_branch, delete_branch);
@@ -2766,7 +2914,8 @@ static int process_renames(struct merge_options *opt,
 			} else if (source_deleted) {
 				/* rename/delete */
 				newinfo->path_conflict = 1;
-				path_msg(opt, newpath, 0,
+				path_msg(opt, CONFLICT_RENAME_DELETE, 0,
+					 newpath, oldpath, NULL, NULL,
 					 _("CONFLICT (rename/delete): %s renamed"
 					   " to %s in %s, but deleted in %s."),
 					 oldpath, newpath,
@@ -3687,7 +3836,8 @@ static void process_entry(struct merge_options *opt,
 		path = unique_path(opt, path, branch);
 		strmap_put(&opt->priv->paths, path, new_ci);
 
-		path_msg(opt, path, 0,
+		path_msg(opt, CONFLICT_FILE_DIRECTORY, 0,
+			 path, old_path, NULL, NULL,
 			 _("CONFLICT (file/directory): directory in the way "
 			   "of %s from %s; moving it to %s instead."),
 			 old_path, branch, path);
@@ -3763,15 +3913,23 @@ static void process_entry(struct merge_options *opt,
 				rename_b = 1;
 			}
 
+			if (rename_a)
+				a_path = unique_path(opt, path, opt->branch1);
+			if (rename_b)
+				b_path = unique_path(opt, path, opt->branch2);
+
 			if (rename_a && rename_b) {
-				path_msg(opt, path, 0,
+				path_msg(opt, CONFLICT_DISTINCT_MODES, 0,
+					 path, a_path, b_path, NULL,
 					 _("CONFLICT (distinct types): %s had "
 					   "different types on each side; "
 					   "renamed both of them so each can "
 					   "be recorded somewhere."),
 					 path);
 			} else {
-				path_msg(opt, path, 0,
+				path_msg(opt, CONFLICT_DISTINCT_MODES, 0,
+					 path, rename_a ? a_path : b_path,
+					 NULL, NULL,
 					 _("CONFLICT (distinct types): %s had "
 					   "different types on each side; "
 					   "renamed one of them so each can be "
@@ -3808,14 +3966,10 @@ static void process_entry(struct merge_options *opt,
 
 			/* Insert entries into opt->priv_paths */
 			assert(rename_a || rename_b);
-			if (rename_a) {
-				a_path = unique_path(opt, path, opt->branch1);
+			if (rename_a)
 				strmap_put(&opt->priv->paths, a_path, ci);
-			}
 
-			if (rename_b)
-				b_path = unique_path(opt, path, opt->branch2);
-			else
+			if (!rename_b)
 				b_path = path;
 			strmap_put(&opt->priv->paths, b_path, new_ci);
 
@@ -3866,7 +4020,8 @@ static void process_entry(struct merge_options *opt,
 				reason = _("add/add");
 			if (S_ISGITLINK(merged_file.mode))
 				reason = _("submodule");
-			path_msg(opt, path, 0,
+			path_msg(opt, CONFLICT_CONTENTS, 0,
+				 path, NULL, NULL, NULL,
 				 _("CONFLICT (%s): Merge conflict in %s"),
 				 reason, path);
 		}
@@ -3910,7 +4065,8 @@ static void process_entry(struct merge_options *opt,
 			 * since the contents were not modified.
 			 */
 		} else {
-			path_msg(opt, path, 0,
+			path_msg(opt, CONFLICT_MODIFY_DELETE, 0,
+				 path, NULL, NULL, NULL,
 				 _("CONFLICT (modify/delete): %s deleted in %s "
 				   "and modified in %s.  Version %s of %s left "
 				   "in tree."),
@@ -4206,7 +4362,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 								     path,
 								     "cruft");
 
-					path_msg(opt, path, 1,
+					path_msg(opt, CONFLICT_PRESENT_DESPITE_SKIPPED, 1,
+						 path, NULL, NULL, NULL,
 						 _("Note: %s not up to date and in way of checking out conflicted version; old copy renamed to %s"),
 						 path, new_name);
 					errs |= rename(path, new_name);
-- 
gitgitgadget

[PATCH v7 14/17] merge-ort: optionally produce machine-readable output

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

With the new `detailed` parameter, a new mode can be triggered when
displaying the merge messages: The `detailed` mode prints NUL-delimited
fields of the following form:

	<path-count> NUL <path>... NUL <conflict-type> NUL <message>

The `<path-count>` field determines how many `<path>` fields there are.

The intention of this mode is to support server-side operations, where
worktree-less merges can lead to conflicts and depending on the type
and/or path count, the caller might know how to handle said conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 builtin/merge-tree.c |  3 ++-
 merge-ort.c          | 22 ++++++++++++++++++++--
 merge-ort.h          |  1 +
 3 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index c61b5b4a10d..b3c5692498e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -468,7 +468,8 @@ static int real_merge(struct merge_tree_options *o,
 	}
 	if (o->show_messages) {
 		putchar(line_termination);
-		merge_display_update_messages(&opt, &result);
+		merge_display_update_messages(&opt, line_termination == '\0',
+					      &result);
 	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
diff --git a/merge-ort.c b/merge-ort.c
index 432937255f6..4a56c189ddf 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4412,6 +4412,7 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 }
 
 void merge_display_update_messages(struct merge_options *opt,
+				   int detailed,
 				   struct merge_result *result)
 {
 	struct merge_options_internal *opti = result->priv;
@@ -4437,8 +4438,25 @@ void merge_display_update_messages(struct merge_options *opt,
 	/* Iterate over the items, printing them */
 	for (int path_nr = 0; path_nr < olist.nr; ++path_nr) {
 		struct string_list *conflicts = olist.items[path_nr].util;
-		for (int i = 0; i < conflicts->nr; i++)
+		for (int i = 0; i < conflicts->nr; i++) {
+			struct logical_conflict_info *info =
+				conflicts->items[i].util;
+
+			if (detailed) {
+				printf("%lu", (unsigned long)info->paths.nr);
+				putchar('\0');
+				for (int n = 0; n < info->paths.nr; n++) {
+					fputs(info->paths.v[n], stdout);
+					putchar('\0');
+				}
+				fputs(type_short_descriptions[info->type],
+				      stdout);
+				putchar('\0');
+			}
 			puts(conflicts->items[i].string);
+			if (detailed)
+				putchar('\0');
+		}
 	}
 	string_list_clear(&olist, 0);
 
@@ -4518,7 +4536,7 @@ void merge_switch_to_result(struct merge_options *opt,
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
 	if (display_update_msgs)
-		merge_display_update_messages(opt, result);
+		merge_display_update_messages(opt, /* detailed */ 0, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index c4909bcbf96..a994c9a5fcd 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -87,6 +87,7 @@ void merge_switch_to_result(struct merge_options *opt,
  * so only call this when bypassing merge_switch_to_result().
  */
 void merge_display_update_messages(struct merge_options *opt,
+				   int detailed,
 				   struct merge_result *result);
 
 struct stage_info {
-- 
gitgitgadget

[PATCH v7 15/17] merge-tree: allow `ls-files -u` style info to be NUL terminated

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Much as `git ls-files` has a -z option, let's add one to merge-tree so
that the conflict-info section can be NUL terminated (and avoid quoting
of unusual filenames).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 21 +++++++++++++---
 builtin/merge-tree.c             |  4 ++-
 t/t4301-merge-tree-write-tree.sh | 42 ++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index b89aabdb98e..75b57f8abab 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -40,6 +40,12 @@ After the merge completes, a new toplevel tree object is created.  See
 OPTIONS
 -------
 
+-z::
+	Do not quote filenames in the <Conflicted file info> section,
+	and end each filename with a NUL character rather than
+	newline.  Also begin the messages section with a NUL character
+	instead of a newline.  See <<OUTPUT>> below for more information.
+
 --name-only::
 	In the Conflicted file info section, instead of writing a list
 	of (mode, oid, stage, path) tuples to output for conflicted
@@ -76,7 +82,8 @@ OID of toplevel tree
 
 This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
-files within this tree may have embedded conflict markers.
+files within this tree may have embedded conflict markers.  This section
+is always followed by a newline (or NUL if `-z` is passed).
 
 [[CFI]]
 Conflicted file info
@@ -89,20 +96,26 @@ This is a sequence of lines with the format
 The filename will be quoted as explained for the configuration
 variable `core.quotePath` (see linkgit:git-config[1]).  However, if
 the `--name-only` option is passed, the mode, object, and stage will
-be omitted.
+be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
+character instead of a newline character.
 
 [[IM]]
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
-This always starts with a blank line to separate it from the previous
-sections, and then has free-form messages about the merge, such as:
+This always starts with a blank line (or NUL if `-z` is passed) to
+separate it from the previous sections, and then has free-form
+messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
   * "Failed to merge submodule <submodule> (<reason>)"
   * "Warning: cannot merge binary files: <filename>"
 
+Note that these free-form messages will never have a NUL character
+in or between them, even if -z is passed.  It is simply a large block
+of text taking up the remainder of the output.
+
 EXIT STATUS
 -----------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index b3c5692498e..c159e317743 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -445,7 +445,7 @@ static int real_merge(struct merge_tree_options *o,
 	if (o->show_messages == -1)
 		o->show_messages = !result.clean;
 
-	puts(oid_to_hex(&result.tree->object.oid));
+	printf("%s%c", oid_to_hex(&result.tree->object.oid), line_termination);
 	if (!result.clean) {
 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
 		const char *last = NULL;
@@ -494,6 +494,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), MODE_TRIVIAL),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_SET_INT('z', NULL, &line_termination,
+			    N_("separate paths with the NUL character"), '\0'),
 		OPT_BOOL_F(0, "name-only",
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 0ec5f0d3f7e..88e75b18cc5 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -173,4 +173,46 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'NUL terminated conflicted file "lines"' '
+	git checkout -b tweak1 side1 &&
+	test_write_lines zero 1 2 3 4 5 6 >numbers &&
+	git add numbers &&
+	git mv numbers "Αυτά μου φαίνονται κινέζικα" &&
+	git commit -m "Renamed numbers" &&
+
+	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
+	anonymize_hash out >actual &&
+	printf "\\n" >>actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
+	echo HASH | lf_to_nul >expect &&
+
+	q_to_tab <<-EOF | lf_to_nul >>expect &&
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~tweak1
+	100644 HASH 2Qwhatever~tweak1
+	100644 HASH 1QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 2QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 3QΑυτά μου φαίνονται κινέζικα
+
+	EOF
+
+	q_to_nul <<-EOF >>expect &&
+	1QgreetingQAuto-mergingQAuto-merging greeting
+	Q1QgreetingQCONFLICT (contents)QCONFLICT (content): Merge conflict in greeting
+	Q2Qwhatever~tweak1QwhateverQCONFLICT (file/directory)QCONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
+	Q1Qwhatever~tweak1QCONFLICT (modify/delete)QCONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
+	Q1QΑυτά μου φαίνονται κινέζικαQAuto-mergingQAuto-merging Αυτά μου φαίνονται κινέζικα
+	Q1QΑυτά μου φαίνονται κινέζικαQCONFLICT (contents)QCONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
+	Q
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v7 16/17] merge-tree: add a --allow-unrelated-histories flag

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 75b57f8abab..628324646d3 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -59,6 +59,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 [[OUTPUT]]
 OUTPUT
 ------
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index c159e317743..ae5782917b9 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -399,6 +399,7 @@ enum mode {
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int name_only;
 };
@@ -434,7 +435,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	merge_bases = get_merge_bases(parent1, parent2);
-	if (!merge_bases)
+	if (!merge_bases && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	merge_bases = reverse_commit_list(merge_bases);
 
@@ -500,6 +501,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.name_only,
 			   N_("list filenames without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 88e75b18cc5..f091259a55e 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -44,7 +44,13 @@ test_expect_success setup '
 	git checkout side3 &&
 	git mv numbers sequence &&
 	test_tick &&
-	git commit -m rename-numbers
+	git commit -m rename-numbers &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Clean merge' '
@@ -215,4 +221,20 @@ test_expect_success 'NUL terminated conflicted file "lines"' '
 	test_cmp expect actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget

[PATCH v7 17/17] git-merge-tree.txt: add a section on potentional usage mistakes

[Elijah Newren via GitGitGadget]

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 53 ++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 628324646d3..d6c356740ef 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -156,6 +156,59 @@ with linkgit:git-merge[1]:
   * any messages that would have been printed to stdout (the
     <<IM,Informational messages>>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <<CFI,Conflicted file info>> section instead.
+Not only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <<CFI,Conflicted file info>> list as a clean
+merge; check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <<CFI,Conflicted file info>> list.  The information there is
+insufficient to do so.  For example: Rename/rename(1to2) conflicts (both
+sides renamed the same file differently) will result in three different
+file having higher order stages (but each only has one higher order
+stage), with no way (short of the <<IM,Informational messages>> section)
+to determine which three files are related.  File/directory conflicts
+also result in a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<<IM,Informational messages>> section has the necessary info, though it
+is not designed to be machine parseable.
+
+Do NOT assume that each paths from <<CFI,Conflicted file info>>, and
+the logical conflicts in the <<IM,Informational messages>> have a
+one-to-one mapping, nor that there is a one-to-many mapping, nor a
+many-to-one mapping.  Many-to-many mappings exist, meaning that each
+path can have many logical conflict types in a single merge, and each
+logical conflict type can affect many paths.
+
+Do NOT assume all filenames listed in the <<IM,Informational messages>>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <<CFI,Conflicted file info>> and
+re-merging them to present the conflicts to the user.  This will lose
+information.  Instead, look up the version of the file found within the
+<<OIDTLT,OID of toplevel tree>> and show that instead.  In particular,
+the latter will have conflict markers annotated with the original
+branch/commit being merged and, if renames were involved, the original
+filename.  While you could include the original branch/commit in the
+conflict marker annotations when re-merging, the original filename is
+not available from the <<CFI,Conflicted file info>> and thus you would
+be losing information that might help the user resolve the conflict.
+
 [[DEPMERGE]]
 DEPRECATED DESCRIPTION
 ----------------------
-- 
gitgitgadget

Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 11064 bytes --]

Hi Elijah,

On Fri, 17 Jun 2022, Elijah Newren wrote:

> On Tue, Jun 7, 2022 at 12:38 AM Elijah Newren <newren@gmail.com> wrote:
> >
> > On Mon, Jun 6, 2022 at 2:37 PM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > [after this update, I will shut up until you chime in, promise!]
> > >
> > > On Mon, 6 Jun 2022, Johannes Schindelin wrote:
> > >
> > > > On Sun, 5 Jun 2022, Johannes Schindelin wrote:
> > > >
> > > > > On Sat, 4 Jun 2022, Johannes Schindelin wrote:
> > > > >
> > > > > > It would be great if you could have a quick look over the commits I
> > > > > > added on top of your branch, to see whether things make more or less
> > > > > > sense to you. But if you're too busy elsewhere, I am one of the best
> > > > > > persons to understand that, too.
> > > > >
> > > > > Hopefully I will get this into a reviewable shape before you get to
> > > > > looking at it, so that your time is spent more wisely than what I
> > > > > asked for ;-)
> > > >
> > > > I hope to find some time to work on this more tomorrow; If not, I will
> > > > get back to the project on Wednesday and push it further.
> > >
> > > I did get a chance, and polished the patch series a bit. I also rebased it
> > > onto the current tip of Git's main branch, mainly to address some merge
> > > conflicts preemptively. The result is pushed up to
> > > https://github.com/dscho/git/tree/js/in-core-merge-tree. This is the
> > > range-diff relative to `newren/wip-for-in-core-merge-tree`:
> >
> > This is so awesome; thanks for working on this.  Sorry I haven't had
> > time to review yet, but I'm hoping to be able to near the end of this
> > week.  I'm excited to see how it looks.  :-)
> >
> > > -- snip --
> > >  1:  cccb3888070 <  -:  ----------- tmp-objdir: new API for creating temporary writable databases
> > >  2:  4e44121c2d7 <  -:  ----------- tmp-objdir: disable ref updates when replacing the primary odb
> > >  3:  0b94724311d <  -:  ----------- show, log: provide a --remerge-diff capability
> > >  4:  f06de6c1b2f <  -:  ----------- log: clean unneeded objects during `log --remerge-diff`
> > >  5:  8d6c3d48f0e <  -:  ----------- ll-merge: make callers responsible for showing warnings
> > >  6:  de8e8f88fa4 <  -:  ----------- merge-ort: capture and print ll-merge warnings in our preferred fashion
> > >  7:  6b535a4d55a <  -:  ----------- merge-ort: mark a few more conflict messages as omittable
> > >  8:  e2441608c63 <  -:  ----------- merge-ort: format messages slightly different for use in headers
> > >  9:  62734beb693 <  -:  ----------- diff: add ability to insert additional headers for paths
> > > 10:  17eccf7e0d6 <  -:  ----------- show, log: include conflict/warning messages in --remerge-diff headers
> > > 11:  b3e7656cfc6 <  -:  ----------- merge-ort: mark conflict/warning messages from inner merges as omittable
> > > 12:  ea5df61cf35 <  -:  ----------- diff-merges: avoid history simplifications when diffing merges
> > > 13:  4a7cd5542bb =  1:  8fb51817ed4 merge-tree: rename merge_trees() to trivial_merge_trees()
> > > 14:  4780ff6784d =  2:  8e0a79fa1ad merge-tree: move logic for existing merge into new function
> > > 15:  60253745f5c =  3:  baf0950bcb6 merge-tree: add option parsing and initial shell for real merge function
> > > 16:  f8266d39c1b =  4:  697470e50ae merge-tree: implement real merges
> > > 17:  6629af14919 =  5:  069af1ecc30 merge-ort: split out a separate display_update_messages() function
> > > 18:  17b57efb714 =  6:  53c92a5d8d9 merge-tree: support including merge messages in output
> > > 19:  4c8f42372dd =  7:  67a728d35f0 merge-ort: provide a merge_get_conflicted_files() helper function
> > > 25:  8fe5be07cd0 !  8:  6419487e26b merge-ort: remove command-line-centric submodule message from merge-ort
> > >     @@ merge-ort.c: static int merge_submodule(struct merge_options *opt,
> > >                 strbuf_release(&sb);
> > >                 break;
> > >         default:
> > >     +
> > >     + ## t/t6437-submodule-merge.sh ##
> > >     +@@ t/t6437-submodule-merge.sh: test_expect_success 'merging should conflict for non fast-forward' '
> > >     +   (cd merge-search &&
> > >     +    git checkout -b test-nonforward b &&
> > >     +    (cd sub &&
> > >     +-    git rev-parse sub-d > ../expect) &&
> > >     ++    git rev-parse --short sub-d > ../expect) &&
> > >     +     if test "$GIT_TEST_MERGE_ALGORITHM" = ort
> > >     +     then
> > >     +           test_must_fail git merge c >actual
> > > 20:  7b1ee417f3d !  9:  c92b81e7366 merge-tree: provide a list of which files have conflicts
> > >     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
> > >      +          string_list_clear(&conflicted_files, 1);
> > >      +  }
> > >         if (o->show_messages) {
> > >     -           printf("\n");
> > >     +-          printf("\n");
> > >     ++          putchar(line_termination);
> > >                 merge_display_update_messages(&opt, &result);
> > >     +   }
> > >     +   merge_finalize(&opt, &result);
> > >      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> > >
> > >         /* Do the relevant type of merge */
> > > 21:  f1231a8fbc8 = 10:  d7360f58f16 merge-tree: provide easy access to `ls-files -u` style info
> > >  -:  ----------- > 11:  b53ef9c2116 merge-ort: store messages in a list, not in a single strbuf
> > >  -:  ----------- > 12:  b16d570d248 merge-ort: make `path_messages` a strmap to a string_list
> > >  -:  ----------- > 13:  b575a6b5f8a merge-ort: store more specific conflict information
> > >  -:  ----------- > 14:  4f245cc28ae merge-ort: optionally produce machine-readable output
> > > 22:  22297e6ce75 ! 15:  6a369f837be merge-tree: allow `ls-files -u` style info to be NUL terminated
> > >     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
> > >         if (!result.clean) {
> > >                 struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> > >                 const char *last = NULL;
> > >     -@@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
> > >     -           string_list_clear(&conflicted_files, 1);
> > >     -   }
> > >     -   if (o->show_messages) {
> > >     --          printf("\n");
> > >     -+          putchar(line_termination);
> > >     -           merge_display_update_messages(&opt, &result);
> > >     -   }
> > >     -   merge_finalize(&opt, &result);
> > >      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> > >                             N_("do a trivial merge only"), MODE_TRIVIAL),
> > >                 OPT_BOOL(0, "messages", &o.show_messages,
> > >     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
> > >      +
> > >      +  test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
> > >      +  anonymize_hash out >actual &&
> > >     ++  printf "\\n" >>actual &&
> > >      +
> > >      +  # Expected results:
> > >      +  #   "greeting" should merge with conflicts
> > >     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
> > >      +
> > >      +  EOF
> > >      +
> > >     -+  cat <<-EOF >>expect &&
> > >     -+  Auto-merging greeting
> > >     -+  CONFLICT (content): Merge conflict in greeting
> > >     -+  CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
> > >     -+  CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
> > >     -+  Auto-merging Αυτά μου φαίνονται κινέζικα
> > >     -+  CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
> > >     ++  q_to_nul <<-EOF >>expect &&
> > >     ++  1QgreetingQAuto-mergingQAuto-merging greeting
> > >     ++  Q1QgreetingQCONFLICT (contents)QCONFLICT (content): Merge conflict in greeting
> > >     ++  Q2Qwhatever~tweak1QwhateverQCONFLICT (file/directory)QCONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
> > >     ++  Q1Qwhatever~tweak1QCONFLICT (modify/delete)QCONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
> > >     ++  Q1QΑυτά μου φαίνονται κινέζικαQAuto-mergingQAuto-merging Αυτά μου φαίνονται κινέζικα
> > >     ++  Q1QΑυτά μου φαίνονται κινέζικαQCONFLICT (contents)QCONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
> > >     ++  Q
> > >      +  EOF
> > >      +
> > >      +  test_cmp expect actual
> > > 23:  db73c6dd823 = 16:  47146dd59dd merge-tree: add a --allow-unrelated-histories flag
> > > 24:  d58a7c7a9f6 ! 17:  3ce28f6fd97 git-merge-tree.txt: add a section on potentional usage mistakes
> > >     @@ Documentation/git-merge-tree.txt: with linkgit:git-merge[1]:
> > >      +<<IM,Informational messages>> section has the necessary info, though it
> > >      +is not designed to be machine parseable.
> > >      +
> > >     ++Do NOT assume that each paths from <<CFI,Conflicted file info>>, and
> > >     ++the logical conflicts in the <<IM,Informational messages>> have a
> > >     ++one-to-one mapping, nor that there is a one-to-many mapping, nor a
> > >     ++many-to-one mapping.  Many-to-many mappings exist, meaning that each
> > >     ++path can have many logical conflict types in a single merge, and each
> > >     ++logical conflict type can affect many paths.
> > >     ++
> > >      +Do NOT assume all filenames listed in the <<IM,Informational messages>>
> > >      +section had conflicts.  Messages can be included for files that have no
> > >      +conflicts, such as "Auto-merging <file>".
> > > 26:  78e1243eca1 <  -:  ----------- WIP
> > > -- snap --
> > >
> > > I am pretty happy with the current state of the patches, and hope that we
> > > can push this patch series over the finish line.
> > >
> > > If you can think of anything I can do to help with this, please do let me
> > > know, I am _very_ interested in getting this done, and finally am in a
> > > position to help.
> >
> > Very much appreciated.  Looks like you're now blocking on my review,
> > so I'll try to make some time by end of week to look over things.
>
> Sorry for the delay.  However, I have looked over the changes in
> detail.  Well done!  I very much like this version.  Thanks for taking
> the time to break up my WIP patches, fix my bugs, address all the
> FIXMEs, and then going and adding some nice cleanups and improvements
> on top!  I'll add my Signed-off-by on a couple patches and have
> gitgitgadget submit this round.

That's great news, thank you so much!

Ciao,
Dscho