git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
@ 2022-01-22 21:55 Elijah Newren via GitGitGadget
  2022-01-22 21:55 ` [PATCH 01/12] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
                   ` (14 more replies)
  0 siblings, 15 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren

Note1: Depends on en/remerge-diff (but no need to pick it up; it's still
RFC).

(Note2: This is a continuation of my series at [0], but I can't change the
base repository for a pull request in GitHub so I had to open a new PR and
that makes it look like a new series. [0]
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/)

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs.

== Updates Log ==

Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
suggestions and comments on v2):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since v1 (thanks to Johannes Altmanninger and Fabian for suggestions
on v1):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

== Other notes ==

Stuff intentionally NOT included, but which others seemed to feel strongly
about; they'd need to convince me more on these:

 * Any form of diff output[1]
 * A way to omit printing the generated tree hash[2][3] In regards to these,
   also see also some of the new info in Documentation/git-merge-tree.txt,
   namely the expanded paragraph on "the second form is deprecated" in the
   description as regards diff output usability and performance, and also
   the final paragraph of the new "Mistakes to avoid" section in regards to
   tree hash.

[1]
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2201101427440.339@tvgsbejvaqbjf.bet/
(section starting with "Providing a tree") [2]
https://lore.kernel.org/git/CABPp-BHvXrP0sTTmuTYfACoJTCcm9+wk_f441nj4TstrmQdqMQ@mail.gmail.com/
(sections starting with "avoid printing" and "Where did I suggest") [3]
https://lore.kernel.org/git/CABPp-BGdbh=HM7jA+_RTwSWVcMr_qvEY7RoNXooeBT2NB4Ubzw@mail.gmail.com/
(section starting with "Providing a tree object")

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-ort: allow update messages to be written to different file
    stream
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

 Documentation/git-merge-tree.txt | 182 +++++++++++++++++++++++++++++--
 builtin/merge-tree.c             | 178 ++++++++++++++++++++++++++++--
 git.c                            |   2 +-
 merge-ort.c                      | 109 ++++++++++++------
 merge-ort.h                      |  30 +++++
 t/t4301-merge-tree-real.sh       | 163 +++++++++++++++++++++++++++
 6 files changed, 603 insertions(+), 61 deletions(-)
 create mode 100755 t/t4301-merge-tree-real.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1122
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 01/12] merge-tree: rename merge_trees() to trivial_merge_trees()
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-22 21:55 ` [PATCH 02/12] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
                   ` (13 subsequent siblings)
  14 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 02/12] merge-tree: move logic for existing merge into new function
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  2022-01-22 21:55 ` [PATCH 01/12] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-22 21:55 ` [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  2022-01-22 21:55 ` [PATCH 01/12] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
  2022-01-22 21:55 ` [PATCH 02/12] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-23  8:05   ` René Scharfe
  2022-01-24  9:46   ` Ævar Arnfjörð Bjarmason
  2022-01-22 21:55 ` [PATCH 04/12] merge-tree: implement real merges Elijah Newren via GitGitGadget
                   ` (11 subsequent siblings)
  14 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 67 ++++++++++++++++++++++++++++++++++++++------
 git.c                |  2 +-
 2 files changed, 59 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..33e47cc1534 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,57 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+struct merge_tree_options {
+	int real;
+	int trivial;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_BOOL(0, "write-tree", &o.real,
+			 N_("do a real merge instead of a trivial merge")),
+		OPT_BOOL(0, "trivial-merge", &o.trivial,
+			 N_("do a trivial merge only")),
+		OPT_END()
+	};
+
+	/* Check for a request for basic help */
+	if (argc == 2 && !strcmp(argv[1], "-h"))
+		usage_with_options(merge_tree_usage, mt_options);
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, 0);
+	if (o.real && o.trivial)
+		die(_("--write-tree and --trivial-merge are incompatible"));
+	if (o.real || o.trivial) {
+		expected_remaining_argc = (o.real ? 2 : 3);
+		if (argc != expected_remaining_argc)
+			usage_with_options(merge_tree_usage, mt_options);
+	} else {
+		if (argc < 2 || argc > 3)
+			usage_with_options(merge_tree_usage, mt_options);
+		o.real = (argc == 2);
+	}
+
+	/* Do the relevant type of merge */
+	if (o.real)
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 04/12] merge-tree: implement real merges
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (2 preceding siblings ...)
  2022-01-22 21:55 ` [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-24  9:51   ` Ævar Arnfjörð Bjarmason
                     ` (2 more replies)
  2022-01-22 21:55 ` [PATCH 05/12] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
                   ` (10 subsequent siblings)
  14 siblings, 3 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean) or 1 (conflicts present)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 74 ++++++++++++++++++++++-----
 builtin/merge-tree.c             | 55 +++++++++++++++++++-
 t/t4301-merge-tree-real.sh       | 87 ++++++++++++++++++++++++++++++++
 3 files changed, 203 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-real.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..b900bc1362c 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,76 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The first form will merge the two branches, doing a full recursive
+merge with rename detection.  The rest of this manual (other than the
+next paragraph) describes the first form in more detail -- including
+options, output format, exit status, and usage notes.
+
+The second form is deprecated; it is kept for backward compatibility
+reasons but may be deleted in the future.  It will only do a trivial
+merge.  It reads three tree-ish, and outputs trivial merge results and
+conflicting stages to the standard output in a semi-diff format.
+Since this was designed for higher level scripts to consume and merge
+the results back into the index, it omits entries that match
+<branch1>.  The result of this second form is is similar to what
+three-way 'git read-tree -m' does, but instead of storing the results
+in the index, the command outputs the entries to the standard output.
+This form not only has limited applicability, the output format is
+also difficult to work with, and it will generally be less performant
+than the first form even on successful merges (especially if working
+in large repositories).  The remainder of this manual will only
+discuss the first form.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1.
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
+be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+However, it does not quite fit into the same category of low-level
+plumbing commands since the possibility of merge conflicts give it a
+much higher chance of the command not succeeding.
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 33e47cc1534..0c19639594d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -393,7 +396,57 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+	/*
+	 * TODO: Support subtree and other -X options?
+	if (use_strategies_nr == 1 &&
+	    !strcmp(use_strategies[0]->name, "subtree"))
+		opt.subtree_shift = "";
+	for (x = 0; x < xopts_nr; x++)
+		if (parse_merge_opt(&opt, xopts[x]))
+			die(_("Unknown strategy option: -X%s"), xopts[x]);
+	*/
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
+	opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	else if (!result.clean)
+		printf(_("Conflicts!\n"));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
new file mode 100755
index 00000000000..e03688515c5
--- /dev/null
+++ b/t/t4301-merge-tree-real.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+}
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(cat .git/AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	cat tmp | sed -e s/HEAD/side1/ >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 05/12] merge-ort: split out a separate display_update_messages() function
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (3 preceding siblings ...)
  2022-01-22 21:55 ` [PATCH 04/12] merge-tree: implement real merges Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-24  9:56   ` Ævar Arnfjörð Bjarmason
  2022-01-28 16:09   ` Johannes Schindelin
  2022-01-22 21:55 ` [PATCH 06/12] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
                   ` (9 subsequent siblings)
  14 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

No functional changes included in this patch; it's just a preparatory
step to allow the printed messages to be handled differently by other
callers, such as in `git merge-tree --write-tree`.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 77 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 36 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..f9e35b0f96b 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4273,42 +4312,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
 
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 06/12] merge-ort: allow update messages to be written to different file stream
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (4 preceding siblings ...)
  2022-01-22 21:55 ` [PATCH 05/12] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-28 16:31   ` Johannes Schindelin
  2022-01-22 21:55 ` [PATCH 07/12] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
                   ` (8 subsequent siblings)
  14 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This modifies the new display_update_messages() function to allow
printing to somewhere other than stdout.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 7 ++++---
 merge-ort.h | 3 ++-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index f9e35b0f96b..b78dde55ad9 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 }
 
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result)
+				   struct merge_result *result,
+				   FILE *stream)
 {
 	struct merge_options_internal *opti = result->priv;
 	struct hashmap_iter iter;
@@ -4263,7 +4264,7 @@ void merge_display_update_messages(struct merge_options *opt,
 	for (i = 0; i < olist.nr; ++i) {
 		struct strbuf *sb = olist.items[i].util;
 
-		printf("%s", sb->buf);
+		fprintf(stream, "%s", sb->buf);
 	}
 	string_list_clear(&olist, 0);
 
@@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
 	}
 
 	if (display_update_msgs)
-		merge_display_update_messages(opt, result);
+		merge_display_update_messages(opt, result, stdout);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..d643b47cb7c 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
  * so only call this when bypassing merge_switch_to_result().
  */
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result);
+				   struct merge_result *result,
+				   FILE *stream);
 
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 07/12] merge-tree: support including merge messages in output
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (5 preceding siblings ...)
  2022-01-22 21:55 ` [PATCH 06/12] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-26 10:42   ` Christian Couder
  2022-01-28 16:37   ` Johannes Schindelin
  2022-01-22 21:55 ` [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
                   ` (7 subsequent siblings)
  14 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
conflicts occur (such as file/directory, modify/delete, add/add with
differing modes, rename/rename (1to2), etc.), these informational
messages are often the only notification since these conflicts are not
representable in the contents of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 24 +++++++++++++----
 t/t4301-merge-tree-real.sh       | 21 +++++++++++++++
 3 files changed, 78 insertions(+), 12 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index b900bc1362c..fd7a867de60 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
 
 DESCRIPTION
@@ -38,17 +38,47 @@ than the first form even on successful merges (especially if working
 in large repositories).  The remainder of this manual will only
 discuss the first form.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
+
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -72,7 +102,8 @@ be used as a part of a series of steps such as
 
 However, it does not quite fit into the same category of low-level
 plumbing commands since the possibility of merge conflicts give it a
-much higher chance of the command not succeeding.
+much higher chance of the command not succeeding (and NEWTREE containing
+a bunch of stuff other than just a toplevel tree).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 0c19639594d..560640ad911 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -391,6 +391,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int real;
 	int trivial;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
 		commit_list_insert(j->item, &merge_bases);
 
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
-	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+
 	if (result.clean < 0)
 		die(_("failure to merge"));
-	else if (!result.clean)
-		printf(_("Conflicts!\n"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result, stdout);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -464,6 +473,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			 N_("do a real merge instead of a trivial merge")),
 		OPT_BOOL(0, "trivial-merge", &o.trivial,
 			 N_("do a trivial merge only")),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
@@ -472,10 +483,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 		usage_with_options(merge_tree_usage, mt_options);
 
 	/* Parse arguments */
+	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, 0);
 	if (o.real && o.trivial)
 		die(_("--write-tree and --trivial-merge are incompatible"));
+	if (!o.real && original_argc < argc)
+		die(_("--write-tree must be specified if any other options are"));
 	if (o.real || o.trivial) {
 		expected_remaining_argc = (o.real ? 2 : 3);
 		if (argc != expected_remaining_argc)
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index e03688515c5..c34f8e6c1ed 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -84,4 +84,25 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (6 preceding siblings ...)
  2022-01-22 21:55 ` [PATCH 07/12] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-26 10:55   ` Christian Couder
                     ` (2 more replies)
  2022-01-22 21:55 ` [PATCH 09/12] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
                   ` (6 subsequent siblings)
  14 siblings, 3 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u` -- conflicted
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index b78dde55ad9..5e7cea6cc8f 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index d643b47cb7c..e635a294ea8 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result,
 				   FILE *stream);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 09/12] merge-tree: provide a list of which files have conflicts
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (7 preceding siblings ...)
  2022-01-22 21:55 ` [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
@ 2022-01-22 21:55 ` Elijah Newren via GitGitGadget
  2022-01-24 10:01   ` Ævar Arnfjörð Bjarmason
  2022-01-28 16:57   ` Johannes Schindelin
  2022-01-22 21:56 ` [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
                   ` (5 subsequent siblings)
  14 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:55 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-real.sh       | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index fd7a867de60..041a4ac2785 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -58,6 +58,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -69,6 +70,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 560640ad911..d8eeeb3f306 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -395,7 +398,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	printf("%s\n", oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result, stdout);
@@ -502,7 +522,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.real)
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index c34f8e6c1ed..43c9950dedb 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -94,6 +94,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -105,4 +107,13 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (8 preceding siblings ...)
  2022-01-22 21:55 ` [PATCH 09/12] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
@ 2022-01-22 21:56 ` Elijah Newren via GitGitGadget
  2022-01-24 10:06   ` Ævar Arnfjörð Bjarmason
  2022-01-22 21:56 ` [PATCH 11/12] merge-tree: add a --allow-unrelated-histories flag Elijah Newren via GitGitGadget
                   ` (4 subsequent siblings)
  14 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:56 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide an --exclude-oids-and-modes option for users to exclude the
mode, oid, and stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 30 ++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-real.sh       | 26 ++++++++++++++++++++++++--
 3 files changed, 58 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 041a4ac2785..beb08269a70 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -41,6 +41,11 @@ discuss the first form.
 OPTIONS
 -------
 
+--exclude-oids-and-modes::
+	Instead of writing a list of (mode, oid, stage, path) tuples
+	to output for conflicted files, just provide a list of
+	filenames with conflicts.
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -58,7 +63,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -70,18 +75,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--exclude-oids-and-modes` option is passed, the mode, object, and
+stage will be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -113,6 +123,14 @@ plumbing commands since the possibility of merge conflicts give it a
 much higher chance of the command not succeeding (and NEWTREE containing
 a bunch of stuff other than just a toplevel tree).
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d8eeeb3f306..7aa7f9fd54a 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -395,6 +395,7 @@ struct merge_tree_options {
 	int real;
 	int trivial;
 	int show_messages;
+	int exclude_oids_and_modes;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -461,7 +462,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->exclude_oids_and_modes)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -495,6 +500,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			 N_("do a trivial merge only")),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F(0, "exclude-oids-and-modes",
+			   &o.exclude_oids_and_modes,
+			   N_("list conflicted files without oids and modes"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index 43c9950dedb..e921115cd2a 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -46,6 +46,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(cat .git/AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -85,7 +86,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --exclude-oids-and-modes side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -108,7 +109,7 @@ test_expect_success 'test conflict notices and such' '
 '
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-oids-and-modes side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -116,4 +117,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 11/12] merge-tree: add a --allow-unrelated-histories flag
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (9 preceding siblings ...)
  2022-01-22 21:56 ` [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
@ 2022-01-22 21:56 ` Elijah Newren via GitGitGadget
  2022-01-22 21:56 ` [PATCH 12/12] git-merge-tree.txt: add a section on potentional usage mistakes Elijah Newren via GitGitGadget
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:56 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-real.sh       | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index beb08269a70..df10a5963c7 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -52,6 +52,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 7aa7f9fd54a..98441d5e05b 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -394,6 +394,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int real;
 	int trivial;
+	int allow_unrelated_histories;
 	int show_messages;
 	int exclude_oids_and_modes;
 };
@@ -440,7 +441,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -504,6 +505,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.exclude_oids_and_modes,
 			   N_("list conflicted files without oids and modes"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
index e921115cd2a..a0447410655 100755
--- a/t/t4301-merge-tree-real.sh
+++ b/t/t4301-merge-tree-real.sh
@@ -37,7 +37,13 @@ test_expect_success setup '
 	>whatever/empty &&
 	git add numbers greeting whatever/empty &&
 	test_tick &&
-	git commit -m other-modifications
+	git commit -m other-modifications &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Content merge and a few conflicts' '
@@ -138,4 +144,20 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH 12/12] git-merge-tree.txt: add a section on potentional usage mistakes
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (10 preceding siblings ...)
  2022-01-22 21:56 ` [PATCH 11/12] merge-tree: add a --allow-unrelated-histories flag Elijah Newren via GitGitGadget
@ 2022-01-22 21:56 ` Elijah Newren via GitGitGadget
  2022-01-26  8:48 ` [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Christian Couder
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-22 21:56 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index df10a5963c7..3c1aa0ffbae 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -136,6 +136,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function
  2022-01-22 21:55 ` [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
@ 2022-01-23  8:05   ` René Scharfe
  2022-01-24 16:43     ` Elijah Newren
  2022-01-24  9:46   ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 215+ messages in thread
From: René Scharfe @ 2022-01-23  8:05 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget, git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	Elijah Newren

Am 22.01.22 um 22:55 schrieb Elijah Newren via GitGitGadget:
> From: Elijah Newren <newren@gmail.com>
>
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.
>
> Note that real merges differ from trivial merges in that they handle:
>   - three way content merges
>   - recursive ancestor consolidation
>   - renames
>   - proper directory/file conflict handling
>   - etc.
> Basically all the stuff you'd expect from `git merge`, just without
> updating the index and working tree.  The initial shell added here does
> nothing more than die with "real merges are not yet implemented", but
> that will be fixed in subsequent commits.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/merge-tree.c | 67 ++++++++++++++++++++++++++++++++++++++------
>  git.c                |  2 +-
>  2 files changed, 59 insertions(+), 10 deletions(-)
>
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 914ec960b7e..33e47cc1534 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -3,13 +3,12 @@
>  #include "tree-walk.h"
>  #include "xdiff-interface.h"
>  #include "object-store.h"
> +#include "parse-options.h"
>  #include "repository.h"
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>
> -static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
> -
>  struct merge_list {
>  	struct merge_list *next;
>  	struct merge_list *link;	/* other stages for this object */
> @@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
>  	return buf;
>  }
>
> -static int trivial_merge(int argc, const char **argv)
> +static int trivial_merge(const char *base,
> +			 const char *branch1,
> +			 const char *branch2)
>  {
>  	struct repository *r = the_repository;
>  	struct tree_desc t[3];
>  	void *buf1, *buf2, *buf3;
>
> -	buf1 = get_tree_descriptor(r, t+0, argv[1]);
> -	buf2 = get_tree_descriptor(r, t+1, argv[2]);
> -	buf3 = get_tree_descriptor(r, t+2, argv[3]);
> +	buf1 = get_tree_descriptor(r, t+0, base);
> +	buf2 = get_tree_descriptor(r, t+1, branch1);
> +	buf3 = get_tree_descriptor(r, t+2, branch2);
>  	trivial_merge_trees(t, "");
>  	free(buf1);
>  	free(buf2);
> @@ -384,9 +385,57 @@ static int trivial_merge(int argc, const char **argv)
>  	return 0;
>  }
>
> +struct merge_tree_options {
> +	int real;
> +	int trivial;
> +};
> +
> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_BOOL(0, "write-tree", &o.real,
> +			 N_("do a real merge instead of a trivial merge")),
> +		OPT_BOOL(0, "trivial-merge", &o.trivial,
> +			 N_("do a trivial merge only")),
> +		OPT_END()
> +	};
> +
> +	/* Check for a request for basic help */
> +	if (argc == 2 && !strcmp(argv[1], "-h"))
> +		usage_with_options(merge_tree_usage, mt_options);

This is unnecessary; parse_options() handles -h already.

> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, 0);
> +	if (o.real && o.trivial)
> +		die(_("--write-tree and --trivial-merge are incompatible"));

12909b6b8a (i18n: turn "options are incompatible" into "cannot be used
together", 2022-01-05) standardized messages of that kind; let's stick
to that to simplify translation:

		die(_("options '%s' and '%s' cannot be used together"),
		    "--write-tree", "--trivial-merge");

> +	if (o.real || o.trivial) {
> +		expected_remaining_argc = (o.real ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.real = (argc == 2);
> +	}
> +
> +	/* Do the relevant type of merge */
> +	if (o.real)
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/git.c b/git.c
> index 5ff21be21f3..6090a1289db 100644
> --- a/git.c
> +++ b/git.c
> @@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
>  	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
> -	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
> +	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
>  	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
>  	{ "mktree", cmd_mktree, RUN_SETUP },
>  	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function
  2022-01-22 21:55 ` [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
  2022-01-23  8:05   ` René Scharfe
@ 2022-01-24  9:46   ` Ævar Arnfjörð Bjarmason
  2022-01-24 16:54     ` Elijah Newren
  1 sibling, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-01-24  9:46 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren


On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.
>
> Note that real merges differ from trivial merges in that they handle:
>   - three way content merges
>   - recursive ancestor consolidation
>   - renames
>   - proper directory/file conflict handling
>   - etc.
> Basically all the stuff you'd expect from `git merge`, just without
> updating the index and working tree.  The initial shell added here does
> nothing more than die with "real merges are not yet implemented", but
> that will be fixed in subsequent commits.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/merge-tree.c | 67 ++++++++++++++++++++++++++++++++++++++------
>  git.c                |  2 +-
>  2 files changed, 59 insertions(+), 10 deletions(-)
>
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 914ec960b7e..33e47cc1534 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -3,13 +3,12 @@
>  #include "tree-walk.h"
>  #include "xdiff-interface.h"
>  #include "object-store.h"
> +#include "parse-options.h"
>  #include "repository.h"
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>  
> -static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
> -
>  struct merge_list {
>  	struct merge_list *next;
>  	struct merge_list *link;	/* other stages for this object */
> @@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
>  	return buf;
>  }
>  
> -static int trivial_merge(int argc, const char **argv)
> +static int trivial_merge(const char *base,
> +			 const char *branch1,
> +			 const char *branch2)
>  {
>  	struct repository *r = the_repository;
>  	struct tree_desc t[3];
>  	void *buf1, *buf2, *buf3;
>  
> -	buf1 = get_tree_descriptor(r, t+0, argv[1]);
> -	buf2 = get_tree_descriptor(r, t+1, argv[2]);
> -	buf3 = get_tree_descriptor(r, t+2, argv[3]);
> +	buf1 = get_tree_descriptor(r, t+0, base);
> +	buf2 = get_tree_descriptor(r, t+1, branch1);
> +	buf3 = get_tree_descriptor(r, t+2, branch2);
>  	trivial_merge_trees(t, "");
>  	free(buf1);
>  	free(buf2);
> @@ -384,9 +385,57 @@ static int trivial_merge(int argc, const char **argv)
>  	return 0;
>  }
>  
> +struct merge_tree_options {
> +	int real;
> +	int trivial;
> +};
> +
> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_BOOL(0, "write-tree", &o.real,
> +			 N_("do a real merge instead of a trivial merge")),
> +		OPT_BOOL(0, "trivial-merge", &o.trivial,
> +			 N_("do a trivial merge only")),
> +		OPT_END()
> +	};
> +
> +	/* Check for a request for basic help */
> +	if (argc == 2 && !strcmp(argv[1], "-h"))
> +		usage_with_options(merge_tree_usage, mt_options);

Is this bit cargo-culted from something else, perhaps
non-parse-options.c usage? I don't think this is needed, the
parse_options() below intercepts "-h" by default.

> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, 0);
> +	if (o.real && o.trivial)
> +		die(_("--write-tree and --trivial-merge are incompatible"));

Shouldn't those two just be OPT_CMDMODE()? Then you get this
incompatibility checking for free. See 485fd2c3dae (cat-file: make
--batch-all-objects a CMDMODE, 2021-12-28).

> +	if (o.real || o.trivial) {
> +		expected_remaining_argc = (o.real ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.real = (argc == 2);
> +	}

And this can also be done like this, but I wonder if using
PARSE_OPT_STOP_AT_NON_OPTION and then routing to a sub-function wouldn't
be better, i.e. to treat these like sub-commands if they've got
different arity etc.

> +	/* Do the relevant type of merge */
> +	if (o.real)
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/git.c b/git.c
> index 5ff21be21f3..6090a1289db 100644
> --- a/git.c
> +++ b/git.c
> @@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
>  	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
> -	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
> +	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
>  	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
>  	{ "mktree", cmd_mktree, RUN_SETUP },
>  	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },


^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 04/12] merge-tree: implement real merges
  2022-01-22 21:55 ` [PATCH 04/12] merge-tree: implement real merges Elijah Newren via GitGitGadget
@ 2022-01-24  9:51   ` Ævar Arnfjörð Bjarmason
  2022-01-24 17:12     ` Elijah Newren
  2022-01-25 17:07   ` Johannes Schindelin
  2022-01-26  9:44   ` Christian Couder
  2 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-01-24  9:51 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren


On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> +	/*
> +	 * TODO: Support subtree and other -X options?
> +	if (use_strategies_nr == 1 &&
> +	    !strcmp(use_strategies[0]->name, "subtree"))
> +		opt.subtree_shift = "";
> +	for (x = 0; x < xopts_nr; x++)
> +		if (parse_merge_opt(&opt, xopts[x]))

Better omitted WIP code in a non-RFC series?

> +			die(_("Unknown strategy option: -X%s"), xopts[x]);

As a general issue with this series, die(), BUG() etc. messages should
start with a non-capital letter.

> +	printf("%s\n", oid_to_hex(&result.tree->object.oid));

And for both this...

> +		printf(_("Conflicts!\n"));

... and this we can just use puts(). For the former it's just less code,
but for the latter translators also don't need to see the always-there
\n in the translated message.

> +# This test is ort-specific
> +test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {

Is this ${} trickery really needed? We're not testing with "set -u". So just:
	
	if test "$GIT_..." != "ort"
	then
		...
	fi

> +test_expect_success 'Barf on too many arguments' '
> +	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
> +
> +	grep "^usage: git merge-tree" expect
> +'

Nit: In most other tests these usage assertions are at the top of the
test, and for those we also make do with just testing the 129 exit code,
which is probably enough here too...

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 05/12] merge-ort: split out a separate display_update_messages() function
  2022-01-22 21:55 ` [PATCH 05/12] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
@ 2022-01-24  9:56   ` Ævar Arnfjörð Bjarmason
  2022-01-25  1:59     ` Elijah Newren
  2022-01-28 16:09   ` Johannes Schindelin
  1 sibling, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-01-24  9:56 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren


On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
> [...]
> +	/* Hack to pre-allocate olist to the desired size */
> +	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> +		   olist.alloc);

Perhaps just add a string_list_grow()? But I wonder if this is really
needed v.s. just using the default growing pattern here.

> +
> +	/* Put every entry from output into olist, then sort */
> +	strmap_for_each_entry(&opti->output, &iter, e) {
> +		string_list_append(&olist, e->key)->util = e->value;
> +	}
> +	string_list_sort(&olist);
> +
> +	/* Iterate over the items, printing them */
> +	for (i = 0; i < olist.nr; ++i) {
> +		struct strbuf *sb = olist.items[i].util;
> +
> +		printf("%s", sb->buf);
> +	}

Shorter/nicer:
	
	for_each_string_list_item(item, &olist) {
		struct strbuf *sb = item->util;
	        puts(sb->buf);
	}
	
> -	if (display_update_msgs) {
> -		struct merge_options_internal *opti = result->priv;
> -		struct hashmap_iter iter;
> -		struct strmap_entry *e;
> -		struct string_list olist = STRING_LIST_INIT_NODUP;
> -		int i;
> -
> -		if (opt->record_conflict_msgs_as_headers)
> -			BUG("Either display conflict messages or record them as headers, not both");
> -
> -		trace2_region_enter("merge", "display messages", opt->repo);
> -
> -		/* Hack to pre-allocate olist to the desired size */
> -		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> -			   olist.alloc);
> -
> -		/* Put every entry from output into olist, then sort */
> -		strmap_for_each_entry(&opti->output, &iter, e) {
> -			string_list_append(&olist, e->key)->util = e->value;
> -		}
> -		string_list_sort(&olist);
> -
> -		/* Iterate over the items, printing them */
> -		for (i = 0; i < olist.nr; ++i) {
> -			struct strbuf *sb = olist.items[i].util;
> -
> -			printf("%s", sb->buf);
> -		}
> -		string_list_clear(&olist, 0);

Ah, at this point I see you're just moving code around :) Sending this
anyway in case it's useful :)

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts
  2022-01-22 21:55 ` [PATCH 09/12] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
@ 2022-01-24 10:01   ` Ævar Arnfjörð Bjarmason
  2022-01-24 17:18     ` Elijah Newren
  2022-01-28 16:57   ` Johannes Schindelin
  1 sibling, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-01-24 10:01 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren


On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Callers of `git merge-tree --write-tree` will often want to know which
> files had conflicts.  While they could potentially attempt to parse the
> CONFLICT notices printed, those messages are not meant to be machine
> readable.  Provide a simpler mechanism of just printing the files (in
> the same format as `git ls-files` with quoting, but restricted to
> [...]
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 560640ad911..d8eeeb3f306 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -11,6 +11,9 @@
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
> +#include "quote.h"
> +
> +static int line_termination = '\n';

But unlike ls-files we don't do anything with line_termination as a !=
'\n', maybe in a later commit?

>  struct merge_list {
>  	struct merge_list *next;
> @@ -395,7 +398,8 @@ struct merge_tree_options {
>  };
>  
>  static int real_merge(struct merge_tree_options *o,
> -		      const char *branch1, const char *branch2)
> +		      const char *branch1, const char *branch2,
> +		      const char *prefix)
>  {
>  	struct commit *parent1, *parent2;
>  	struct commit_list *common;
> @@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
>  		o->show_messages = !result.clean;
>  
>  	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +	if (!result.clean) {
> +		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> +		const char *last = NULL;
> +		int i;
> +
> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);

But here it's never \0 or whatever.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info
  2022-01-22 21:56 ` [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
@ 2022-01-24 10:06   ` Ævar Arnfjörð Bjarmason
  2022-01-24 17:30     ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-01-24 10:06 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Elijah Newren


On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Much like `git merge` updates the index with information of the form
>     (mode, oid, stage, name)
> provide this output for conflicted files for merge-tree as well.
> Provide an --exclude-oids-and-modes option for users to exclude the
> mode, oid, and stage and only get the list of conflicted filenames.
> [...]
> +--exclude-oids-and-modes::
> +	Instead of writing a list of (mode, oid, stage, path) tuples
> +	to output for conflicted files, just provide a list of
> +	filenames with conflicts.
> +
> [...]
> -This is a sequence of lines containing a filename on each line, quoted
> -as explained for the configuration variable `core.quotePath` (see
> -linkgit:git-config[1]).
> +This is a sequence of lines with the format
> +
> +	<mode> <object> <stage> <filename>
> +
> +The filename will be quoted as explained for the configuration
> +variable `core.quotePath` (see linkgit:git-config[1]).  However, if
> +the `--exclude-oids-and-modes` option is passed, the mode, object, and
> +stage will be omitted.
>  
>  Informational messages
>  ~~~~~~~~~~~~~~~~~~~~~~
>  
>  This always starts with a blank line to separate it from the previous
> -section, and then has free-form messages about the merge, such as:
> +sections, and then has free-form messages about the merge, such as:
>  
>    * "Auto-merging <file>"
>    * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
> @@ -113,6 +123,14 @@ plumbing commands since the possibility of merge conflicts give it a
>  much higher chance of the command not succeeding (and NEWTREE containing
>  a bunch of stuff other than just a toplevel tree).
>  
> +git-merge-tree was written to provide users with the same information
> +that they'd have access to if using `git merge`:
> +  * what would be written to the working tree (the <OID of toplevel tree>)
> +  * the higher order stages that would be written to the index (the
> +    <Conflicted file info>)
> +  * any messages that would have been printed to stdout (the <Informational
> +    messages>)
> +
>  GIT
>  ---
>  Part of the linkgit:git[1] suite
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index d8eeeb3f306..7aa7f9fd54a 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -395,6 +395,7 @@ struct merge_tree_options {
>  	int real;
>  	int trivial;
>  	int show_messages;
> +	int exclude_oids_and_modes;
>  };
>  
>  static int real_merge(struct merge_tree_options *o,
> @@ -461,7 +462,11 @@ static int real_merge(struct merge_tree_options *o,
>  		merge_get_conflicted_files(&result, &conflicted_files);
>  		for (i = 0; i < conflicted_files.nr; i++) {
>  			const char *name = conflicted_files.items[i].string;
> -			if (last && !strcmp(last, name))
> +			struct stage_info *c = conflicted_files.items[i].util;
> +			if (!o->exclude_oids_and_modes)
> +				printf("%06o %s %d\t",
> +				       c->mode, oid_to_hex(&c->oid), c->stage);
> +			else if (last && !strcmp(last, name))
>  				continue;
>  			write_name_quoted_relative(
>  				name, prefix, stdout, line_termination);
> @@ -495,6 +500,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			 N_("do a trivial merge only")),
>  		OPT_BOOL(0, "messages", &o.show_messages,
>  			 N_("also show informational/conflict messages")),
> +		OPT_BOOL_F(0, "exclude-oids-and-modes",
> +			   &o.exclude_oids_and_modes,
> +			   N_("list conflicted files without oids and modes"),
> +			   PARSE_OPT_NONEG),
>  		OPT_END()
>  	};

Perhaps this really is the last formatting information anyone will want,
but with a default of "<mode> <object> <stage> <filename>" being made
"<stage> <filename>" with --exclude-oids-and-modes perhaps we'll want
--exclude-all-except-filename etc. later.

It seems a lot simpler for new code to just support a --conflict-format
option, lifting some code from the in-flight
https://lore.kernel.org/git/db058bf670c5668fc5b95baf83667cc282cb739b.1641978175.git.dyroneteng@gmail.com/

I.e. that inner loop becomes a slightly more verbose strbuf_expand(),
but it's all easy boilerplate code.

Then we just feed "%(objectmode) %(objectname) %(objectstage)
%(filename)" into it by default, and allow the user to customize it.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function
  2022-01-23  8:05   ` René Scharfe
@ 2022-01-24 16:43     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-24 16:43 UTC (permalink / raw)
  To: René Scharfe
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder

On Sun, Jan 23, 2022 at 12:05 AM René Scharfe <l.s.r@web.de> wrote:
>
> Am 22.01.22 um 22:55 schrieb Elijah Newren via GitGitGadget:
...
> > +     /* Check for a request for basic help */
> > +     if (argc == 2 && !strcmp(argv[1], "-h"))
> > +             usage_with_options(merge_tree_usage, mt_options);
>
> This is unnecessary; parse_options() handles -h already.
>
> > +
> > +     /* Parse arguments */
> > +     argc = parse_options(argc, argv, prefix, mt_options,
> > +                          merge_tree_usage, 0);
> > +     if (o.real && o.trivial)
> > +             die(_("--write-tree and --trivial-merge are incompatible"));
>
> 12909b6b8a (i18n: turn "options are incompatible" into "cannot be used
> together", 2022-01-05) standardized messages of that kind; let's stick
> to that to simplify translation:
>
>                 die(_("options '%s' and '%s' cannot be used together"),
>                     "--write-tree", "--trivial-merge");

Ah, thanks for both the pointers; will fix.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 03/12] merge-tree: add option parsing and initial shell for real merge function
  2022-01-24  9:46   ` Ævar Arnfjörð Bjarmason
@ 2022-01-24 16:54     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-24 16:54 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe

On Mon, Jan 24, 2022 at 1:50 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
...
> > +     /* Check for a request for basic help */
> > +     if (argc == 2 && !strcmp(argv[1], "-h"))
> > +             usage_with_options(merge_tree_usage, mt_options);
>
> Is this bit cargo-culted from something else, perhaps
> non-parse-options.c usage? I don't think this is needed, the
> parse_options() below intercepts "-h" by default.

Yep, sure was cargo-culted from somewhere else (my parse-options usage
always is), but I'm pretty sure it was from another place also using
parse-options.  Probably one of these 15 places:

 $ comm -12 <(git grep -l parse-options builtin/ | sort) <(git grep -l
strcmp.*-h\\b builtin/ | sort)
builtin/am.c
builtin/branch.c
builtin/checkout-index.c
builtin/checkout--worker.c
builtin/commit.c
builtin/commit-tree.c
builtin/gc.c
builtin/ls-files.c
builtin/merge.c
builtin/merge-tree.c
builtin/rebase.c
builtin/rev-parse.c
builtin/sparse-checkout.c
builtin/submodule--helper.c
builtin/update-index.c

> > +     /* Parse arguments */
> > +     argc = parse_options(argc, argv, prefix, mt_options,
> > +                          merge_tree_usage, 0);
> > +     if (o.real && o.trivial)
> > +             die(_("--write-tree and --trivial-merge are incompatible"));
>
> Shouldn't those two just be OPT_CMDMODE()? Then you get this
> incompatibility checking for free. See 485fd2c3dae (cat-file: make
> --batch-all-objects a CMDMODE, 2021-12-28).

TIL.  Thanks.

> > +     if (o.real || o.trivial) {
> > +             expected_remaining_argc = (o.real ? 2 : 3);
> > +             if (argc != expected_remaining_argc)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +     } else {
> > +             if (argc < 2 || argc > 3)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +             o.real = (argc == 2);
> > +     }
>
> And this can also be done like this, but I wonder if using
> PARSE_OPT_STOP_AT_NON_OPTION and then routing to a sub-function wouldn't
> be better, i.e. to treat these like sub-commands if they've got
> different arity etc.

Not sure what you mean; I already route to sub-functions.  But I
should definitely add PARSE_OPT_STOP_AT_NON_OPTION; it's unfortunate
that it's not the default.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 04/12] merge-tree: implement real merges
  2022-01-24  9:51   ` Ævar Arnfjörð Bjarmason
@ 2022-01-24 17:12     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-24 17:12 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe

On Mon, Jan 24, 2022 at 1:55 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > +     /*
> > +      * TODO: Support subtree and other -X options?
> > +     if (use_strategies_nr == 1 &&
> > +         !strcmp(use_strategies[0]->name, "subtree"))
> > +             opt.subtree_shift = "";
> > +     for (x = 0; x < xopts_nr; x++)
> > +             if (parse_merge_opt(&opt, xopts[x]))
>
> Better omitted WIP code in a non-RFC series?

It's RFC: https://lore.kernel.org/git/pull.1122.git.1642888562.gitgitgadget@gmail.com/

But yeah, I should drop it.  Previous rounds of this RFC submission
got me feedback that the other commented-out code bit I used to have
was something folks wanted in the initial version of the series so I
uncommented and cleaned it up.  The fact that no one has commented on
this part suggests these options don't need to be supported from the
start.

> > +                     die(_("Unknown strategy option: -X%s"), xopts[x]);
>
> As a general issue with this series, die(), BUG() etc. messages should
> start with a non-capital letter.

Right, thanks for the reminder.  I'll go through and try to fix up.

> > +     printf("%s\n", oid_to_hex(&result.tree->object.oid));
>
> And for both this...
>
> > +             printf(_("Conflicts!\n"));
>
> ... and this we can just use puts(). For the former it's just less code,
> but for the latter translators also don't need to see the always-there
> \n in the translated message.

Makes sense.

> > +# This test is ort-specific
> > +test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {
>
> Is this ${} trickery really needed? We're not testing with "set -u". So just:
>
>         if test "$GIT_..." != "ort"
>         then
>                 ...
>         fi

Ah, that would be simpler; thanks.

> > +test_expect_success 'Barf on too many arguments' '
> > +     test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
> > +
> > +     grep "^usage: git merge-tree" expect
> > +'
>
> Nit: In most other tests these usage assertions are at the top of the
> test, and for those we also make do with just testing the 129 exit code,
> which is probably enough here too...

I see a fair number of counterexamples searching for 129 in the test
suite, and I've been bitten enough times seeing tests expect an error
but get a different kind of error than the commit message stated they
were expecting that I prefer the extra check beyond the error code.
Anyway, I'll leave this piece as-is.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts
  2022-01-24 10:01   ` Ævar Arnfjörð Bjarmason
@ 2022-01-24 17:18     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-24 17:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe

On Mon, Jan 24, 2022 at 2:02 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Callers of `git merge-tree --write-tree` will often want to know which
> > files had conflicts.  While they could potentially attempt to parse the
> > CONFLICT notices printed, those messages are not meant to be machine
> > readable.  Provide a simpler mechanism of just printing the files (in
> > the same format as `git ls-files` with quoting, but restricted to
> > [...]
> > diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> > index 560640ad911..d8eeeb3f306 100644
> > --- a/builtin/merge-tree.c
> > +++ b/builtin/merge-tree.c
> > @@ -11,6 +11,9 @@
> >  #include "blob.h"
> >  #include "exec-cmd.h"
> >  #include "merge-blobs.h"
> > +#include "quote.h"
> > +
> > +static int line_termination = '\n';
>
> But unlike ls-files we don't do anything with line_termination as a !=
> '\n', maybe in a later commit?
>
> >  struct merge_list {
> >       struct merge_list *next;
> > @@ -395,7 +398,8 @@ struct merge_tree_options {
> >  };
> >
> >  static int real_merge(struct merge_tree_options *o,
> > -                   const char *branch1, const char *branch2)
> > +                   const char *branch1, const char *branch2,
> > +                   const char *prefix)
> >  {
> >       struct commit *parent1, *parent2;
> >       struct commit_list *common;
> > @@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
> >               o->show_messages = !result.clean;
> >
> >       printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +     if (!result.clean) {
> > +             struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> > +             const char *last = NULL;
> > +             int i;
> > +
> > +             merge_get_conflicted_files(&result, &conflicted_files);
> > +             for (i = 0; i < conflicted_files.nr; i++) {
> > +                     const char *name = conflicted_files.items[i].string;
> > +                     if (last && !strcmp(last, name))
> > +                             continue;
> > +                     write_name_quoted_relative(
> > +                             name, prefix, stdout, line_termination);
>
> But here it's never \0 or whatever.

Correct, I didn't add any option for changing it.  But why hardcode it
to "\n"?  Leaving it this way makes it easier to change later if folks
say they want NUL-terminated output.  Since the series is RFC and the
output already has changed drastically and appears to be the primary
discussion and disagreement point, I wanted to provide what seemed
like a reasonable suggestion and maintain flexiibility to address
feedback (though who knows -- I might need to just completely redo the
output again in ways much bigger than adding a -z option).

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 10/12] merge-tree: provide easy access to `ls-files -u` style info
  2022-01-24 10:06   ` Ævar Arnfjörð Bjarmason
@ 2022-01-24 17:30     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-24 17:30 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe

On Mon, Jan 24, 2022 at 2:11 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Much like `git merge` updates the index with information of the form
> >     (mode, oid, stage, name)
> > provide this output for conflicted files for merge-tree as well.
> > Provide an --exclude-oids-and-modes option for users to exclude the
> > mode, oid, and stage and only get the list of conflicted filenames.
> > [...]
> > +--exclude-oids-and-modes::
> > +     Instead of writing a list of (mode, oid, stage, path) tuples
> > +     to output for conflicted files, just provide a list of
> > +     filenames with conflicts.
> > +
> > [...]
> > -This is a sequence of lines containing a filename on each line, quoted
> > -as explained for the configuration variable `core.quotePath` (see
> > -linkgit:git-config[1]).
> > +This is a sequence of lines with the format
> > +
> > +     <mode> <object> <stage> <filename>
> > +
> > +The filename will be quoted as explained for the configuration
> > +variable `core.quotePath` (see linkgit:git-config[1]).  However, if
> > +the `--exclude-oids-and-modes` option is passed, the mode, object, and
> > +stage will be omitted.
> >
> >  Informational messages
> >  ~~~~~~~~~~~~~~~~~~~~~~
> >
> >  This always starts with a blank line to separate it from the previous
> > -section, and then has free-form messages about the merge, such as:
> > +sections, and then has free-form messages about the merge, such as:
> >
> >    * "Auto-merging <file>"
> >    * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
> > @@ -113,6 +123,14 @@ plumbing commands since the possibility of merge conflicts give it a
> >  much higher chance of the command not succeeding (and NEWTREE containing
> >  a bunch of stuff other than just a toplevel tree).
> >
> > +git-merge-tree was written to provide users with the same information
> > +that they'd have access to if using `git merge`:
> > +  * what would be written to the working tree (the <OID of toplevel tree>)
> > +  * the higher order stages that would be written to the index (the
> > +    <Conflicted file info>)
> > +  * any messages that would have been printed to stdout (the <Informational
> > +    messages>)
> > +
> >  GIT
> >  ---
> >  Part of the linkgit:git[1] suite
> > diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> > index d8eeeb3f306..7aa7f9fd54a 100644
> > --- a/builtin/merge-tree.c
> > +++ b/builtin/merge-tree.c
> > @@ -395,6 +395,7 @@ struct merge_tree_options {
> >       int real;
> >       int trivial;
> >       int show_messages;
> > +     int exclude_oids_and_modes;
> >  };
> >
> >  static int real_merge(struct merge_tree_options *o,
> > @@ -461,7 +462,11 @@ static int real_merge(struct merge_tree_options *o,
> >               merge_get_conflicted_files(&result, &conflicted_files);
> >               for (i = 0; i < conflicted_files.nr; i++) {
> >                       const char *name = conflicted_files.items[i].string;
> > -                     if (last && !strcmp(last, name))
> > +                     struct stage_info *c = conflicted_files.items[i].util;
> > +                     if (!o->exclude_oids_and_modes)
> > +                             printf("%06o %s %d\t",
> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
> > +                     else if (last && !strcmp(last, name))
> >                               continue;
> >                       write_name_quoted_relative(
> >                               name, prefix, stdout, line_termination);
> > @@ -495,6 +500,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >                        N_("do a trivial merge only")),
> >               OPT_BOOL(0, "messages", &o.show_messages,
> >                        N_("also show informational/conflict messages")),
> > +             OPT_BOOL_F(0, "exclude-oids-and-modes",
> > +                        &o.exclude_oids_and_modes,
> > +                        N_("list conflicted files without oids and modes"),
> > +                        PARSE_OPT_NONEG),
> >               OPT_END()
> >       };
>
> Perhaps this really is the last formatting information anyone will want,
> but with a default of "<mode> <object> <stage> <filename>" being made
> "<stage> <filename>" with --exclude-oids-and-modes perhaps we'll want
> --exclude-all-except-filename etc. later.

Um, that's actually what this option does.  Maybe my chosen name was bad.

--name-only like ls-files uses would have been nice, but it's
misleading since it only affects the <conflict info> section of the
output, not the printed tree or the informational messages.

> It seems a lot simpler for new code to just support a --conflict-format
> option, lifting some code from the in-flight
> https://lore.kernel.org/git/db058bf670c5668fc5b95baf83667cc282cb739b.1641978175.git.dyroneteng@gmail.com/
>
> I.e. that inner loop becomes a slightly more verbose strbuf_expand(),
> but it's all easy boilerplate code.
>
> Then we just feed "%(objectmode) %(objectname) %(objectstage)
> %(filename)" into it by default, and allow the user to customize it.

"simpler"?  More flexible certainly.

I'm not sure that the flexibility is warranted, in this case, though.
In ls-trees, where users don't need to process the output and can feed
it directly to something else, that flexibility makes sense.  But here
the output *needs* special post-processing anyway since it's mixing
three different types of output by default: top-level tree, conflicted
file info, and informational conflict messages.  (In my previous round
I tried to split these kinds of output to separate locations so they
could be parsed separately, but both Dscho and Christian didn't like
that).

So, the default I figured should be to just provide all the
information and just let others process it as wanted.  But Taylor had
said earlier that if there were conflicts the only information he
wanted was a list of conflicted files (no stages, oids, or modes), so
I figured an option making that a bit easier to extract was
worthwhile.  And that's what this patch is about.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 05/12] merge-ort: split out a separate display_update_messages() function
  2022-01-24  9:56   ` Ævar Arnfjörð Bjarmason
@ 2022-01-25  1:59     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-25  1:59 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe

On Mon, Jan 24, 2022 at 2:00 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> On Sat, Jan 22 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> > [...]
> > +     /* Hack to pre-allocate olist to the desired size */
> > +     ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> > +                olist.alloc);
>
> Perhaps just add a string_list_grow()? But I wonder if this is really
> needed v.s. just using the default growing pattern here.

A string_list_grow() would probably be helpful to add at some point;
then it could also be used in process_entries().

> > +
> > +     /* Put every entry from output into olist, then sort */
> > +     strmap_for_each_entry(&opti->output, &iter, e) {
> > +             string_list_append(&olist, e->key)->util = e->value;
> > +     }
> > +     string_list_sort(&olist);
> > +
> > +     /* Iterate over the items, printing them */
> > +     for (i = 0; i < olist.nr; ++i) {
> > +             struct strbuf *sb = olist.items[i].util;
> > +
> > +             printf("%s", sb->buf);
> > +     }
>
> Shorter/nicer:
>
>         for_each_string_list_item(item, &olist) {
>                 struct strbuf *sb = item->util;
>                 puts(sb->buf);
>         }

How did I not know about and not find for_each_string_list_item() when
I was writing this code a couple years ago?  (and still didn't learn
of it until now?)

Thanks for the pointer.  Won't change anything right now, though, since...

> > -     if (display_update_msgs) {
> > -             struct merge_options_internal *opti = result->priv;
> > -             struct hashmap_iter iter;
> > -             struct strmap_entry *e;
> > -             struct string_list olist = STRING_LIST_INIT_NODUP;
> > -             int i;
> > -
> > -             if (opt->record_conflict_msgs_as_headers)
> > -                     BUG("Either display conflict messages or record them as headers, not both");
> > -
> > -             trace2_region_enter("merge", "display messages", opt->repo);
> > -
> > -             /* Hack to pre-allocate olist to the desired size */
> > -             ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
> > -                        olist.alloc);
> > -
> > -             /* Put every entry from output into olist, then sort */
> > -             strmap_for_each_entry(&opti->output, &iter, e) {
> > -                     string_list_append(&olist, e->key)->util = e->value;
> > -             }
> > -             string_list_sort(&olist);
> > -
> > -             /* Iterate over the items, printing them */
> > -             for (i = 0; i < olist.nr; ++i) {
> > -                     struct strbuf *sb = olist.items[i].util;
> > -
> > -                     printf("%s", sb->buf);
> > -             }
> > -             string_list_clear(&olist, 0);
>
> Ah, at this point I see you're just moving code around :) Sending this
> anyway in case it's useful :)

yep, so I won't change anything now, but yes th
for_each_string_list_item() tip is still useful as a heads up.
Thanks.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 04/12] merge-tree: implement real merges
  2022-01-22 21:55 ` [PATCH 04/12] merge-tree: implement real merges Elijah Newren via GitGitGadget
  2022-01-24  9:51   ` Ævar Arnfjörð Bjarmason
@ 2022-01-25 17:07   ` Johannes Schindelin
  2022-01-26  9:44   ` Christian Couder
  2 siblings, 0 replies; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-25 17:07 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe, Elijah Newren,
	Elijah Newren

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> +The second form is deprecated; it is kept for backward compatibility
> +reasons but may be deleted in the future.  It will only do a trivial
> +merge.  It reads three tree-ish, and outputs trivial merge results and
> +conflicting stages to the standard output in a semi-diff format.
> +Since this was designed for higher level scripts to consume and merge
> +the results back into the index, it omits entries that match
> +<branch1>.  The result of this second form is is similar to what

There is a double "is" in this line. Taking a step back, I would suggest
to not only remove this paragraph, but to mark the `[--trivial-merge]`
option clearly as `(DEPRECATED)`.

> +three-way 'git read-tree -m' does, but instead of storing the results
> +in the index, the command outputs the entries to the standard output.
> +This form not only has limited applicability, the output format is
> +also difficult to work with, and it will generally be less performant
> +than the first form even on successful merges (especially if working
> +in large repositories).  The remainder of this manual will only
> +discuss the first form.

Thank you,
Dscho

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (11 preceding siblings ...)
  2022-01-22 21:56 ` [PATCH 12/12] git-merge-tree.txt: add a section on potentional usage mistakes Elijah Newren via GitGitGadget
@ 2022-01-26  8:48 ` Christian Couder
  2022-01-26 12:02   ` Johannes Schindelin
  2022-01-29  7:03   ` Elijah Newren
  2022-01-28 17:00 ` Johannes Schindelin
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
  14 siblings, 2 replies; 215+ messages in thread
From: Christian Couder @ 2022-01-26  8:48 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, René Scharfe,
	Elijah Newren

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
> suggestions and comments on v2):
>
>  * Significant changes to output format:
>    * Flags no longer take a filename for additional output; they write to
>      stdout instead.
>    * More information included by default when there are conflicts (no need
>      to request it with additional flags, instead flags can be used to
>      suppress it).
>    * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
>      information -- when there are conflicts. Add a flag to only list
>      conflicted files if that's preferred.

The above changes seem good to me.

>  * Much more thorough manual for git-merge-tree.txt
>  * Renamed option from --real to --write-tree
>  * Accept an optional --trivial-merge option to get old style merge-tree
>    behavior
>  * Allow both --write-tree and --trivial-merge to be omitted since we can
>    deduce which from number of arguments

I still think that it might be simpler and cleaner to leave 'git
merge-tree' alone for now, and just add a new command named for
example 'git write-merge-tree'. Later we can always add flags to 'git
merge-tree' or add 'git trivial-merge-tree' as an alias for 'git
merge-tree', and eventually slowly switch 'git merge-tree' to mean
only 'git write-merge-tree' if that's where we want to go.

>  * Document exit code when the merge cannot be run (so we can distinguish
>    other error cases from conflicts)
>  * testcase cleanups: test_tick, early skip of test when using recursive
>    backend, variable renames, etc.
>  * various minor code cleanups
>  * Add a new --allow-unrelated-histories option (with same meaning as the
>    one used in git merge)

The above changes seem good to me too.

> Stuff intentionally NOT included, but which others seemed to feel strongly
> about; they'd need to convince me more on these:
>
>  * Any form of diff output[1]

It's not a big issue for me to not include them right now as long as
it's possible to add cli options later that add them. The reason is
that I think in many cases when there are conflicts, the conflicts
will be small and the user will want to see them. So it would be
simpler to just have an option to show any conflict right away, rather
than have the user launch another command (a diff-tree against which
tree and with which options?).

Thanks for working on this anyway!

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 04/12] merge-tree: implement real merges
  2022-01-22 21:55 ` [PATCH 04/12] merge-tree: implement real merges Elijah Newren via GitGitGadget
  2022-01-24  9:51   ` Ævar Arnfjörð Bjarmason
  2022-01-25 17:07   ` Johannes Schindelin
@ 2022-01-26  9:44   ` Christian Couder
  2022-01-29  4:09     ` Elijah Newren
  2 siblings, 1 reply; 215+ messages in thread
From: Christian Couder @ 2022-01-26  9:44 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, René Scharfe,
	Elijah Newren

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Elijah Newren <newren@gmail.com>
>
> This adds the ability to perform real merges rather than just trivial
> merges (meaning handling three way content merges, recursive ancestor
> consolidation, renames, proper directory/file conflict handling, and so
> forth).  However, unlike `git merge`, the working tree and index are
> left alone and no branch is updated.
>
> The only output is:
>   - the toplevel resulting tree printed on stdout
>   - exit status of 0 (clean) or 1 (conflicts present)

The exit status can now actually be something other than 0 and 1
according to the doc and code below.

> +Performs a merge, but does not make any new commits and does not read
> +from or write to either the working tree or index.
> +
> +The first form will merge the two branches, doing a full recursive
> +merge with rename detection.

Maybe this could already tell that the first form will also write a
tree with the result of the merge (even in case of conflict) as this
could help understand the reason why the associated option is called
'--write-tree'. It could also help to say that we call such a merge a
'real' merge.

> The rest of this manual (other than the
> +next paragraph) describes the first form in more detail -- including
> +options, output format, exit status, and usage notes.

>  static int real_merge(struct merge_tree_options *o,
>                       const char *branch1, const char *branch2)
>  {
> -       die(_("real merges are not yet implemented"));
> +       struct commit *parent1, *parent2;
> +       struct commit_list *common;
> +       struct commit_list *merge_bases = NULL;
> +       struct commit_list *j;
> +       struct merge_options opt;
> +       struct merge_result result = { 0 };
> +
> +       parent1 = get_merge_parent(branch1);
> +       if (!parent1)
> +               help_unknown_ref(branch1, "merge",
> +                                _("not something we can merge"));

The second argument is supposed to be the command (it's called "cmd"),
so maybe "merge-tree" instead of "merge".

> +       parent2 = get_merge_parent(branch2);
> +       if (!parent2)
> +               help_unknown_ref(branch2, "merge",
> +                                _("not something we can merge"));

idem

> +       opt.show_rename_progress = 0;
> +
> +       opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
> +       opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */

I think just:

       opt.branch1 = branch1
       opt.branch2 = branch2

might be better for users as it should show the name as it was passed
to the command.

> +       merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> +       printf("%s\n", oid_to_hex(&result.tree->object.oid));

I wonder if we can actually always output a valid tree when
result.clean < 0. In case we might not, the printing should go a few
lines below.

> +       if (result.clean < 0)
> +               die(_("failure to merge"));
> +       else if (!result.clean)

The "else" is not necessary above.

> +               printf(_("Conflicts!\n"));
> +       merge_finalize(&opt, &result);
> +       return !result.clean; /* result.clean < 0 handled above */
>  }

> diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> new file mode 100755
> index 00000000000..e03688515c5
> --- /dev/null
> +++ b/t/t4301-merge-tree-real.sh

I wonder if it would be better named 't4301-merge-tree-write-tree.sh'...

> @@ -0,0 +1,87 @@
> +#!/bin/sh
> +
> +test_description='git merge-tree --write-tree'

... especially given this description.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 07/12] merge-tree: support including merge messages in output
  2022-01-22 21:55 ` [PATCH 07/12] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
@ 2022-01-26 10:42   ` Christian Couder
  2022-01-29  4:52     ` Elijah Newren
  2022-01-28 16:37   ` Johannes Schindelin
  1 sibling, 1 reply; 215+ messages in thread
From: Christian Couder @ 2022-01-26 10:42 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, René Scharfe,
	Elijah Newren

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

>  EXIT STATUS
>  -----------
> @@ -72,7 +102,8 @@ be used as a part of a series of steps such as
>
>  However, it does not quite fit into the same category of low-level
>  plumbing commands since the possibility of merge conflicts give it a
> -much higher chance of the command not succeeding.
> +much higher chance of the command not succeeding (and NEWTREE containing
> +a bunch of stuff other than just a toplevel tree).

Is this hunk really related to this commit or should it go into a
previous commit?

> @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
>                 commit_list_insert(j->item, &merge_bases);
>
>         merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> -       printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +
>         if (result.clean < 0)
>                 die(_("failure to merge"));

So this addresses the comment I made in a previous commit related to
the fact that if result.clean < 0 we might not have a valid tree that
we can print. I think though that it would be better if that was
addressed in a previous commit.

> -       else if (!result.clean)
> -               printf(_("Conflicts!\n"));

Ok, so we don't print "Conflicts!\n" now, which makes me wonder if we
should have printed it in the first place in previous commits.

>         if (o.real && o.trivial)
>                 die(_("--write-tree and --trivial-merge are incompatible"));
> +       if (!o.real && original_argc < argc)
> +               die(_("--write-tree must be specified if any other options are"));

Is this necessary? It looks to me like another thing that would be
simplified if we were just adding a new command...

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-22 21:55 ` [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
@ 2022-01-26 10:55   ` Christian Couder
  2022-01-29  4:55     ` Elijah Newren
  2022-01-26 11:07   ` Christian Couder
  2022-01-28 16:55   ` Johannes Schindelin
  2 siblings, 1 reply; 215+ messages in thread
From: Christian Couder @ 2022-01-26 10:55 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, René Scharfe,
	Elijah Newren

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> After a merge, this function allows the user to extract the same
> information that would be printed by `ls-files -u` -- conflicted

This made me wonder if "-- conflicted" should be part of the `ls-files
-u` command. Maybe "by `ls-files -u`, which means" would make things a
bit clearer.

> files with their mode, oid, and stage.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-22 21:55 ` [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
  2022-01-26 10:55   ` Christian Couder
@ 2022-01-26 11:07   ` Christian Couder
  2022-01-29  5:06     ` Elijah Newren
  2022-01-28 16:55   ` Johannes Schindelin
  2 siblings, 1 reply; 215+ messages in thread
From: Christian Couder @ 2022-01-26 11:07 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, René Scharfe,
	Elijah Newren

On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
<gitgitgadget@gmail.com> wrote:

> +void merge_get_conflicted_files(struct merge_result *result,
> +                               struct string_list *conflicted_files)
> +{
> +       struct hashmap_iter iter;
> +       struct strmap_entry *e;
> +       struct merge_options_internal *opti = result->priv;
> +
> +       strmap_for_each_entry(&opti->conflicted, &iter, e) {
> +               const char *path = e->key;
> +               struct conflict_info *ci = e->value;
> +               int i;
> +
> +               VERIFY_CI(ci);
> +
> +               for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> +                       struct stage_info *si;
> +
> +                       if (!(ci->filemask & (1ul << i)))
> +                               continue;
> +
> +                       si = xmalloc(sizeof(*si));

It's probably a premature optimization, so feel free to ignore, but as
MERGE_BASE and MERGE_SIDE2 are constants, and ci->filemask is constant
inside the 'for' loop, we could compute before the 'for' loop how many
'struct stage_info' we will need and allocate them all at once before
the 'for' loop.

> +                       si->stage = i+1;
> +                       si->mode = ci->stages[i].mode;
> +                       oidcpy(&si->oid, &ci->stages[i].oid);
> +                       string_list_append(conflicted_files, path)->util = si;
> +               }
> +       }
> +       /* string_list_sort() uses a stable sort, so we're good */
> +       string_list_sort(conflicted_files);
> +}

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-26  8:48 ` [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Christian Couder
@ 2022-01-26 12:02   ` Johannes Schindelin
  2022-01-26 14:44     ` Christian Couder
  2022-01-29  7:03   ` Elijah Newren
  1 sibling, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-26 12:02 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	René Scharfe, Elijah Newren

Hi Christian,

On Wed, 26 Jan 2022, Christian Couder wrote:

> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> >  * Accept an optional --trivial-merge option to get old style merge-tree
> >    behavior
> >  * Allow both --write-tree and --trivial-merge to be omitted since we can
> >    deduce which from number of arguments
>
> I still think that it might be simpler and cleaner to leave 'git
> merge-tree' alone for now, and just add a new command named for
> example 'git write-merge-tree'.

That would assume that the original `git merge-tree` implementation was
useful. That notion has been thoroughly refuted in the meantime, though.

I am really opposed to introducing a new command here. Elijah took the
best approach we can take here: save the `merge-tree` command by teaching
it to do something useful.

> Later we can always add flags to 'git merge-tree' or add 'git
> trivial-merge-tree' as an alias for 'git merge-tree', and eventually
> slowly switch 'git merge-tree' to mean only 'git write-merge-tree' if
> that's where we want to go.

I suggested before, and seem to need to repeat again, that we need to let
ourselves be guided less by hypothetical scenarios, and more by actual,
concrete use cases where the revamped `merge-tree` command is useful.

And since I already provided some feedback based on my work from working
on a server-side backend, I am fairly certain that we already have a
pretty good idea where we want to go.

> > Stuff intentionally NOT included, but which others seemed to feel strongly
> > about; they'd need to convince me more on these:
> >
> >  * Any form of diff output[1]
>
> It's not a big issue for me to not include them right now as long as
> it's possible to add cli options later that add them.

But why? That _so_ smells like a hypothetical scenario.

We do not need the diffs. It is highly unlikely that the server-side wants
to have diffs, and if a user does want the diffs, it is very, very easy to
generate them by chaining low-level commands.

So there is absolutely no need for `git merge-tree` to produce diffs.

> The reason is that I think in many cases when there are conflicts, the
> conflicts will be small and the user will want to see them. So it would
> be simpler to just have an option to show any conflict right away,
> rather than have the user launch another command (a diff-tree against
> which tree and with which options?).

That assumes that server-side merge UIs will present merge conflicts in
the form of diffs containing merge conflict markers. Which I don't think
will happen, like, ever.

In short: I completely disagree that we should introduce a new command,
and I also completely disagree that the `merge-tree` command should output
any diffs.

I do agree that we need to be mindful of what we actually need, and in
that regard, I reiterate that we need to let concrete use cases guide us.
As part of GitLab, you might be in an excellent position to look at
GitLab's concrete server-side needs when it comes to use `git merge-tree`
to perform merges.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-26 12:02   ` Johannes Schindelin
@ 2022-01-26 14:44     ` Christian Couder
  2022-01-28 12:58       ` Johannes Schindelin
  0 siblings, 1 reply; 215+ messages in thread
From: Christian Couder @ 2022-01-26 14:44 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	René Scharfe, Elijah Newren

Hi Dscho,

On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Christian,
>
> On Wed, 26 Jan 2022, Christian Couder wrote:
>
> > On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >
> > >  * Accept an optional --trivial-merge option to get old style merge-tree
> > >    behavior
> > >  * Allow both --write-tree and --trivial-merge to be omitted since we can
> > >    deduce which from number of arguments
> >
> > I still think that it might be simpler and cleaner to leave 'git
> > merge-tree' alone for now, and just add a new command named for
> > example 'git write-merge-tree'.
>
> That would assume that the original `git merge-tree` implementation was
> useful. That notion has been thoroughly refuted in the meantime, though.
>
> I am really opposed to introducing a new command here. Elijah took the
> best approach we can take here: save the `merge-tree` command by teaching
> it to do something useful.

I think it's a question of point of view. If a command is completely
useless, then most of the time it needs to die, not be "saved". We
would need good statistics, but I doubt we have "saved" many useless
commands before, compared to commands we have just killed.

> > Later we can always add flags to 'git merge-tree' or add 'git
> > trivial-merge-tree' as an alias for 'git merge-tree', and eventually
> > slowly switch 'git merge-tree' to mean only 'git write-merge-tree' if
> > that's where we want to go.
>
> I suggested before, and seem to need to repeat again, that we need to let
> ourselves be guided less by hypothetical scenarios, and more by actual,
> concrete use cases where the revamped `merge-tree` command is useful.

Ok, see below.

> And since I already provided some feedback based on my work from working
> on a server-side backend, I am fairly certain that we already have a
> pretty good idea where we want to go.
>
> > > Stuff intentionally NOT included, but which others seemed to feel strongly
> > > about; they'd need to convince me more on these:
> > >
> > >  * Any form of diff output[1]
> >
> > It's not a big issue for me to not include them right now as long as
> > it's possible to add cli options later that add them.
>
> But why? That _so_ smells like a hypothetical scenario.
>
> We do not need the diffs. It is highly unlikely that the server-side wants
> to have diffs, and if a user does want the diffs, it is very, very easy to
> generate them by chaining low-level commands.
>
> So there is absolutely no need for `git merge-tree` to produce diffs.
>
> > The reason is that I think in many cases when there are conflicts, the
> > conflicts will be small and the user will want to see them. So it would
> > be simpler to just have an option to show any conflict right away,
> > rather than have the user launch another command (a diff-tree against
> > which tree and with which options?).
>
> That assumes that server-side merge UIs will present merge conflicts in
> the form of diffs containing merge conflict markers. Which I don't think
> will happen, like, ever.

Please take a look at:

https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor

As you can see in the image there are conflict markers in the file
displayed by the server UI.

> In short: I completely disagree that we should introduce a new command,
> and I also completely disagree that the `merge-tree` command should output
> any diffs.
>
> I do agree that we need to be mindful of what we actually need, and in
> that regard, I reiterate that we need to let concrete use cases guide us.
> As part of GitLab, you might be in an excellent position to look at
> GitLab's concrete server-side needs when it comes to use `git merge-tree`
> to perform merges.

I hope I provided a concrete use case with the link above.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-26 14:44     ` Christian Couder
@ 2022-01-28 12:58       ` Johannes Schindelin
  2022-01-28 13:37         ` Christian Couder
  0 siblings, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 12:58 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	René Scharfe, Elijah Newren

Hi Christian,

On Wed, 26 Jan 2022, Christian Couder wrote:

> On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Wed, 26 Jan 2022, Christian Couder wrote:
> >
> > > The reason is that I think in many cases when there are conflicts,
> > > the conflicts will be small and the user will want to see them. So
> > > it would be simpler to just have an option to show any conflict
> > > right away, rather than have the user launch another command (a
> > > diff-tree against which tree and with which options?).
> >
> > That assumes that server-side merge UIs will present merge conflicts in
> > the form of diffs containing merge conflict markers. Which I don't think
> > will happen, like, ever.
>
> Please take a look at:
>
> https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor
>
> As you can see in the image there are conflict markers in the file
> displayed by the server UI.

Please note the difference between what I wrote above (present merge
conflicts in the form of diffs containing merge conflict markers) and what
is shown in the document you linked to (present a file annotated with
merge conflict markers).

There is no diff in that page.

What's more: there are not only conflict markers in the editor, there is
clearly a visual marker next to the line number that indicates that the
editor has a fundamental understanding where the conflict markers are.
Which means that the conflict markers must have been generated
independently of Git rather than parsed in some random diff that was
produced by Git.

In other words: you are making my case for me that `git merge-tree` should
not generate diff output because it would not even be used.

> > In short: I completely disagree that we should introduce a new command,
> > and I also completely disagree that the `merge-tree` command should output
> > any diffs.
> >
> > I do agree that we need to be mindful of what we actually need, and in
> > that regard, I reiterate that we need to let concrete use cases guide us.
> > As part of GitLab, you might be in an excellent position to look at
> > GitLab's concrete server-side needs when it comes to use `git merge-tree`
> > to perform merges.
>
> I hope I provided a concrete use case with the link above.

Sorry, I apparently was a bit unclear.

In the context of discussing `git merge-tree`, a low-level Git command,
when I talk about a user, I do not mean a human being, but a program that
calls that command and parses its output.

Corollary: by "use case" I refer to a concrete implementation of a
server-side merge operation, I refer to backend code that currently calls
into libgit2 to perform the merge, and would benefit from calling `git
merge-tree` instead. Such a use case informs us about the type and amount
of information that is required of the code that is currently being
discussed in this mail thread. And I highly doubt that you will find such
a use case that wants libgit2 (and later, `git merge-tree`) to generate
diffs. Because _diffs_ are certainly _not_ what is consumed by the inline
editor you referenced.

Of course, I am still left guessing what the server-side needs concretely,
because that is not at all obvious from the user-facing web site to which
you sent me. What is needed is a good, hard look at the actual _code_, the
code that calls into libgit2 to perform a merge, and that could instead
spawn a `git merge-tree` process to accomplish the same thing.

We need to get away from hypothetical scenarios. They're not helping.

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-28 12:58       ` Johannes Schindelin
@ 2022-01-28 13:37         ` Christian Couder
  2022-01-28 16:05           ` Johannes Schindelin
  0 siblings, 1 reply; 215+ messages in thread
From: Christian Couder @ 2022-01-28 13:37 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	René Scharfe, Elijah Newren

Hi Dscho,

On Fri, Jan 28, 2022 at 1:58 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Christian,
>
> On Wed, 26 Jan 2022, Christian Couder wrote:
>
> > On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > On Wed, 26 Jan 2022, Christian Couder wrote:
> > >
> > > > The reason is that I think in many cases when there are conflicts,
> > > > the conflicts will be small and the user will want to see them. So
> > > > it would be simpler to just have an option to show any conflict
> > > > right away, rather than have the user launch another command (a
> > > > diff-tree against which tree and with which options?).
> > >
> > > That assumes that server-side merge UIs will present merge conflicts in
> > > the form of diffs containing merge conflict markers. Which I don't think
> > > will happen, like, ever.
> >
> > Please take a look at:
> >
> > https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor
> >
> > As you can see in the image there are conflict markers in the file
> > displayed by the server UI.
>
> Please note the difference between what I wrote above (present merge
> conflicts in the form of diffs containing merge conflict markers) and what
> is shown in the document you linked to (present a file annotated with
> merge conflict markers).
>
> There is no diff in that page.

The server UI could just get a diff with the conflicts inside instead
of the full files with conflict inside, as the diff would be smaller
to parse than the full files. So even if it's not shown, the diff
could still be useful.

Also just above the section of the link I sent, there is this section

https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-interactive-mode

where one can see diff markers in the image. There are no conflict
markers in those images, but it's possible that a future UI could
combine both a diff and conflict markers.

Also please note that I don't absolutely require diffs. At the
beginning of the paragraph from my original email that you quoted
above, there was:

"It's not a big issue for me to not include them right now as long as
it's possible to add cli options later that add them."

So I was just saying that the format and code should be flexible
enough to be able to easily accommodate sending further data like
diffs with additional options. I think it's a very reasonable request.
So please don't make it a huge issue. You can always NACK a patch
adding such an option later.

> What's more: there are not only conflict markers in the editor,

You don't see the ">>>>>>>"?

> there is
> clearly a visual marker next to the line number that indicates that the
> editor has a fundamental understanding where the conflict markers are.

Yeah, so this shows that those markers can be important for the editor.

> Which means that the conflict markers must have been generated
> independently of Git rather than parsed in some random diff that was
> produced by Git.

Why couldn't they be generated by Git and then just parsed from a diff
in the future, even if that was not the case here?

> In other words: you are making my case for me that `git merge-tree` should
> not generate diff output because it would not even be used.

The other link above in this email actually shows that diffs are used
right now to resolve conflicts.

> > > In short: I completely disagree that we should introduce a new command,
> > > and I also completely disagree that the `merge-tree` command should output
> > > any diffs.
> > >
> > > I do agree that we need to be mindful of what we actually need, and in
> > > that regard, I reiterate that we need to let concrete use cases guide us.
> > > As part of GitLab, you might be in an excellent position to look at
> > > GitLab's concrete server-side needs when it comes to use `git merge-tree`
> > > to perform merges.
> >
> > I hope I provided a concrete use case with the link above.
>
> Sorry, I apparently was a bit unclear.
>
> In the context of discussing `git merge-tree`, a low-level Git command,
> when I talk about a user, I do not mean a human being, but a program that
> calls that command and parses its output.

So it could very well parse diffs containing conflict markers and show
those conflict markers.

[...]

> Of course, I am still left guessing what the server-side needs concretely,
> because that is not at all obvious from the user-facing web site to which
> you sent me. What is needed is a good, hard look at the actual _code_, the
> code that calls into libgit2 to perform a merge, and that could instead
> spawn a `git merge-tree` process to accomplish the same thing.
>
> We need to get away from hypothetical scenarios. They're not helping.

I am not even asking for a feature, just to make it possible to extend
the output of a brand new command in an RFC with some possibly useful
things, and you are making such requests...

Please relax a bit on this.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-28 13:37         ` Christian Couder
@ 2022-01-28 16:05           ` Johannes Schindelin
  0 siblings, 0 replies; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 16:05 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	René Scharfe, Elijah Newren

Hi Christian,

On Fri, 28 Jan 2022, Christian Couder wrote:

> On Fri, Jan 28, 2022 at 1:58 PM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Wed, 26 Jan 2022, Christian Couder wrote:
> >
> > > On Wed, Jan 26, 2022 at 1:03 PM Johannes Schindelin
> > > <Johannes.Schindelin@gmx.de> wrote:
> > > >
> > > > On Wed, 26 Jan 2022, Christian Couder wrote:
> > > >
> > > > > The reason is that I think in many cases when there are conflicts,
> > > > > the conflicts will be small and the user will want to see them. So
> > > > > it would be simpler to just have an option to show any conflict
> > > > > right away, rather than have the user launch another command (a
> > > > > diff-tree against which tree and with which options?).
> > > >
> > > > That assumes that server-side merge UIs will present merge conflicts in
> > > > the form of diffs containing merge conflict markers. Which I don't think
> > > > will happen, like, ever.
> > >
> > > Please take a look at:
> > >
> > > https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-the-inline-editor
> > >
> > > As you can see in the image there are conflict markers in the file
> > > displayed by the server UI.
> >
> > Please note the difference between what I wrote above (present merge
> > conflicts in the form of diffs containing merge conflict markers) and what
> > is shown in the document you linked to (present a file annotated with
> > merge conflict markers).
> >
> > There is no diff in that page.
>
> The server UI could just get a diff with the conflicts inside instead
> of the full files with conflict inside, as the diff would be smaller
> to parse than the full files. So even if it's not shown, the diff
> could still be useful.

You really need to get away from talking about this in hypothetical terms.

> Also just above the section of the link I sent, there is this section
>
> https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#resolve-conflicts-in-interactive-mode
>
> where one can see diff markers in the image.

That's a side-by-side diff. Git cannot even produce those.

> > What's more: there are not only conflict markers in the editor,
>
> You don't see the ">>>>>>>"?

Yes, I do. And not only that. I also see that the editor knows very
specifically where the conflict happens.

And since any file can contain `>>>>>>>` _without_ it being a conflict
marker, the editor most likely does not parse the output of Git that
contains a conflict marker. At least I hope it does not because it would
then very easily be confused by strings that look like conflict markers,
but aren't.

Think about our very own test suite, and why we specifically set
`conflict-marker-size=32` for those files. Same reason why the server
backend cannot simply ingest files with conflict markers and then hope to
figure out which `>>>>>>>` are real conflict markers and which are not.

> > there is clearly a visual marker next to the line number that
> > indicates that the editor has a fundamental understanding where the
> > conflict markers are.
>
> Yeah, so this shows that those markers can be important for the editor.

Of course they are important! That's my point!

> > In other words: you are making my case for me that `git merge-tree` should
> > not generate diff output because it would not even be used.
>
> The other link above in this email actually shows that diffs are used
> right now to resolve conflicts.

It shows that Git was not used to generate the diff, is what it shows.

I see that you are still trying to guess what the server-side needs
actually are. It really is time to stop guessing. So I will keep
challenging you to actually look at the GitLab code, to take a stab at
teaching it to use `git merge-tree` to perform merges. And then to come
back with what you learned. I guarantee you that that will be multiple
times more useful than talking about it in hypotheticals.

And you are in such an almost unique position to contribute to this patch
series, to provide that very valuable feedback how `git merge-tree` could
be improved to support actual, real-life server-side code that is
currently in use! So why not make the most out of it?

Ciao,
Johannes

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 05/12] merge-ort: split out a separate display_update_messages() function
  2022-01-22 21:55 ` [PATCH 05/12] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
  2022-01-24  9:56   ` Ævar Arnfjörð Bjarmason
@ 2022-01-28 16:09   ` Johannes Schindelin
  1 sibling, 0 replies; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 16:09 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe, Elijah Newren,
	Elijah Newren

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> No functional changes included in this patch; it's just a preparatory
> step to allow the printed messages to be handled differently by other
> callers, such as in `git merge-tree --write-tree`.

Looks good. FWIW I cheated and looked at the output of

	git show pr-1122/newren/in-core-merge-tree-v1~7 \
		--color-moved \
		--color-moved-ws=allow-indentation-change

(after fetching the tag from https://github.com/gitgitgadget/git) instead
of this patch (which is a lot easier to digest for me, because of the
color-coding, and since GitGitGadget sent the patch, I know that it is
identical to the commit).

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 06/12] merge-ort: allow update messages to be written to different file stream
  2022-01-22 21:55 ` [PATCH 06/12] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
@ 2022-01-28 16:31   ` Johannes Schindelin
  2022-01-29  4:33     ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 16:31 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe, Elijah Newren,
	Elijah Newren

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> diff --git a/merge-ort.c b/merge-ort.c
> index f9e35b0f96b..b78dde55ad9 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
>  }
>
>  void merge_display_update_messages(struct merge_options *opt,
> -				   struct merge_result *result)
> +				   struct merge_result *result,
> +				   FILE *stream)
>  {
>  	struct merge_options_internal *opti = result->priv;
>  	struct hashmap_iter iter;
> @@ -4263,7 +4264,7 @@ void merge_display_update_messages(struct merge_options *opt,
>  	for (i = 0; i < olist.nr; ++i) {
>  		struct strbuf *sb = olist.items[i].util;
>
> -		printf("%s", sb->buf);
> +		fprintf(stream, "%s", sb->buf);

Maybe `strbuf_write(sb, stream);` instead? Whenever I see a `"%s"`, I feel
like it's unnecessary churn...

>  	}
>  	string_list_clear(&olist, 0);
>

Missing from this hunk:

        /* Also include needed rename limit adjustment now */
        diff_warn_rename_limit("merge.renamelimit",
                               opti->renames.needed_limit, 0);

This function explicitly writes to `stdout`, and will need to be adjusted,
I think, before we can include an adjustment to this call in this patch.

Unless we override `warn_routine()` (which is used inside that function),
that is. Which is hacky, and we would not have addressed the
`fflush(stdout)` in `diff_warn_rename_limit()`. So I would much prefer
something like this:

-- snip --
diff --git a/diff.c b/diff.c
index 861282db1c3..87966602cbd 100644
--- a/diff.c
+++ b/diff.c
@@ -6312,17 +6312,25 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");

-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out)
 {
-	fflush(stdout);
+	const char *fmt = NULL;
+
 	if (degraded_cc)
-		warning(_(degrade_cc_to_c_warning));
+		fmt = _(degrade_cc_to_c_warning);
 	else if (needed)
-		warning(_(rename_limit_warning));
+		fmt = _(rename_limit_warning);
 	else
 		return;
 	if (0 < needed)
-		warning(_(rename_limit_advice), varname, needed);
+		fmt = _(rename_limit_advice);
+
+	fflush(out);
+	if (out == stdout)
+		warning(fmt, varname, needed);
+	else
+		fprintf(out, fmt, varname, needed);
 }

 static void diff_flush_patch_all_file_pairs(struct diff_options *o)
@@ -6754,7 +6762,7 @@ int diff_result_code(struct diff_options *opt, int status)

 	diff_warn_rename_limit("diff.renameLimit",
 			       opt->needed_rename_limit,
-			       opt->degraded_cc_to_c);
+			       opt->degraded_cc_to_c, stdout);
 	if (!opt->flags.exit_with_status &&
 	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
 		return status;
diff --git a/diff.h b/diff.h
index 8ba85c5e605..be4ee68c0a2 100644
--- a/diff.h
+++ b/diff.h
@@ -596,7 +596,8 @@ void diffcore_fix_diff_index(void);
 int diff_queue_is_empty(void);
 void diff_flush(struct diff_options*);
 void diff_free(struct diff_options*);
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out);

 /* diff-raw status letters */
 #define DIFF_STATUS_ADDED		'A'
diff --git a/merge-ort.c b/merge-ort.c
index 0342f104836..e6b5a0e7c64 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4264,7 +4264,7 @@ void merge_switch_to_result(struct merge_options *opt,

 		/* Also include needed rename limit adjustment now */
 		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
+				       opti->renames.needed_limit, 0, stdout);

 		trace2_region_leave("merge", "display messages", opt->repo);
 	}
diff --git a/merge-recursive.c b/merge-recursive.c
index d9457797dbb..10b2948678c 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3731,7 +3731,8 @@ static void merge_finalize(struct merge_options *opt)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->priv->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0,
+				       stdout);
 	FREE_AND_NULL(opt->priv);
 }

-- snap --

The rest of the patch looks good to me.

Thanks,
Dscho

> @@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
>  	}
>
>  	if (display_update_msgs)
> -		merge_display_update_messages(opt, result);
> +		merge_display_update_messages(opt, result, stdout);
>
>  	merge_finalize(opt, result);
>  }
> diff --git a/merge-ort.h b/merge-ort.h
> index e5aec45b18f..d643b47cb7c 100644
> --- a/merge-ort.h
> +++ b/merge-ort.h
> @@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
>   * so only call this when bypassing merge_switch_to_result().
>   */
>  void merge_display_update_messages(struct merge_options *opt,
> -				   struct merge_result *result);
> +				   struct merge_result *result,
> +				   FILE *stream);
>
>  /* Do needed cleanup when not calling merge_switch_to_result() */
>  void merge_finalize(struct merge_options *opt,
> --
> gitgitgadget
>
>

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 07/12] merge-tree: support including merge messages in output
  2022-01-22 21:55 ` [PATCH 07/12] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
  2022-01-26 10:42   ` Christian Couder
@ 2022-01-28 16:37   ` Johannes Schindelin
  2022-01-29  4:46     ` Elijah Newren
  1 sibling, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 16:37 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe, Elijah Newren,
	Elijah Newren

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> When running `git merge-tree --write-tree`, we previously would only
> return an exit status reflecting the cleanness of a merge, and print out
> the toplevel tree of the resulting merge.  Merges also have
> informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
> ...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
> conflicts occur (such as file/directory, modify/delete, add/add with
> differing modes, rename/rename (1to2), etc.), these informational
> messages are often the only notification since these conflicts are not
> representable in the contents of the file.
>
> Add a --[no-]messages option so that callers can request these messages
> be included at the end of the output.  Include such messages by default
> when there are conflicts, and omit them by default when the merge is
> clean.

Makes sense.

> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 0c19639594d..560640ad911 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
>  		commit_list_insert(j->item, &merge_bases);
>
>  	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> -	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +
>  	if (result.clean < 0)
>  		die(_("failure to merge"));
> -	else if (!result.clean)
> -		printf(_("Conflicts!\n"));
> +
> +	if (o->show_messages == -1)
> +		o->show_messages = !result.clean;
> +
> +	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +	if (o->show_messages) {
> +		printf("\n");
> +		merge_display_update_messages(&opt, &result, stdout);
> +	}

Excellent.

>  	merge_finalize(&opt, &result);
>  	return !result.clean; /* result.clean < 0 handled above */
>  }
>
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	struct merge_tree_options o = { 0 };
> +	struct merge_tree_options o = { .show_messages = -1 };
>  	int expected_remaining_argc;
> +	int original_argc;
>
>  	const char * const merge_tree_usage[] = {
> -		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
>  		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>  		NULL
>  	};
> @@ -464,6 +473,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			 N_("do a real merge instead of a trivial merge")),
>  		OPT_BOOL(0, "trivial-merge", &o.trivial,
>  			 N_("do a trivial merge only")),
> +		OPT_BOOL(0, "messages", &o.show_messages,
> +			 N_("also show informational/conflict messages")),
>  		OPT_END()
>  	};
>
> @@ -472,10 +483,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  		usage_with_options(merge_tree_usage, mt_options);
>
>  	/* Parse arguments */
> +	original_argc = argc;
>  	argc = parse_options(argc, argv, prefix, mt_options,
>  			     merge_tree_usage, 0);
>  	if (o.real && o.trivial)
>  		die(_("--write-tree and --trivial-merge are incompatible"));
> +	if (!o.real && original_argc < argc)
> +		die(_("--write-tree must be specified if any other options are"));

Hmm. Well. Hmm.


I'd rather keep `--write-tree` neat and optional. What's wrong with
allowing

	git merge-tree --no-messages HEAD MERGE_HEAD

?

To be clear, I think we need this instead:

	if (o.trivial && o.show_messages >= 0)
		die(_("--trivial-merge is incompatible with additional options"));

I like the rest of the patch very much!

Thank you,
Dscho

>  	if (o.real || o.trivial) {
>  		expected_remaining_argc = (o.real ? 2 : 3);
>  		if (argc != expected_remaining_argc)
> diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> index e03688515c5..c34f8e6c1ed 100755
> --- a/t/t4301-merge-tree-real.sh
> +++ b/t/t4301-merge-tree-real.sh
> @@ -84,4 +84,25 @@ test_expect_success 'Barf on too many arguments' '
>  	grep "^usage: git merge-tree" expect
>  '
>
> +test_expect_success 'test conflict notices and such' '
> +	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
> +	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
> +
> +	# Expected results:
> +	#   "greeting" should merge with conflicts
> +	#   "numbers" should merge cleanly
> +	#   "whatever" has *both* a modify/delete and a file/directory conflict
> +	cat <<-EOF >expect &&
> +	HASH
> +
> +	Auto-merging greeting
> +	CONFLICT (content): Merge conflict in greeting
> +	Auto-merging numbers
> +	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
> +	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
> +	EOF
> +
> +	test_cmp expect actual
> +'
> +
>  test_done
> --
> gitgitgadget
>
>

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-22 21:55 ` [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
  2022-01-26 10:55   ` Christian Couder
  2022-01-26 11:07   ` Christian Couder
@ 2022-01-28 16:55   ` Johannes Schindelin
  2022-01-29  6:08     ` Elijah Newren
  2 siblings, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 16:55 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe, Elijah Newren,
	Elijah Newren

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> After a merge, this function allows the user to extract the same
> information that would be printed by `ls-files -u` -- conflicted
> files with their mode, oid, and stage.

Hmm. Okay.

I am not really a fan of that output where we use a variable
number of lines per file. As in: in the regular case, where a file was
modified in divergent ways, we get three lines. But if it was deleted in
one branch, or if it was created in both branches, we have only two lines.

Frankly, I'd much rather have 3 lines for each and every conflict.

Granted, we currently only have three stages, and we can pretty much
guarantee that at least two of these stages are non-empty (otherwise where
would be the conflict?).

Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
missing from the second conflict, in the output we would see stages 1, 2,
2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
different conflicts.

But still. It makes me uneasy to have that much variability in
machine-consumable output.

And who knows, maybe if we're ever implementing more sophisticated merge
strategies for octopus merges, we might end up with more stages...

In other words...

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-ort.c | 31 +++++++++++++++++++++++++++++++
>  merge-ort.h | 21 +++++++++++++++++++++
>  2 files changed, 52 insertions(+)
>
> diff --git a/merge-ort.c b/merge-ort.c
> index b78dde55ad9..5e7cea6cc8f 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
>  	trace2_region_leave("merge", "display messages", opt->repo);
>  }
>
> +void merge_get_conflicted_files(struct merge_result *result,
> +				struct string_list *conflicted_files)
> +{
> +	struct hashmap_iter iter;
> +	struct strmap_entry *e;
> +	struct merge_options_internal *opti = result->priv;
> +
> +	strmap_for_each_entry(&opti->conflicted, &iter, e) {
> +		const char *path = e->key;
> +		struct conflict_info *ci = e->value;
> +		int i;
> +
> +		VERIFY_CI(ci);
> +
> +		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> +			struct stage_info *si;
> +
> +			if (!(ci->filemask & (1ul << i)))
> +				continue;
> +
> +			si = xmalloc(sizeof(*si));
> +			si->stage = i+1;
> +			si->mode = ci->stages[i].mode;
> +			oidcpy(&si->oid, &ci->stages[i].oid);
> +			string_list_append(conflicted_files, path)->util = si;
> +		}
> +	}
> +	/* string_list_sort() uses a stable sort, so we're good */
> +	string_list_sort(conflicted_files);
> +}
> +
>  void merge_switch_to_result(struct merge_options *opt,
>  			    struct tree *head,
>  			    struct merge_result *result,
> diff --git a/merge-ort.h b/merge-ort.h
> index d643b47cb7c..e635a294ea8 100644
> --- a/merge-ort.h
> +++ b/merge-ort.h
> @@ -2,6 +2,7 @@
>  #define MERGE_ORT_H
>
>  #include "merge-recursive.h"
> +#include "hash.h"
>
>  struct commit;
>  struct tree;
> @@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
>  				   struct merge_result *result,
>  				   FILE *stream);
>
> +struct stage_info {
> +	struct object_id oid;
> +	int mode;
> +	int stage;
> +};

... I'd rather not tack this onto a `string_list` but instead have
something like:

	struct conflict_info {
		struct {
			struct object_id oid;
			int mode;
			const char *path;
		} stages[3]
	};

where `path` can be `NULL` to indicate that this stage is missing.

Apart from this concern about the overall design, the patch looks good, of
course.

Ciao,
Dscho

> +
> +/*
> + * Provide a list of path -> {struct stage_info*} mappings for
> + * all conflicted files.  Note that each path could appear up to three
> + * times in the list, corresponding to 3 different stage entries.  In short,
> + * this basically provides the info that would be printed by `ls-files -u`.
> + *
> + * result should have been populated by a call to
> + * one of the merge_incore_[non]recursive() functions.
> + *
> + * conflicted_files should be empty before calling this function.
> + */
> +void merge_get_conflicted_files(struct merge_result *result,
> +				struct string_list *conflicted_files);
> +
>  /* Do needed cleanup when not calling merge_switch_to_result() */
>  void merge_finalize(struct merge_options *opt,
>  		    struct merge_result *result);
> --
> gitgitgadget
>
>

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts
  2022-01-22 21:55 ` [PATCH 09/12] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
  2022-01-24 10:01   ` Ævar Arnfjörð Bjarmason
@ 2022-01-28 16:57   ` Johannes Schindelin
  2022-01-29  6:21     ` Elijah Newren
  1 sibling, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 16:57 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe, Elijah Newren,
	Elijah Newren

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Callers of `git merge-tree --write-tree` will often want to know which
> files had conflicts.  While they could potentially attempt to parse the
> CONFLICT notices printed, those messages are not meant to be machine
> readable.  Provide a simpler mechanism of just printing the files (in
> the same format as `git ls-files` with quoting, but restricted to
> unmerged files) in the output before the free-form messages.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  Documentation/git-merge-tree.txt |  8 ++++++++
>  builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
>  t/t4301-merge-tree-real.sh       | 11 +++++++++++
>  3 files changed, 41 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index fd7a867de60..041a4ac2785 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -58,6 +58,7 @@ simply one line:
>  Whereas for a conflicted merge, the output is by default of the form:
>
>  	<OID of toplevel tree>
> +	<Conflicted file list>
>  	<Informational messages>

To distinguish between the list of conflicted files and the informational
messages, I think it would be good to insert an empty line, as a
separator, like. And...

>
>  These are discussed individually below.
> @@ -69,6 +70,13 @@ This is a tree object that represents what would be checked out in the
>  working tree at the end of `git merge`.  If there were conflicts, then
>  files within this tree may have embedded conflict markers.
>
> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).
> +
>  Informational messages
>  ~~~~~~~~~~~~~~~~~~~~~~
>
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 560640ad911..d8eeeb3f306 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -11,6 +11,9 @@
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
> +#include "quote.h"
> +
> +static int line_termination = '\n';
>
>  struct merge_list {
>  	struct merge_list *next;
> @@ -395,7 +398,8 @@ struct merge_tree_options {
>  };
>
>  static int real_merge(struct merge_tree_options *o,
> -		      const char *branch1, const char *branch2)
> +		      const char *branch1, const char *branch2,
> +		      const char *prefix)
>  {
>  	struct commit *parent1, *parent2;
>  	struct commit_list *common;
> @@ -449,6 +453,22 @@ static int real_merge(struct merge_tree_options *o,
>  		o->show_messages = !result.clean;
>
>  	printf("%s\n", oid_to_hex(&result.tree->object.oid));
> +	if (!result.clean) {
> +		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> +		const char *last = NULL;
> +		int i;
> +
> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;
> +		}
> +		string_list_clear(&conflicted_files, 1);
> +	}
>  	if (o->show_messages) {
>  		printf("\n");

... it seems that we do this already...

>  		merge_display_update_messages(&opt, &result, stdout);
> @@ -502,7 +522,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>
>  	/* Do the relevant type of merge */
>  	if (o.real)
> -		return real_merge(&o, argv[0], argv[1]);
> +		return real_merge(&o, argv[0], argv[1], prefix);
>  	else
>  		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> index c34f8e6c1ed..43c9950dedb 100755
> --- a/t/t4301-merge-tree-real.sh
> +++ b/t/t4301-merge-tree-real.sh
> @@ -94,6 +94,8 @@ test_expect_success 'test conflict notices and such' '
>  	#   "whatever" has *both* a modify/delete and a file/directory conflict
>  	cat <<-EOF >expect &&
>  	HASH
> +	greeting
> +	whatever~side1
>
>  	Auto-merging greeting
>  	CONFLICT (content): Merge conflict in greeting

... as illustrated by the test, too. I guess the documentation should show
the empty line, too?

Ciao,
Dscho

> @@ -105,4 +107,13 @@ test_expect_success 'test conflict notices and such' '
>  	test_cmp expect actual
>  '
>
> +test_expect_success 'Just the conflicted files without the messages' '
> +	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
> +	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
> +
> +	test_write_lines HASH greeting whatever~side1 >expect &&
> +
> +	test_cmp expect actual
> +'
> +
>  test_done
> --
> gitgitgadget
>
>

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (12 preceding siblings ...)
  2022-01-26  8:48 ` [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Christian Couder
@ 2022-01-28 17:00 ` Johannes Schindelin
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
  14 siblings, 0 replies; 215+ messages in thread
From: Johannes Schindelin @ 2022-01-28 17:00 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe, Elijah Newren

[-- Attachment #1: Type: text/plain, Size: 1897 bytes --]

Hi Elijah,

On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:

> Note1: Depends on en/remerge-diff (but no need to pick it up; it's still
> RFC).

For reasons, I will have to backport this on top of v2.33.1 ;-)

> == Updates Log ==
>
> Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
> suggestions and comments on v2):
>
>  * Significant changes to output format:
>    * Flags no longer take a filename for additional output; they write to
>      stdout instead.
>    * More information included by default when there are conflicts (no need
>      to request it with additional flags, instead flags can be used to
>      suppress it).
>    * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
>      information -- when there are conflicts. Add a flag to only list
>      conflicted files if that's preferred.
>  * Much more thorough manual for git-merge-tree.txt
>  * Renamed option from --real to --write-tree
>  * Accept an optional --trivial-merge option to get old style merge-tree
>    behavior
>  * Allow both --write-tree and --trivial-merge to be omitted since we can
>    deduce which from number of arguments
>  * Document exit code when the merge cannot be run (so we can distinguish
>    other error cases from conflicts)
>  * testcase cleanups: test_tick, early skip of test when using recursive
>    backend, variable renames, etc.
>  * various minor code cleanups
>  * Add a new --allow-unrelated-histories option (with same meaning as the
>    one used in git merge)
>  * Rebased on top of en/remerge-diff to avoid a small conflict

I am really happy with the way this patch series is going, and hope to be
a lot more active on it in the near future.

I've read through the current iteration and left a few suggestions,
nothing major.

Thank you so much for your outstanding work!
Dscho

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 04/12] merge-tree: implement real merges
  2022-01-26  9:44   ` Christian Couder
@ 2022-01-29  4:09     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  4:09 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

On Wed, Jan 26, 2022 at 1:45 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> >
> > From: Elijah Newren <newren@gmail.com>
> >
> > This adds the ability to perform real merges rather than just trivial
> > merges (meaning handling three way content merges, recursive ancestor
> > consolidation, renames, proper directory/file conflict handling, and so
> > forth).  However, unlike `git merge`, the working tree and index are
> > left alone and no branch is updated.
> >
> > The only output is:
> >   - the toplevel resulting tree printed on stdout
> >   - exit status of 0 (clean) or 1 (conflicts present)
>
> The exit status can now actually be something other than 0 and 1
> according to the doc and code below.

Thanks for catching; will fix.

> > +Performs a merge, but does not make any new commits and does not read
> > +from or write to either the working tree or index.
> > +
> > +The first form will merge the two branches, doing a full recursive
> > +merge with rename detection.
>
> Maybe this could already tell that the first form will also write a
> tree with the result of the merge (even in case of conflict) as this
> could help understand the reason why the associated option is called
> '--write-tree'. It could also help to say that we call such a merge a
> 'real' merge.

Makes sense.

> > The rest of this manual (other than the
> > +next paragraph) describes the first form in more detail -- including
> > +options, output format, exit status, and usage notes.
>
> >  static int real_merge(struct merge_tree_options *o,
> >                       const char *branch1, const char *branch2)
> >  {
> > -       die(_("real merges are not yet implemented"));
> > +       struct commit *parent1, *parent2;
> > +       struct commit_list *common;
> > +       struct commit_list *merge_bases = NULL;
> > +       struct commit_list *j;
> > +       struct merge_options opt;
> > +       struct merge_result result = { 0 };
> > +
> > +       parent1 = get_merge_parent(branch1);
> > +       if (!parent1)
> > +               help_unknown_ref(branch1, "merge",
> > +                                _("not something we can merge"));
>
> The second argument is supposed to be the command (it's called "cmd"),
> so maybe "merge-tree" instead of "merge".

Oh, good catch; thanks for pointing this out.

>
> > +       parent2 = get_merge_parent(branch2);
> > +       if (!parent2)
> > +               help_unknown_ref(branch2, "merge",
> > +                                _("not something we can merge"));
>
> idem
>
> > +       opt.show_rename_progress = 0;
> > +
> > +       opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
> > +       opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */
>
> I think just:
>
>        opt.branch1 = branch1
>        opt.branch2 = branch2
>
> might be better for users as it should show the name as it was passed
> to the command.

After digging for a bit, I think in this case there actually isn't a
difference to users because both will give the same result.  But, if
that's the case, the simpler code is warranted.

> > +       merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > +       printf("%s\n", oid_to_hex(&result.tree->object.oid));
>
> I wonder if we can actually always output a valid tree when
> result.clean < 0. In case we might not, the printing should go a few
> lines below.

Yeah, I caught that and fixed it, but got it squashed into a later
commit.  I'll fix it up.

> > +       if (result.clean < 0)
> > +               die(_("failure to merge"));
> > +       else if (!result.clean)
>
> The "else" is not necessary above.
>
> > +               printf(_("Conflicts!\n"));

Yes, and the else clause should just be ripped out.

> > +       merge_finalize(&opt, &result);
> > +       return !result.clean; /* result.clean < 0 handled above */
> >  }
>
> > diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> > new file mode 100755
> > index 00000000000..e03688515c5
> > --- /dev/null
> > +++ b/t/t4301-merge-tree-real.sh
>
> I wonder if it would be better named 't4301-merge-tree-write-tree.sh'...
>
> > @@ -0,0 +1,87 @@
> > +#!/bin/sh
> > +
> > +test_description='git merge-tree --write-tree'
>
> ... especially given this description.

Makes sense; will rename.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 06/12] merge-ort: allow update messages to be written to different file stream
  2022-01-28 16:31   ` Johannes Schindelin
@ 2022-01-29  4:33     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  4:33 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

On Fri, Jan 28, 2022 at 8:31 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > diff --git a/merge-ort.c b/merge-ort.c
> > index f9e35b0f96b..b78dde55ad9 100644
> > --- a/merge-ort.c
> > +++ b/merge-ort.c
> > @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
> >  }
> >
> >  void merge_display_update_messages(struct merge_options *opt,
> > -                                struct merge_result *result)
> > +                                struct merge_result *result,
> > +                                FILE *stream)
> >  {
> >       struct merge_options_internal *opti = result->priv;
> >       struct hashmap_iter iter;
> > @@ -4263,7 +4264,7 @@ void merge_display_update_messages(struct merge_options *opt,
> >       for (i = 0; i < olist.nr; ++i) {
> >               struct strbuf *sb = olist.items[i].util;
> >
> > -             printf("%s", sb->buf);
> > +             fprintf(stream, "%s", sb->buf);
>
> Maybe `strbuf_write(sb, stream);` instead? Whenever I see a `"%s"`, I feel
> like it's unnecessary churn...

Makes sense.

> >       }
> >       string_list_clear(&olist, 0);
> >
>
> Missing from this hunk:
>
>         /* Also include needed rename limit adjustment now */
>         diff_warn_rename_limit("merge.renamelimit",
>                                opti->renames.needed_limit, 0);
>
> This function explicitly writes to `stdout`, and will need to be adjusted,
> I think, before we can include an adjustment to this call in this patch.

Oh, good point.

> Unless we override `warn_routine()` (which is used inside that function),
> that is. Which is hacky, and we would not have addressed the
> `fflush(stdout)` in `diff_warn_rename_limit()`. So I would much prefer
> something like this:
>
> -- snip --
> diff --git a/diff.c b/diff.c
> index 861282db1c3..87966602cbd 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -6312,17 +6312,25 @@ static const char rename_limit_advice[] =
>  N_("you may want to set your %s variable to at least "
>     "%d and retry the command.");
>
> -void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
> +void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> +                           FILE *out)
>  {
> -       fflush(stdout);
> +       const char *fmt = NULL;
> +
>         if (degraded_cc)
> -               warning(_(degrade_cc_to_c_warning));
> +               fmt = _(degrade_cc_to_c_warning);
>         else if (needed)
> -               warning(_(rename_limit_warning));
> +               fmt = _(rename_limit_warning);
>         else
>                 return;
>         if (0 < needed)
> -               warning(_(rename_limit_advice), varname, needed);
> +               fmt = _(rename_limit_advice);
> +
> +       fflush(out);
> +       if (out == stdout)
> +               warning(fmt, varname, needed);
> +       else
> +               fprintf(out, fmt, varname, needed);
>  }
>
>  static void diff_flush_patch_all_file_pairs(struct diff_options *o)
> @@ -6754,7 +6762,7 @@ int diff_result_code(struct diff_options *opt, int status)
>
>         diff_warn_rename_limit("diff.renameLimit",
>                                opt->needed_rename_limit,
> -                              opt->degraded_cc_to_c);
> +                              opt->degraded_cc_to_c, stdout);
>         if (!opt->flags.exit_with_status &&
>             !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
>                 return status;
> diff --git a/diff.h b/diff.h
> index 8ba85c5e605..be4ee68c0a2 100644
> --- a/diff.h
> +++ b/diff.h
> @@ -596,7 +596,8 @@ void diffcore_fix_diff_index(void);
>  int diff_queue_is_empty(void);
>  void diff_flush(struct diff_options*);
>  void diff_free(struct diff_options*);
> -void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
> +void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> +                           FILE *out);
>
>  /* diff-raw status letters */
>  #define DIFF_STATUS_ADDED              'A'
> diff --git a/merge-ort.c b/merge-ort.c
> index 0342f104836..e6b5a0e7c64 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4264,7 +4264,7 @@ void merge_switch_to_result(struct merge_options *opt,
>
>                 /* Also include needed rename limit adjustment now */
>                 diff_warn_rename_limit("merge.renamelimit",
> -                                      opti->renames.needed_limit, 0);
> +                                      opti->renames.needed_limit, 0, stdout);
>
>                 trace2_region_leave("merge", "display messages", opt->repo);
>         }
> diff --git a/merge-recursive.c b/merge-recursive.c
> index d9457797dbb..10b2948678c 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -3731,7 +3731,8 @@ static void merge_finalize(struct merge_options *opt)
>                 strbuf_release(&opt->obuf);
>         if (show(opt, 2))
>                 diff_warn_rename_limit("merge.renamelimit",
> -                                      opt->priv->needed_rename_limit, 0);
> +                                      opt->priv->needed_rename_limit, 0,
> +                                      stdout);
>         FREE_AND_NULL(opt->priv);
>  }
>
> -- snap --

I like this, but believe it makes more sense as a preparatory patch.
So I created one and made you the author, and included your
signed-off-by.  That okay with you?

>
> The rest of the patch looks good to me.

Thanks!

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 07/12] merge-tree: support including merge messages in output
  2022-01-28 16:37   ` Johannes Schindelin
@ 2022-01-29  4:46     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  4:46 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

On Fri, Jan 28, 2022 at 8:37 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > When running `git merge-tree --write-tree`, we previously would only
> > return an exit status reflecting the cleanness of a merge, and print out
> > the toplevel tree of the resulting merge.  Merges also have
> > informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
> > ...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
> > conflicts occur (such as file/directory, modify/delete, add/add with
> > differing modes, rename/rename (1to2), etc.), these informational
> > messages are often the only notification since these conflicts are not
> > representable in the contents of the file.
> >
> > Add a --[no-]messages option so that callers can request these messages
> > be included at the end of the output.  Include such messages by default
> > when there are conflicts, and omit them by default when the merge is
> > clean.
>
> Makes sense.
>
> > diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> > index 0c19639594d..560640ad911 100644
> > --- a/builtin/merge-tree.c
> > +++ b/builtin/merge-tree.c
> > @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
> >               commit_list_insert(j->item, &merge_bases);
> >
> >       merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > -     printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +
> >       if (result.clean < 0)
> >               die(_("failure to merge"));
> > -     else if (!result.clean)
> > -             printf(_("Conflicts!\n"));
> > +
> > +     if (o->show_messages == -1)
> > +             o->show_messages = !result.clean;
> > +
> > +     printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +     if (o->show_messages) {
> > +             printf("\n");
> > +             merge_display_update_messages(&opt, &result, stdout);
> > +     }
>
> Excellent.
>
> >       merge_finalize(&opt, &result);
> >       return !result.clean; /* result.clean < 0 handled above */
> >  }
> >
> >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >  {
> > -     struct merge_tree_options o = { 0 };
> > +     struct merge_tree_options o = { .show_messages = -1 };
> >       int expected_remaining_argc;
> > +     int original_argc;
> >
> >       const char * const merge_tree_usage[] = {
> > -             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> > +             N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
> >               N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> >               NULL
> >       };
> > @@ -464,6 +473,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >                        N_("do a real merge instead of a trivial merge")),
> >               OPT_BOOL(0, "trivial-merge", &o.trivial,
> >                        N_("do a trivial merge only")),
> > +             OPT_BOOL(0, "messages", &o.show_messages,
> > +                      N_("also show informational/conflict messages")),
> >               OPT_END()
> >       };
> >
> > @@ -472,10 +483,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >               usage_with_options(merge_tree_usage, mt_options);
> >
> >       /* Parse arguments */
> > +     original_argc = argc;
> >       argc = parse_options(argc, argv, prefix, mt_options,
> >                            merge_tree_usage, 0);
> >       if (o.real && o.trivial)
> >               die(_("--write-tree and --trivial-merge are incompatible"));
> > +     if (!o.real && original_argc < argc)
> > +             die(_("--write-tree must be specified if any other options are"));
>
> Hmm. Well. Hmm.
>
>
> I'd rather keep `--write-tree` neat and optional. What's wrong with
> allowing
>
>         git merge-tree --no-messages HEAD MERGE_HEAD
>
> ?

Yeah, oops.  Nothing is wrong with that...but do note that there is
something wrong with this:

    git merge-tree --no-messages MERGE_BASE HEAD MERGE_HEAD

because the three argument form is signalling the deprecated trivial
merge, and the trivial merge code doesn't handle any options.  You
were right to call out my code since I placed it too early and made it
a bit obtuse.  I've changed it to:

    if (o.mode == 't' && original_argc < argc)
        die(_("--trivial-merge is incompatible with all other options"));

where o.mode is guaranteed to be set before that check.

> To be clear, I think we need this instead:
>
>         if (o.trivial && o.show_messages >= 0)
>                 die(_("--trivial-merge is incompatible with additional options"));

Most of that is better, assuming the o.trivial check handles the
implicit case too.  Thanks.  I'm not so sure about the check on
o.show_messages, though, because then I'll have to keep adding
additional checks for each new additional option (and future patches
will add more).  So, I think I'll keep the original_argc < argc part
of the check instead of that.  :-)

> I like the rest of the patch very much!

Thanks!

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 07/12] merge-tree: support including merge messages in output
  2022-01-26 10:42   ` Christian Couder
@ 2022-01-29  4:52     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  4:52 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

On Wed, Jan 26, 2022 at 2:42 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> >  EXIT STATUS
> >  -----------
> > @@ -72,7 +102,8 @@ be used as a part of a series of steps such as
> >
> >  However, it does not quite fit into the same category of low-level
> >  plumbing commands since the possibility of merge conflicts give it a
> > -much higher chance of the command not succeeding.
> > +much higher chance of the command not succeeding (and NEWTREE containing
> > +a bunch of stuff other than just a toplevel tree).
>
> Is this hunk really related to this commit or should it go into a
> previous commit?

It's meant to first be related to this commit, though as you pointed
out below, I had some accidental stuff not cleaned out of the earlier
commit.

> > @@ -440,22 +441,30 @@ static int real_merge(struct merge_tree_options *o,
> >                 commit_list_insert(j->item, &merge_bases);
> >
> >         merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > -       printf("%s\n", oid_to_hex(&result.tree->object.oid));
> > +
> >         if (result.clean < 0)
> >                 die(_("failure to merge"));
>
> So this addresses the comment I made in a previous commit related to
> the fact that if result.clean < 0 we might not have a valid tree that
> we can print. I think though that it would be better if that was
> addressed in a previous commit.
>
> > -       else if (!result.clean)
> > -               printf(_("Conflicts!\n"));
>
> Ok, so we don't print "Conflicts!\n" now, which makes me wonder if we
> should have printed it in the first place in previous commits.

Yep, good flag on both of these last two comments.  When I was fixing
this up I didn't squash it back in early enough.  Thanks for reading
carefully.

> >         if (o.real && o.trivial)
> >                 die(_("--write-tree and --trivial-merge are incompatible"));
> > +       if (!o.real && original_argc < argc)
> > +               die(_("--write-tree must be specified if any other options are"));
>
> Is this necessary? It looks to me like another thing that would be
> simplified if we were just adding a new command...

I think the code is harder to read than it should be.  I changed it to:

    if (o.mode == 't' && original_argc < argc)
        die(_("--trivial-merge is incompatible with all other options"));

which I think is clearer; it points out that it's only about the
deprecated --trivial-merge option and how it's incompatible with all
other options.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-26 10:55   ` Christian Couder
@ 2022-01-29  4:55     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  4:55 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

On Wed, Jan 26, 2022 at 2:55 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> > After a merge, this function allows the user to extract the same
> > information that would be printed by `ls-files -u` -- conflicted
>
> This made me wonder if "-- conflicted" should be part of the `ls-files
> -u` command. Maybe "by `ls-files -u`, which means" would make things a
> bit clearer.
>
> > files with their mode, oid, and stage.

Yes, I like that wording better.  Thanks!

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-26 11:07   ` Christian Couder
@ 2022-01-29  5:06     ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  5:06 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

On Wed, Jan 26, 2022 at 3:07 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> > +void merge_get_conflicted_files(struct merge_result *result,
> > +                               struct string_list *conflicted_files)
> > +{
> > +       struct hashmap_iter iter;
> > +       struct strmap_entry *e;
> > +       struct merge_options_internal *opti = result->priv;
> > +
> > +       strmap_for_each_entry(&opti->conflicted, &iter, e) {
> > +               const char *path = e->key;
> > +               struct conflict_info *ci = e->value;
> > +               int i;
> > +
> > +               VERIFY_CI(ci);
> > +
> > +               for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> > +                       struct stage_info *si;
> > +
> > +                       if (!(ci->filemask & (1ul << i)))
> > +                               continue;
> > +
> > +                       si = xmalloc(sizeof(*si));
>
> It's probably a premature optimization, so feel free to ignore, but as
> MERGE_BASE and MERGE_SIDE2 are constants, and ci->filemask is constant
> inside the 'for' loop, we could compute before the 'for' loop how many
> 'struct stage_info' we will need and allocate them all at once before
> the 'for' loop.

That's an interesting idea, but if we allocate all at once, then we'd
need to be able to deallocate all at once.  I'm not sure how to do
that, but basically the straightforward
    string_list_free(conflicted_files, 1);
that callers can use with the current function would no longer be
valid, and it might be somewhat difficult for callers to figure out
how to free the memory that was allocated.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-28 16:55   ` Johannes Schindelin
@ 2022-01-29  6:08     ` Elijah Newren
  2022-01-29  8:23       ` Johannes Sixt
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  6:08 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > After a merge, this function allows the user to extract the same
> > information that would be printed by `ls-files -u` -- conflicted
> > files with their mode, oid, and stage.
>
> Hmm. Okay.
>
> I am not really a fan of that output where we use a variable
> number of lines per file. As in: in the regular case, where a file was
> modified in divergent ways, we get three lines. But if it was deleted in
> one branch, or if it was created in both branches, we have only two lines.
>
> Frankly, I'd much rather have 3 lines for each and every conflict.

Out of curiosity, does this also mean you feel like `git ls-files -u`
is mis-designed (even if we're stuck with it for backward
compatibility reasons)?

Anyway, if we make this change guaranteeing there are 3 lines for each
and every conflict, that probably implies we can remove the stage
field (just letting it be implicit), right?  And it also implies that
the "path" field of the lines is now duplicate information.  Should
the path be printed on each line regardless of the duplication?
Should the path be printed separately on its own line, followed by the
three lines of (mode, oid) pairs?  Or should the format be something
else entirely?

> Granted, we currently only have three stages...

I thought that was true too, but Junio informed me otherwise[1].
Granted, ort does not currently make use of that, but it could.  Not
sure I want to rule it out.

[1] https://lore.kernel.org/git/xmqqr1t89gi8.fsf@gitster.c.googlers.com/

> ...and we can pretty much
> guarantee that at least two of these stages are non-empty (otherwise where
> would be the conflict?).

No, that's not true.  See the paragraph about "conflict types" from
the final patch in this series, where I mentioned three different
types of conflicts that can result in a path having a single higher
order stage.

> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> missing from the second conflict, in the output we would see stages 1, 2,
> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> different conflicts.

I don't understand why you're fixating on the stage here.  Why would
you want to group all the stage 2s together, count them up, and then
determine there are N conflicting files because there are N stage 2's?
 In such a design, you'd have to do the extra post-processing to
recognize the all-zero modes and hashes and toss them.

To me, that seems like more work than just handling the `ls-files -u`
style of output, and doesn't have the advantage of showing things in a
format users may already be familiar with (see above about dropping
stages and maybe also pathname).  Since the `ls-files -u` output is
simply several lines of:
   <mode> <hash> <stage> <filename>
You can get the number of conflicted files by counting the number of
*unique* filenames.  If you want to see which lines are about the same
file, get all the lines with the same filename -- they are sorted by
(<filename>, <stage>) for convenience, much like `ls-files -u` is.


On a different angle, I'm also slightly worried there's an embedded
assumption here, one that I tried to address in the "Mistake to avoid"
section I added in my last patch of this series.  What if you have a
conflicting merge and 0 lines of output in the conflicting info
section?  Is there something about the reason you're asking for three
lines of output per conflicted file which is going to make you
overlook that particular possibility and not handle it?

> But still. It makes me uneasy to have that much variability in
> machine-consumable output.
>
> And who knows, maybe if we're ever implementing more sophisticated merge
> strategies for octopus merges, we might end up with more stages...

Um, if we need to handle more stages, wouldn't that undercut your
request for exactly 3 stages?  Wouldn't it suggest that we would
indeed want to have a flexible number of lines?

> In other words...
>
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  merge-ort.c | 31 +++++++++++++++++++++++++++++++
> >  merge-ort.h | 21 +++++++++++++++++++++
> >  2 files changed, 52 insertions(+)
> >
> > diff --git a/merge-ort.c b/merge-ort.c
> > index b78dde55ad9..5e7cea6cc8f 100644
> > --- a/merge-ort.c
> > +++ b/merge-ort.c
> > @@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
> >       trace2_region_leave("merge", "display messages", opt->repo);
> >  }
> >
> > +void merge_get_conflicted_files(struct merge_result *result,
> > +                             struct string_list *conflicted_files)
> > +{
> > +     struct hashmap_iter iter;
> > +     struct strmap_entry *e;
> > +     struct merge_options_internal *opti = result->priv;
> > +
> > +     strmap_for_each_entry(&opti->conflicted, &iter, e) {
> > +             const char *path = e->key;
> > +             struct conflict_info *ci = e->value;
> > +             int i;
> > +
> > +             VERIFY_CI(ci);
> > +
> > +             for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
> > +                     struct stage_info *si;
> > +
> > +                     if (!(ci->filemask & (1ul << i)))
> > +                             continue;
> > +
> > +                     si = xmalloc(sizeof(*si));
> > +                     si->stage = i+1;
> > +                     si->mode = ci->stages[i].mode;
> > +                     oidcpy(&si->oid, &ci->stages[i].oid);
> > +                     string_list_append(conflicted_files, path)->util = si;
> > +             }
> > +     }
> > +     /* string_list_sort() uses a stable sort, so we're good */
> > +     string_list_sort(conflicted_files);
> > +}
> > +
> >  void merge_switch_to_result(struct merge_options *opt,
> >                           struct tree *head,
> >                           struct merge_result *result,
> > diff --git a/merge-ort.h b/merge-ort.h
> > index d643b47cb7c..e635a294ea8 100644
> > --- a/merge-ort.h
> > +++ b/merge-ort.h
> > @@ -2,6 +2,7 @@
> >  #define MERGE_ORT_H
> >
> >  #include "merge-recursive.h"
> > +#include "hash.h"
> >
> >  struct commit;
> >  struct tree;
> > @@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
> >                                  struct merge_result *result,
> >                                  FILE *stream);
> >
> > +struct stage_info {
> > +     struct object_id oid;
> > +     int mode;
> > +     int stage;
> > +};
>
> ... I'd rather not tack this onto a `string_list` but instead have
> something like:
>
>         struct conflict_info {

Nit: "conflict_info" is already taken for another structure; we'd need
a different name.  But it does illustrate the idea just fine.

>                 struct {
>                         struct object_id oid;
>                         int mode;
>                         const char *path;
>                 } stages[3]
>         };
>
> where `path` can be `NULL` to indicate that this stage is missing.

I'll get back to the data structure in a second, but the note about
path being `NULL` is interesting.  I'm presuming that oid and mode are
all-zeros as well, yes?  Now, your original request was that you
wanted a line printed for all three stages.  If we're printing oid,
mode, and path for each stage, what do we print for the path one these
lines?  Is it (1) "(null)", (2) "", or (3) the real pathname (via
carefully comparing to surrounding stages to find out what it is)?  If
we're changing the format to only print the name of the path once,
does the code need to loop over the list of conflicts in order to find
out the name (since the first or second stage might have a NULL path
field)?

In regards to the overall data structure and your comment about
string_list: I'm slightly confused.  You say you want to avoid a
string_list, but you've only accounted for there being one conflict in
this data structure.  I don't see how this suggestion avoids the need
for an array or a string_list or some kind of container to hold
multiple of these.

The inclusion of path within the stages array is also interesting --
should there be (up to) three copies of the pathname allocated per
conflict?  Seems a bit wasteful.  Wouldn't it make more sense to have
something like

    struct conflicts {
        struct {
            struct object_id oid;
            unsigned short mode;
        } stages[3]
    };

and then have a string_list storing the pathname -> conflicts mappings
to avoid the pathname duplication?  Or is there something you really
find problematic about the string_list and you really want an array or
other data structure?

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts
  2022-01-28 16:57   ` Johannes Schindelin
@ 2022-01-29  6:21     ` Elijah Newren
  2022-02-04 23:12       ` Johannes Schindelin
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  6:21 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

On Fri, Jan 28, 2022 at 8:57 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
[...]
> > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > index fd7a867de60..041a4ac2785 100644
> > --- a/Documentation/git-merge-tree.txt
> > +++ b/Documentation/git-merge-tree.txt
> > @@ -58,6 +58,7 @@ simply one line:
> >  Whereas for a conflicted merge, the output is by default of the form:
> >
> >       <OID of toplevel tree>
> > +     <Conflicted file list>
> >       <Informational messages>
>
> To distinguish between the list of conflicted files and the informational
> messages, I think it would be good to insert an empty line, as a
> separator, like.

Yes, I agree; that's why I did so.  :-)

[...]
> >       if (o->show_messages) {
> >               printf("\n");
>
> ... it seems that we do this already...

Yep.

[...]
> > diff --git a/t/t4301-merge-tree-real.sh b/t/t4301-merge-tree-real.sh
> > index c34f8e6c1ed..43c9950dedb 100755
> > --- a/t/t4301-merge-tree-real.sh
> > +++ b/t/t4301-merge-tree-real.sh
> > @@ -94,6 +94,8 @@ test_expect_success 'test conflict notices and such' '
> >       #   "whatever" has *both* a modify/delete and a file/directory conflict
> >       cat <<-EOF >expect &&
> >       HASH
> > +     greeting
> > +     whatever~side1
> >
> >       Auto-merging greeting
> >       CONFLICT (content): Merge conflict in greeting
>
> ... as illustrated by the test, too. I guess the documentation should show
> the empty line, too?

It does:

    Informational messages
    ~~~~~~~~~~~~~~~~~~~~~~

    This always starts with a blank line to separate it from the previous
    sections, and then has free-form messages about the merge, such as:

The newline should not be split out separately, because --no-messages
suppresses the newline.  (Without the messages section, the newline
isn't needed.)

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-26  8:48 ` [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Christian Couder
  2022-01-26 12:02   ` Johannes Schindelin
@ 2022-01-29  7:03   ` Elijah Newren
  2022-01-29  8:17     ` Christian Couder
  1 sibling, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-01-29  7:03 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>
> > Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
> > suggestions and comments on v2):
> >
> >  * Significant changes to output format:
> >    * Flags no longer take a filename for additional output; they write to
> >      stdout instead.
> >    * More information included by default when there are conflicts (no need
> >      to request it with additional flags, instead flags can be used to
> >      suppress it).
> >    * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
> >      information -- when there are conflicts. Add a flag to only list
> >      conflicted files if that's preferred.
>
> The above changes seem good to me.

:-)

> >  * Much more thorough manual for git-merge-tree.txt
> >  * Renamed option from --real to --write-tree
> >  * Accept an optional --trivial-merge option to get old style merge-tree
> >    behavior
> >  * Allow both --write-tree and --trivial-merge to be omitted since we can
> >    deduce which from number of arguments
>
> I still think that it might be simpler and cleaner to leave 'git
> merge-tree' alone for now, and just add a new command named for
> example 'git write-merge-tree'. Later we can always add flags to 'git
> merge-tree' or add 'git trivial-merge-tree' as an alias for 'git
> merge-tree', and eventually slowly switch 'git merge-tree' to mean
> only 'git write-merge-tree' if that's where we want to go.

Understood.  Since we can't immediately kill the old merge-tree, I
don't think there's a perfectly clean solution here, and it's totally
understandable that different folks would have different opinions on
which interim choice would be cleanest.  What you suggest is
reasonable, I just personally prefer the path in this series.

> >  * Document exit code when the merge cannot be run (so we can distinguish
> >    other error cases from conflicts)
> >  * testcase cleanups: test_tick, early skip of test when using recursive
> >    backend, variable renames, etc.
> >  * various minor code cleanups
> >  * Add a new --allow-unrelated-histories option (with same meaning as the
> >    one used in git merge)
>
> The above changes seem good to me too.

Thanks for reading over things carefully and for providing many
detailed, helpful comments!

> > Stuff intentionally NOT included, but which others seemed to feel strongly
> > about; they'd need to convince me more on these:
> >
> >  * Any form of diff output[1]
>
> It's not a big issue for me to not include them right now as long as
> it's possible to add cli options later that add them.

My main concern is just that `merge-tree` remain a low-level tool and
have machine-parseable output.  I was a little worried that both you
and Dscho wanted everything on stdout rather than in separate files,
as the <Informational messages> part of the output is rather
free-form.  But since it's at the end, and has a machine-parseable
beginning, it can just be slurped in and we're all good.  The diff
output raises my eyebrow because I'm worried we're losing this
property.  If there are clear usecases for adding more output, and we
can do so without losing this machine-parseable property, I don't have
a problem with adding an option for it.

One analogy we might use here is that `git merge` provides a diffstat
at the end.  What you're asking is more than a diffstat, but might be
considered similar-ish in nature.

> The reason is
> that I think in many cases when there are conflicts, the conflicts
> will be small and the user will want to see them.

I'm a little worried about the assumption here that conflict size is
measurable and visible via diffs.  That might be true in some cases,
but a UI written with that assumption is going to be very confusing
when hitting cases where that assumption does not hold.  For example:

  * What if there is a binary file conflict, or a modify/delete or
rename/delete conflict, or failed-to-merge submodule conflict, or a
file location conflict? (For these, there is no diff relative to the
first parent and hence this conflict would have no diff output for
it)?
  * What if there was a simple file/directory conflict?  A diff would
show a rename (even when neither side did any renames), but not any
conflict markers.
  * What if there was a rename/rename conflict (both sides renamed
same file differently) or a distinct types conflict?  The former
results in three different conflicting files, none of them with
conflict markers, while the latter results in two different
conflicting files both without conflict markers?  Showing individual
per-file diffs totally loses all context here -- it'll show no-diff
for one of the files, and totally new additions for the ones.

Such a problem statement just seems fraught with edge cases to me, and
suggests that the problem statement might be in need of revisiting.

Don't read this as me closing the door on the possibility of diffs;
I'm not trying to do that.  I'm listing my misgivings about how I
think they might be used (i.e. be careful if you're headed down this
path as you might be digging yourself a never-ending support hole).
You can also think of my comments as feedback to consider and address
when you propose a future feature addition for adding diffs.  If/when
you propose such a feature, we'd probably be able to dive more into
specifics and usecases at that time, which may or may not circumvent
my concerns.

> So it would be
> simpler to just have an option to show any conflict right away, rather
> than have the user launch another command (a diff-tree against which
> tree and with which options?).

Um, this part I'm not sure I get.  I thought the reason for the diffs
was performance -- you knew you wanted the diffs, and you wanted it
done as part of the same process.  But why would this be simpler?
Your patch series included three different diffs, and the emails you
pointed me at suggested all kinds of configurability.  That suggests
the merge-tree command would have to take the exact same options the
user would supply to diff, and thus would have to be told all the same
options, right?  I don't see how this removes any complexity at all
for the user.

Unless...is the request in some way similar to merge's diffstat where
there is always a very specific type of diff that is wanted and you
aren't envisioning much flexibility in what kind of diff or what to
diff against -- is that where the simplification comes from?

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-29  7:03   ` Elijah Newren
@ 2022-01-29  8:17     ` Christian Couder
  2022-01-29 17:43       ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Christian Couder @ 2022-01-29  8:17 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

On Sat, Jan 29, 2022 at 8:04 AM Elijah Newren <newren@gmail.com> wrote:
>
> On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
> <christian.couder@gmail.com> wrote:
> >
> > On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:

> > > Stuff intentionally NOT included, but which others seemed to feel strongly
> > > about; they'd need to convince me more on these:
> > >
> > >  * Any form of diff output[1]
> >
> > It's not a big issue for me to not include them right now as long as
> > it's possible to add cli options later that add them.
>
> My main concern is just that `merge-tree` remain a low-level tool and
> have machine-parseable output.  I was a little worried that both you
> and Dscho wanted everything on stdout rather than in separate files,
> as the <Informational messages> part of the output is rather
> free-form.  But since it's at the end, and has a machine-parseable
> beginning, it can just be slurped in and we're all good.  The diff
> output raises my eyebrow because I'm worried we're losing this
> property.  If there are clear usecases for adding more output, and we
> can do so without losing this machine-parseable property, I don't have
> a problem with adding an option for it.

That's ok for me for now. I will certainly not work on adding options
for diff output without any usecase.

> One analogy we might use here is that `git merge` provides a diffstat
> at the end.  What you're asking is more than a diffstat, but might be
> considered similar-ish in nature.
>
> > The reason is
> > that I think in many cases when there are conflicts, the conflicts
> > will be small and the user will want to see them.
>
> I'm a little worried about the assumption here that conflict size is
> measurable and visible via diffs.  That might be true in some cases,
> but a UI written with that assumption is going to be very confusing
> when hitting cases where that assumption does not hold.  For example:
>
>   * What if there is a binary file conflict, or a modify/delete or
> rename/delete conflict, or failed-to-merge submodule conflict, or a
> file location conflict? (For these, there is no diff relative to the
> first parent and hence this conflict would have no diff output for
> it)?
>   * What if there was a simple file/directory conflict?  A diff would
> show a rename (even when neither side did any renames), but not any
> conflict markers.
>   * What if there was a rename/rename conflict (both sides renamed
> same file differently) or a distinct types conflict?  The former
> results in three different conflicting files, none of them with
> conflict markers, while the latter results in two different
> conflicting files both without conflict markers?  Showing individual
> per-file diffs totally loses all context here -- it'll show no-diff
> for one of the files, and totally new additions for the ones.

In those cases we just tell users that they cannot resolve those
conflicts in the user interface, see the following doc:

https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#conflicts-you-can-resolve-in-the-user-interface

> Such a problem statement just seems fraught with edge cases to me, and
> suggests that the problem statement might be in need of revisiting.

Users understand that some kinds of conflicts cannot yet be resolved
using a user interface. Maybe we will be able to make improvements so
that more kinds of conflicts can be resolved in a UI in the future
though. That's why a flexible and extensible output could help.

> Don't read this as me closing the door on the possibility of diffs;
> I'm not trying to do that.  I'm listing my misgivings about how I
> think they might be used (i.e. be careful if you're headed down this
> path as you might be digging yourself a never-ending support hole).
> You can also think of my comments as feedback to consider and address
> when you propose a future feature addition for adding diffs.  If/when
> you propose such a feature, we'd probably be able to dive more into
> specifics and usecases at that time, which may or may not circumvent
> my concerns.

I know that diffs, or any new single feature, will likely not be a
silver bullet.

> > So it would be
> > simpler to just have an option to show any conflict right away, rather
> > than have the user launch another command (a diff-tree against which
> > tree and with which options?).
>
> Um, this part I'm not sure I get.  I thought the reason for the diffs
> was performance -- you knew you wanted the diffs, and you wanted it
> done as part of the same process.  But why would this be simpler?

In the commit message of 4/12 you show an example of using it in simple scripts:

NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
test $? -eq 0 || die "There were conflicts..."
...

So I think it would be simpler for someone interested in seeing the
conflicts, like a script writer, or maybe someone using it manually
for example as a dry run before performing a merge, to be able to get
them right away from the command rather than to have to use another
command (which means finding the right command, arguments and options
for that) to get them.

> Your patch series included three different diffs, and the emails you
> pointed me at suggested all kinds of configurability.  That suggests
> the merge-tree command would have to take the exact same options the
> user would supply to diff, and thus would have to be told all the same
> options, right?  I don't see how this removes any complexity at all
> for the user.
>
> Unless...is the request in some way similar to merge's diffstat where
> there is always a very specific type of diff that is wanted and you
> aren't envisioning much flexibility in what kind of diff or what to
> diff against -- is that where the simplification comes from?

Well I just think the default diff output could be tailored for the
most likely usecases and options made available later for more
advanced usecases or users.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-29  6:08     ` Elijah Newren
@ 2022-01-29  8:23       ` Johannes Sixt
  2022-01-29 16:47         ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Johannes Sixt @ 2022-01-29  8:23 UTC (permalink / raw)
  To: Elijah Newren, Johannes Schindelin
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

Just a heckling from the peanut gallery...

Am 29.01.22 um 07:08 schrieb Elijah Newren:
> On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
>> missing from the second conflict, in the output we would see stages 1, 2,
>> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
>> different conflicts.
> 
> I don't understand why you're fixating on the stage here.  Why would
> you want to group all the stage 2s together, count them up, and then
> determine there are N conflicting files because there are N stage 2's?

Looks like you are misunderstanding Dscho's point: When you have two
conflicts, the first with stages 1 and 2, the second with stages 2 and
3, then the 2s occur lumped together when the 4 lines are printed in a
row, and that is the cue to the parser where the new conflict begins.
Dscho did not mean that all N 2s of should be listed together.

-- Hannes

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-29  8:23       ` Johannes Sixt
@ 2022-01-29 16:47         ` Elijah Newren
  2022-02-04 23:10           ` Johannes Schindelin
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-01-29 16:47 UTC (permalink / raw)
  To: Johannes Sixt
  Cc: Johannes Schindelin, Elijah Newren via GitGitGadget,
	Git Mailing List, Christian Couder, Taylor Blau,
	Johannes Altmanninger, Ramsay Jones, Christian Couder,
	René Scharfe

On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
>
> Just a heckling from the peanut gallery...
>
> Am 29.01.22 um 07:08 schrieb Elijah Newren:
> > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> >> missing from the second conflict, in the output we would see stages 1, 2,
> >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> >> different conflicts.
> >
> > I don't understand why you're fixating on the stage here.  Why would
> > you want to group all the stage 2s together, count them up, and then
> > determine there are N conflicting files because there are N stage 2's?
>
> Looks like you are misunderstanding Dscho's point: When you have two
> conflicts, the first with stages 1 and 2, the second with stages 2 and
> 3, then the 2s occur lumped together when the 4 lines are printed in a
> row, and that is the cue to the parser where the new conflict begins.
> Dscho did not mean that all N 2s of should be listed together.

Ah, so...I didn't understand his misunderstanding?  Using stages as a
cue to the parser where the new conflict begins is broken; you should
instead check for when the filename listed on a line does not match
the filename on the previous line.  In particular, if one conflict has
stages 1 and 2, and the next conflict has only stage 3, then looking
at stages only might cause you to accidentally lump unrelated
conflicts together.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-29  8:17     ` Christian Couder
@ 2022-01-29 17:43       ` Elijah Newren
  2022-01-31 17:45         ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-01-29 17:43 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

Hi Christian,

On Sat, Jan 29, 2022 at 12:18 AM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Sat, Jan 29, 2022 at 8:04 AM Elijah Newren <newren@gmail.com> wrote:
> >
> > On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
> > <christian.couder@gmail.com> wrote:
> > >
> > > On Sat, Jan 22, 2022 at 10:56 PM Elijah Newren via GitGitGadget
> > > <gitgitgadget@gmail.com> wrote:
>
> > > > Stuff intentionally NOT included, but which others seemed to feel strongly
> > > > about; they'd need to convince me more on these:
> > > >
> > > >  * Any form of diff output[1]
> > >
> > > It's not a big issue for me to not include them right now as long as
> > > it's possible to add cli options later that add them.
> >
> > My main concern is just that `merge-tree` remain a low-level tool and
> > have machine-parseable output.  I was a little worried that both you
> > and Dscho wanted everything on stdout rather than in separate files,
> > as the <Informational messages> part of the output is rather
> > free-form.  But since it's at the end, and has a machine-parseable
> > beginning, it can just be slurped in and we're all good.  The diff
> > output raises my eyebrow because I'm worried we're losing this
> > property.  If there are clear usecases for adding more output, and we
> > can do so without losing this machine-parseable property, I don't have
> > a problem with adding an option for it.
>
> That's ok for me for now. I will certainly not work on adding options
> for diff output without any usecase.
>
> > One analogy we might use here is that `git merge` provides a diffstat
> > at the end.  What you're asking is more than a diffstat, but might be
> > considered similar-ish in nature.
> >
> > > The reason is
> > > that I think in many cases when there are conflicts, the conflicts
> > > will be small and the user will want to see them.
> >
> > I'm a little worried about the assumption here that conflict size is
> > measurable and visible via diffs.  That might be true in some cases,
> > but a UI written with that assumption is going to be very confusing
> > when hitting cases where that assumption does not hold.  For example:
> >
> >   * What if there is a binary file conflict, or a modify/delete or
> > rename/delete conflict, or failed-to-merge submodule conflict, or a
> > file location conflict? (For these, there is no diff relative to the
> > first parent and hence this conflict would have no diff output for
> > it)?
> >   * What if there was a simple file/directory conflict?  A diff would
> > show a rename (even when neither side did any renames), but not any
> > conflict markers.
> >   * What if there was a rename/rename conflict (both sides renamed
> > same file differently) or a distinct types conflict?  The former
> > results in three different conflicting files, none of them with
> > conflict markers, while the latter results in two different
> > conflicting files both without conflict markers?  Showing individual
> > per-file diffs totally loses all context here -- it'll show no-diff
> > for one of the files, and totally new additions for the ones.
>
> In those cases we just tell users that they cannot resolve those
> conflicts in the user interface, see the following doc:
>
> https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#conflicts-you-can-resolve-in-the-user-interface

So...I think you may have just convinced me that my fears were
justified and that I should probably NAK any attempt to add diffs to
the merge-tree command.  I won't jump to conclusions but you've
provided some pretty strong signal to me against going down that
route.  The list of limitations in the link you provide do mostly
avoid the broken cases I listed above, but it enshrines those
limitations on that webpage as fundamental rather than just as current
implementation shortcomings.  You may not be able to remove those
limitations on that webpage without either expunging the diffs from
the UI or exposing the brokenness of the various cases above.

If you do propose a diff option in the future, come prepared to
discuss how you'll avoid accidentally leading others down into paths
with the same fundamental issues, and/or how the above types of
conflicts might still be meaningfully handled.

Also, the list of limitations you have may not be quite comprehensive
enough to avoid all problems (though it certainly avoids most of
them).  Can I ask a couple clarifying questions about your list of
limitations in that link? :

  * When that page says the file cannot already contain conflict
markers, is the check performed on the version of the file in the two
trees being merged, or is the check performed on the 2nd and 3rd index
stage of the merge result (these are not equivalent checks, even if
they often give the same answer)?
  * When that page says the file must already exist in the same path
on both branches, is the check performed on by checking the path in
the two trees being merged, or is the check performed on the 2nd and
3rd index stage of the merge result (again, these are not equivalent
checks)?

> > Such a problem statement just seems fraught with edge cases to me, and
> > suggests that the problem statement might be in need of revisiting.
>
> Users understand that some kinds of conflicts cannot yet be resolved
> using a user interface. Maybe we will be able to make improvements so
> that more kinds of conflicts can be resolved in a UI in the future
> though. That's why a flexible and extensible output could help.

I think future improvements to handle more conflict types may well
hinge on removing the diff-output-using portion of your interface; I
think it may well be fundamentally incompatible.

I agree we want to leave the output format open for extension, I'm
just saying we have to be careful about what extensions are included
and this one worries me.

> > Don't read this as me closing the door on the possibility of diffs;
> > I'm not trying to do that.  I'm listing my misgivings about how I
> > think they might be used (i.e. be careful if you're headed down this
> > path as you might be digging yourself a never-ending support hole).
> > You can also think of my comments as feedback to consider and address
> > when you propose a future feature addition for adding diffs.  If/when
> > you propose such a feature, we'd probably be able to dive more into
> > specifics and usecases at that time, which may or may not circumvent
> > my concerns.
>
> I know that diffs, or any new single feature, will likely not be a
> silver bullet.

Sure that's fair; not being a silver bullet is fine.  We do need to
avoid providing a kryptonite bullet, though.

> > > So it would be
> > > simpler to just have an option to show any conflict right away, rather
> > > than have the user launch another command (a diff-tree against which
> > > tree and with which options?).
> >
> > Um, this part I'm not sure I get.  I thought the reason for the diffs
> > was performance -- you knew you wanted the diffs, and you wanted it
> > done as part of the same process.  But why would this be simpler?
>
> In the commit message of 4/12 you show an example of using it in simple scripts:
>
> NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> test $? -eq 0 || die "There were conflicts..."
> ...
>
> So I think it would be simpler for someone interested in seeing the
> conflicts, like a script writer, or maybe someone using it manually
> for example as a dry run before performing a merge, to be able to get
> them right away from the command...

Okay, but the command already does that.  When there are conflicts,
NEWTREE won't actually be a tree; it'll be lots of lines of output.
That's (part of) the reason for the exit status check.  So users
already get that information from the command.

> ...rather than to have to use another
> command (which means finding the right command, arguments and options
> for that) to get them.

As for finding the right arguments and options...

> > Your patch series included three different diffs, and the emails you
> > pointed me at suggested all kinds of configurability.  That suggests
> > the merge-tree command would have to take the exact same options the
> > user would supply to diff, and thus would have to be told all the same
> > options, right?  I don't see how this removes any complexity at all
> > for the user.
> >
> > Unless...is the request in some way similar to merge's diffstat where
> > there is always a very specific type of diff that is wanted and you
> > aren't envisioning much flexibility in what kind of diff or what to
> > diff against -- is that where the simplification comes from?
>
> Well I just think the default diff output could be tailored for the
> most likely usecases and options made available later for more
> advanced usecases or users.

Ah, so this may go against your earlier comments at [1] about a
merge-tree on steroids and a huge array of diff options, or against
your comments about diffs not being provided by default (also [1]).
Because if you have that huge range of diff options, and the diffs are
not provided by default, then it's not clear how you've simplified
things because users would still need to figure out the right
arguments and options to pass, it's just that the user would have to
pass all (or maybe just most?) of those arguments and options to
merge-tree instead of to diff.  Or is the simpler UI you're discussing
really just about not needing to include 1 argument, the name of the
new toplevel tree, since that single argument could be implicit?

I'm having a hard time buying the "simpler UI for script writers"
angle of argument here, especially for script writers who should fully
be able to look up the appropriate commands and use them.  Your
earlier arguments about performance being important (having both the
merge & the diff in the same process) seemed much more convincing to
me.  But maybe I'm still just missing something about your "simpler"
angle?

[1] https://lore.kernel.org/git/CAP8UFD0wKnAg5oyMWchXysPTg3K9Vb4M1tRcPzPE81QM903pYg@mail.gmail.com/

^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 00/13] In-core git merge-tree ("Server side merges")
  2022-01-22 21:55 [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                   ` (13 preceding siblings ...)
  2022-01-28 17:00 ` Johannes Schindelin
@ 2022-01-29 18:07 ` Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 01/13] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
                     ` (13 more replies)
  14 siblings, 14 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren

Note: Depends on en/remerge-diff

(Note2: This is a continuation of my series at [0], but I can't change the
base repository for a pull request in GitHub so I had to open a new PR and
that makes it look like a new series. [0]
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/)

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs.

== Updates Log ==

Stuff NOT included that reviewers brought up in the last round:

 * Very generic (mode, oid, stage, filename) printing formatting[1]
 * Always printing 3 stages for each filename with conflicts[2] [1]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [2]
   https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/

Updates since v3 (which looks like v1 due to the re-submit; thanks to René,
Ævar, Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

Updates since v2 (thanks to Christian, Dscho, Ramsay, and René for
suggestions and comments on v2):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since v1 (thanks to Johannes Altmanninger and Fabian for suggestions
on v1):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-ort: allow update messages to be written to different file
    stream
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

Johannes Schindelin (1):
  diff: allow diff_warn_rename_limit to write somewhere besides stdout

 Documentation/git-merge-tree.txt | 179 ++++++++++++++++++++++++++++---
 builtin/merge-tree.c             | 162 +++++++++++++++++++++++++---
 diff.c                           |  20 ++--
 diff.h                           |   3 +-
 git.c                            |   2 +-
 merge-ort.c                      | 109 ++++++++++++-------
 merge-ort.h                      |  30 ++++++
 merge-recursive.c                |   3 +-
 t/t4301-merge-tree-write-tree.sh | 164 ++++++++++++++++++++++++++++
 9 files changed, 603 insertions(+), 69 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v1:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  65fdae9ddba !  3:  63f42df21ae merge-tree: add option parsing and initial shell for real merge function
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
       }
       
      +struct merge_tree_options {
     -+	int real;
     -+	int trivial;
     ++	int mode;
      +};
      +
      +static int real_merge(struct merge_tree_options *o,
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
      +		NULL
      +	};
      +	struct option mt_options[] = {
     -+		OPT_BOOL(0, "write-tree", &o.real,
     -+			 N_("do a real merge instead of a trivial merge")),
     -+		OPT_BOOL(0, "trivial-merge", &o.trivial,
     -+			 N_("do a trivial merge only")),
     ++		OPT_CMDMODE(0, "write-tree", &o.mode,
     ++			    N_("do a real merge instead of a trivial merge"),
     ++			    'w'),
     ++		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     ++			    N_("do a trivial merge only"), 't'),
      +		OPT_END()
      +	};
      +
     -+	/* Check for a request for basic help */
     -+	if (argc == 2 && !strcmp(argv[1], "-h"))
     -+		usage_with_options(merge_tree_usage, mt_options);
     -+
      +	/* Parse arguments */
      +	argc = parse_options(argc, argv, prefix, mt_options,
     -+			     merge_tree_usage, 0);
     -+	if (o.real && o.trivial)
     -+		die(_("--write-tree and --trivial-merge are incompatible"));
     -+	if (o.real || o.trivial) {
     -+		expected_remaining_argc = (o.real ? 2 : 3);
     ++			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     ++	if (o.mode) {
     ++		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
      +		if (argc != expected_remaining_argc)
      +			usage_with_options(merge_tree_usage, mt_options);
      +	} else {
      +		if (argc < 2 || argc > 3)
      +			usage_with_options(merge_tree_usage, mt_options);
     -+		o.real = (argc == 2);
     ++		o.mode = (argc == 2 ? 'w' : 't');
      +	}
      +
      +	/* Do the relevant type of merge */
     -+	if (o.real)
     ++	if (o.mode == 'w')
      +		return real_merge(&o, argv[0], argv[1]);
      +	else
      +		return trivial_merge(argv[0], argv[1], argv[2]);
  4:  05bd17686e1 !  4:  02c29f920d0 merge-tree: implement real merges
     @@ Commit message
      
          The only output is:
            - the toplevel resulting tree printed on stdout
     -      - exit status of 0 (clean) or 1 (conflicts present)
     +      - exit status of 0 (clean), 1 (conflicts present), anything else
     +        (merge could not be performed; unknown if clean or conflicted)
      
          This output is meant to be used by some higher level script, perhaps in
          a sequence of steps like this:
     @@ Commit message
          preliminary implementation, but subsequent commits will add that
          ability.
      
     +    This also marks the traditional trivial merge of merge-tree as
     +    deprecated.  The trivial merge not only had limited applicability, the
     +    output format was also difficult to work with (and its format
     +    undocumented), and will generally be less performant than real merges.
     +
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
       [verse]
      -'git merge-tree' <base-tree> <branch1> <branch2>
      +'git merge-tree' [--write-tree] <branch1> <branch2>
     -+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
     ++'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
       DESCRIPTION
       -----------
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +Performs a merge, but does not make any new commits and does not read
      +from or write to either the working tree or index.
      +
     -+The first form will merge the two branches, doing a full recursive
     -+merge with rename detection.  The rest of this manual (other than the
     -+next paragraph) describes the first form in more detail -- including
     -+options, output format, exit status, and usage notes.
     -+
     -+The second form is deprecated; it is kept for backward compatibility
     -+reasons but may be deleted in the future.  It will only do a trivial
     -+merge.  It reads three tree-ish, and outputs trivial merge results and
     -+conflicting stages to the standard output in a semi-diff format.
     -+Since this was designed for higher level scripts to consume and merge
     -+the results back into the index, it omits entries that match
     -+<branch1>.  The result of this second form is is similar to what
     -+three-way 'git read-tree -m' does, but instead of storing the results
     -+in the index, the command outputs the entries to the standard output.
     -+This form not only has limited applicability, the output format is
     -+also difficult to work with, and it will generally be less performant
     -+than the first form even on successful merges (especially if working
     -+in large repositories).  The remainder of this manual will only
     -+discuss the first form.
     ++The second form is deprecated and supported only for backward
     ++compatibility.  It will likely be removed in the future, and will not
     ++be discussed further in this manual.
     ++
     ++The first form will merge the two branches, doing a real merge.  A real
     ++merge is distinguished from a trivial merge in that it includes:
     ++
     ++  * three way content merges of individual files
     ++  * rename detection
     ++  * proper directory/file conflict handling
     ++  * recursive ancestor consolidation (i.e. when there is more than one
     ++    merge base, creating a virtual merge base by merging the merge bases)
     ++  * etc.
     ++
     ++After the merge completes, it will create a new toplevel tree object.
     ++See `OUTPUT` below for details.
      +
      +OUTPUT
      +------
     @@ builtin/merge-tree.c: struct merge_tree_options {
      +
      +	parent1 = get_merge_parent(branch1);
      +	if (!parent1)
     -+		help_unknown_ref(branch1, "merge",
     ++		help_unknown_ref(branch1, "merge-tree",
      +				 _("not something we can merge"));
      +
      +	parent2 = get_merge_parent(branch2);
      +	if (!parent2)
     -+		help_unknown_ref(branch2, "merge",
     ++		help_unknown_ref(branch2, "merge-tree",
      +				 _("not something we can merge"));
      +
      +	init_merge_options(&opt, the_repository);
     -+	/*
     -+	 * TODO: Support subtree and other -X options?
     -+	if (use_strategies_nr == 1 &&
     -+	    !strcmp(use_strategies[0]->name, "subtree"))
     -+		opt.subtree_shift = "";
     -+	for (x = 0; x < xopts_nr; x++)
     -+		if (parse_merge_opt(&opt, xopts[x]))
     -+			die(_("Unknown strategy option: -X%s"), xopts[x]);
     -+	*/
      +
      +	opt.show_rename_progress = 0;
      +
     -+	opt.branch1 = merge_remote_util(parent1)->name; /* or just branch1? */
     -+	opt.branch2 = merge_remote_util(parent2)->name; /* or just branch2? */
     ++	opt.branch1 = branch1;
     ++	opt.branch2 = branch2;
      +
      +	/*
      +	 * Get the merge bases, in reverse order; see comment above
     @@ builtin/merge-tree.c: struct merge_tree_options {
      +		commit_list_insert(j->item, &merge_bases);
      +
      +	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
     -+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
      +	if (result.clean < 0)
      +		die(_("failure to merge"));
     -+	else if (!result.clean)
     -+		printf(_("Conflicts!\n"));
     ++	puts(oid_to_hex(&result.tree->object.oid));
      +	merge_finalize(&opt, &result);
      +	return !result.clean; /* result.clean < 0 handled above */
       }
       
       int cmd_merge_tree(int argc, const char **argv, const char *prefix)
      
     - ## t/t4301-merge-tree-real.sh (new) ##
     + ## t/t4301-merge-tree-write-tree.sh (new) ##
      @@
      +#!/bin/sh
      +
     @@ t/t4301-merge-tree-real.sh (new)
      +. ./test-lib.sh
      +
      +# This test is ort-specific
     -+test "${GIT_TEST_MERGE_ALGORITHM:-ort}" = ort || {
     ++if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
     ++then
      +	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
      +	test_done
     -+}
     ++fi
      +
      +test_expect_success setup '
      +	test_write_lines 1 2 3 4 5 >numbers &&
  -:  ----------- >  5:  6fb4f4580a5 diff: allow diff_warn_rename_limit to write somewhere besides stdout
  5:  095aa266c2b !  6:  28368c03898 merge-ort: split out a separate display_update_messages() function
     @@ Metadata
       ## Commit message ##
          merge-ort: split out a separate display_update_messages() function
      
     -    No functional changes included in this patch; it's just a preparatory
     -    step to allow the printed messages to be handled differently by other
     -    callers, such as in `git merge-tree --write-tree`.
     +    This patch includes no new code; it simply moves a bunch of lines into a
     +    new function.  As such, there are no functional changes.  This is just a
     +    preparatory step to allow the printed messages to be handled differently
     +    by other callers, such as in `git merge-tree --write-tree`.
     +
     +    (Patch best viewed with
     +         --color-moved --color-moved-ws=allow-indentation-change
     +     to see that it is a simple code movement.)
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
      +
      +	/* Also include needed rename limit adjustment now */
      +	diff_warn_rename_limit("merge.renamelimit",
     -+			       opti->renames.needed_limit, 0);
     ++			       opti->renames.needed_limit, 0, stdout);
      +
      +	trace2_region_leave("merge", "display messages", opt->repo);
      +}
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
      -
      -		/* Also include needed rename limit adjustment now */
      -		diff_warn_rename_limit("merge.renamelimit",
     --				       opti->renames.needed_limit, 0);
     +-				       opti->renames.needed_limit, 0, stdout);
      -
      -		trace2_region_leave("merge", "display messages", opt->repo);
      -	}
  6:  e3ef17eb46f !  7:  593d0c00b57 merge-ort: allow update messages to be written to different file stream
     @@ merge-ort.c: void merge_display_update_messages(struct merge_options *opt,
       		struct strbuf *sb = olist.items[i].util;
       
      -		printf("%s", sb->buf);
     -+		fprintf(stream, "%s", sb->buf);
     ++		strbuf_write(sb, stream);
       	}
       	string_list_clear(&olist, 0);
       
     + 	/* Also include needed rename limit adjustment now */
     + 	diff_warn_rename_limit("merge.renamelimit",
     +-			       opti->renames.needed_limit, 0, stdout);
     ++			       opti->renames.needed_limit, 0, stream);
     + 
     + 	trace2_region_leave("merge", "display messages", opt->repo);
     + }
      @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
       	}
       
  7:  2f296aeeefb !  8:  d0d30e92ecd merge-tree: support including merge messages in output
     @@ Commit message
          When running `git merge-tree --write-tree`, we previously would only
          return an exit status reflecting the cleanness of a merge, and print out
          the toplevel tree of the resulting merge.  Merges also have
     -    informational messages, ("Auto-merging <PATH>", "CONFLICT (content):
     -    ...", "CONFLICT (file/directory)", etc.)  In fact, when non-content
     -    conflicts occur (such as file/directory, modify/delete, add/add with
     -    differing modes, rename/rename (1to2), etc.), these informational
     -    messages are often the only notification since these conflicts are not
     -    representable in the contents of the file.
     +    informational messages, such as:
     +      * "Auto-merging <PATH>"
     +      * "CONFLICT (content): ..."
     +      * "CONFLICT (file/directory)"
     +      * etc.
     +    In fact, when non-content conflicts occur (such as file/directory,
     +    modify/delete, add/add with differing modes, rename/rename (1to2),
     +    etc.), these informational messages may be the only notification the
     +    user gets since these conflicts are not representable in the contents
     +    of the file.
      
          Add a --[no-]messages option so that callers can request these messages
          be included at the end of the output.  Include such messages by default
     @@ Documentation/git-merge-tree.txt: git-merge-tree - Perform merge without touchin
       [verse]
      -'git merge-tree' [--write-tree] <branch1> <branch2>
      +'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
     - 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2>
     + 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
       DESCRIPTION
     -@@ Documentation/git-merge-tree.txt: than the first form even on successful merges (especially if working
     - in large repositories).  The remainder of this manual will only
     - discuss the first form.
     +@@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in that it includes:
     + After the merge completes, it will create a new toplevel tree object.
     + See `OUTPUT` below for details.
       
      +OPTIONS
      +-------
     @@ Documentation/git-merge-tree.txt: be used as a part of a series of steps such as
      
       ## builtin/merge-tree.c ##
      @@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     + 
       struct merge_tree_options {
     - 	int real;
     - 	int trivial;
     + 	int mode;
      +	int show_messages;
       };
       
       static int real_merge(struct merge_tree_options *o,
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
     - 		commit_list_insert(j->item, &merge_bases);
     - 
       	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
     --	printf("%s\n", oid_to_hex(&result.tree->object.oid));
     -+
       	if (result.clean < 0)
       		die(_("failure to merge"));
     --	else if (!result.clean)
     --		printf(_("Conflicts!\n"));
      +
      +	if (o->show_messages == -1)
      +		o->show_messages = !result.clean;
      +
     -+	printf("%s\n", oid_to_hex(&result.tree->object.oid));
     + 	puts(oid_to_hex(&result.tree->object.oid));
      +	if (o->show_messages) {
      +		printf("\n");
      +		merge_display_update_messages(&opt, &result, stdout);
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		NULL
       	};
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			 N_("do a real merge instead of a trivial merge")),
     - 		OPT_BOOL(0, "trivial-merge", &o.trivial,
     - 			 N_("do a trivial merge only")),
     + 			    'w'),
     + 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     + 			    N_("do a trivial merge only"), 't'),
      +		OPT_BOOL(0, "messages", &o.show_messages,
      +			 N_("also show informational/conflict messages")),
       		OPT_END()
       	};
       
     -@@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 		usage_with_options(merge_tree_usage, mt_options);
     - 
       	/* Parse arguments */
      +	original_argc = argc;
       	argc = parse_options(argc, argv, prefix, mt_options,
     - 			     merge_tree_usage, 0);
     - 	if (o.real && o.trivial)
     - 		die(_("--write-tree and --trivial-merge are incompatible"));
     -+	if (!o.real && original_argc < argc)
     -+		die(_("--write-tree must be specified if any other options are"));
     - 	if (o.real || o.trivial) {
     - 		expected_remaining_argc = (o.real ? 2 : 3);
     - 		if (argc != expected_remaining_argc)
     + 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     + 	if (o.mode) {
     +@@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     + 			usage_with_options(merge_tree_usage, mt_options);
     + 		o.mode = (argc == 2 ? 'w' : 't');
     + 	}
     ++	if (o.mode == 't' && original_argc < argc)
     ++		die(_("--trivial-merge is incompatible with all other options"));
     + 
     + 	/* Do the relevant type of merge */
     + 	if (o.mode == 'w')
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Barf on too many arguments' '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
       	grep "^usage: git merge-tree" expect
       '
       
  8:  35e0ed9271a !  9:  9c2334ae9f2 merge-ort: provide a merge_get_conflicted_files() helper function
     @@ Commit message
          merge-ort: provide a merge_get_conflicted_files() helper function
      
          After a merge, this function allows the user to extract the same
     -    information that would be printed by `ls-files -u` -- conflicted
     +    information that would be printed by `ls-files -u`, which means
          files with their mode, oid, and stage.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
  9:  fcbb087fa88 ! 10:  243134dc247 merge-tree: provide a list of which files have conflicts
     @@ builtin/merge-tree.c: struct merge_tree_options {
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		o->show_messages = !result.clean;
       
     - 	printf("%s\n", oid_to_hex(&result.tree->object.oid));
     + 	puts(oid_to_hex(&result.tree->object.oid));
      +	if (!result.clean) {
      +		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
      +		const char *last = NULL;
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
       
       	/* Do the relevant type of merge */
     - 	if (o.real)
     + 	if (o.mode == 'w')
      -		return real_merge(&o, argv[0], argv[1]);
      +		return real_merge(&o, argv[0], argv[1], prefix);
       	else
       		return trivial_merge(argv[0], argv[1], argv[2]);
       }
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such' '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
       	#   "whatever" has *both* a modify/delete and a file/directory conflict
       	cat <<-EOF >expect &&
       	HASH
     @@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such'
       
       	Auto-merging greeting
       	CONFLICT (content): Merge conflict in greeting
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
       	test_cmp expect actual
       '
       
 10:  050add3e498 ! 11:  c322e4c6938 merge-tree: provide easy access to `ls-files -u` style info
     @@ Commit message
          Much like `git merge` updates the index with information of the form
              (mode, oid, stage, name)
          provide this output for conflicted files for merge-tree as well.
     -    Provide an --exclude-oids-and-modes option for users to exclude the
     -    mode, oid, and stage and only get the list of conflicted filenames.
     +    Provide an --exclude-modes-oids-stages/-l option for users to exclude
     +    the mode, oid, and stage and only get the list of conflicted filenames.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: discuss the first form.
     +@@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
       OPTIONS
       -------
       
     @@ Documentation/git-merge-tree.txt: plumbing commands since the possibility of mer
       Part of the linkgit:git[1] suite
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: struct merge_tree_options {
     - 	int real;
     - 	int trivial;
     +@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     + struct merge_tree_options {
     + 	int mode;
       	int show_messages;
     -+	int exclude_oids_and_modes;
     ++	int exclude_modes_oids_stages;
       };
       
       static int real_merge(struct merge_tree_options *o,
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			const char *name = conflicted_files.items[i].string;
      -			if (last && !strcmp(last, name))
      +			struct stage_info *c = conflicted_files.items[i].util;
     -+			if (!o->exclude_oids_and_modes)
     ++			if (!o->exclude_modes_oids_stages)
      +				printf("%06o %s %d\t",
      +				       c->mode, oid_to_hex(&c->oid), c->stage);
      +			else if (last && !strcmp(last, name))
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			write_name_quoted_relative(
       				name, prefix, stdout, line_termination);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			 N_("do a trivial merge only")),
     + 			    N_("do a trivial merge only"), 't'),
       		OPT_BOOL(0, "messages", &o.show_messages,
       			 N_("also show informational/conflict messages")),
     -+		OPT_BOOL_F(0, "exclude-oids-and-modes",
     -+			   &o.exclude_oids_and_modes,
     -+			   N_("list conflicted files without oids and modes"),
     ++		OPT_BOOL_F('l', "exclude-modes-oids-stages",
     ++			   &o.exclude_modes_oids_stages,
     ++			   N_("list conflicted files without modes/oids/stages"),
      +			   PARSE_OPT_NONEG),
       		OPT_END()
       	};
       
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Content merge and a few conflicts' '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Content merge and a few conflicts' '
       	expected_tree=$(cat .git/AUTO_MERGE) &&
       
       	# We will redo the merge, while we are still in a conflicted state!
     @@ t/t4301-merge-tree-real.sh: test_expect_success 'Content merge and a few conflic
       	test_when_finished "git reset --hard" &&
       
       	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Barf on too many arguments' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
       '
       
       test_expect_success 'test conflict notices and such' '
      -	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --exclude-oids-and-modes side1 side2 >out &&
     ++	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
       	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
       
       	# Expected results:
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'test conflict notices and such' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
       '
       
       test_expect_success 'Just the conflicted files without the messages' '
      -	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-oids-and-modes side1 side2 >out &&
     ++	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
       	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
       
       	test_write_lines HASH greeting whatever~side1 >expect &&
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Just the conflicted files without the messages' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Just the conflicted files without the messages' '
       	test_cmp expect actual
       '
       
 11:  ba8a50f03cb ! 12:  25677d5038c merge-tree: add a --allow-unrelated-histories flag
     @@ Documentation/git-merge-tree.txt: OPTIONS
      
       ## builtin/merge-tree.c ##
      @@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     + 
       struct merge_tree_options {
     - 	int real;
     - 	int trivial;
     + 	int mode;
      +	int allow_unrelated_histories;
       	int show_messages;
     - 	int exclude_oids_and_modes;
     + 	int exclude_modes_oids_stages;
       };
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	 * merge_incore_recursive in merge-ort.h
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	for (j = common; j; j = j->next)
       		commit_list_insert(j->item, &merge_bases);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			   &o.exclude_oids_and_modes,
     - 			   N_("list conflicted files without oids and modes"),
     + 			   &o.exclude_modes_oids_stages,
     + 			   N_("list conflicted files without modes/oids/stages"),
       			   PARSE_OPT_NONEG),
      +		OPT_BOOL_F(0, "allow-unrelated-histories",
      +			   &o.allow_unrelated_histories,
     @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char
       	};
       
      
     - ## t/t4301-merge-tree-real.sh ##
     -@@ t/t4301-merge-tree-real.sh: test_expect_success setup '
     + ## t/t4301-merge-tree-write-tree.sh ##
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
       	>whatever/empty &&
       	git add numbers greeting whatever/empty &&
       	test_tick &&
     @@ t/t4301-merge-tree-real.sh: test_expect_success setup '
       '
       
       test_expect_success 'Content merge and a few conflicts' '
     -@@ t/t4301-merge-tree-real.sh: test_expect_success 'Check conflicted oids and modes without messages' '
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and modes without messages' '
       	test_cmp conflicted-file-info actual
       '
       
 12:  4123209cafc = 13:  e7c63425a0e git-merge-tree.txt: add a section on potentional usage mistakes

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 01/13] merge-tree: rename merge_trees() to trivial_merge_trees()
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 02/13] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
                     ` (12 subsequent siblings)
  13 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 02/13] merge-tree: move logic for existing merge into new function
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 01/13] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 03/13] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
                     ` (11 subsequent siblings)
  13 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 03/13] merge-tree: add option parsing and initial shell for real merge function
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 01/13] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 02/13] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-02-02 21:30     ` Junio C Hamano
  2022-01-29 18:07   ` [PATCH v2 04/13] merge-tree: implement real merges Elijah Newren via GitGitGadget
                     ` (10 subsequent siblings)
  13 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 61 +++++++++++++++++++++++++++++++++++++-------
 git.c                |  2 +-
 2 files changed, 53 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..e98ec8a9f1d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,51 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    'w'),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), 't'),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	if (o.mode) {
+		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
+		if (argc != expected_remaining_argc)
+			usage_with_options(merge_tree_usage, mt_options);
+	} else {
+		if (argc < 2 || argc > 3)
+			usage_with_options(merge_tree_usage, mt_options);
+		o.mode = (argc == 2 ? 'w' : 't');
+	}
+
+	/* Do the relevant type of merge */
+	if (o.mode == 'w')
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 04/13] merge-tree: implement real merges
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (2 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 03/13] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-02-02 21:30     ` Junio C Hamano
  2022-01-29 18:07   ` [PATCH v2 05/13] diff: allow diff_warn_rename_limit to write somewhere besides stdout Johannes Schindelin via GitGitGadget
                     ` (9 subsequent siblings)
  13 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability.

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 71 +++++++++++++++++++++-----
 builtin/merge-tree.c             | 44 +++++++++++++++-
 t/t4301-merge-tree-write-tree.sh | 88 ++++++++++++++++++++++++++++++++
 3 files changed, 190 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..569485815a0 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,73 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The second form is deprecated and supported only for backward
+compatibility.  It will likely be removed in the future, and will not
+be discussed further in this manual.
+
+The first form will merge the two branches, doing a real merge.  A real
+merge is distinguished from a trivial merge in that it includes:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, it will create a new toplevel tree object.
+See `OUTPUT` below for details.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1.
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
+be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+However, it does not quite fit into the same category of low-level
+plumbing commands since the possibility of merge conflicts give it a
+much higher chance of the command not succeeding.
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index e98ec8a9f1d..d14c9f6e44e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -392,7 +395,46 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..66c3eaf2021
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,88 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(cat .git/AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	cat tmp | sed -e s/HEAD/side1/ >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 05/13] diff: allow diff_warn_rename_limit to write somewhere besides stdout
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (3 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 04/13] merge-tree: implement real merges Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Johannes Schindelin via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 06/13] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
                     ` (8 subsequent siblings)
  13 siblings, 0 replies; 215+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Johannes Schindelin

From: Johannes Schindelin <Johannes.Schindelin@gmx.de>

diff_warn_rename_limit() is hardcoded to write to stdout.  Make it
accept an output location parameter to make it more flexible.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 diff.c            | 20 ++++++++++++++------
 diff.h            |  3 ++-
 merge-ort.c       |  2 +-
 merge-recursive.c |  3 ++-
 4 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/diff.c b/diff.c
index 1bfb01c18ec..6952035046f 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,17 +6377,25 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out)
 {
-	fflush(stdout);
+	const char *fmt = NULL;
+
 	if (degraded_cc)
-		warning(_(degrade_cc_to_c_warning));
+		fmt = _(degrade_cc_to_c_warning);
 	else if (needed)
-		warning(_(rename_limit_warning));
+		fmt = _(rename_limit_warning);
 	else
 		return;
 	if (0 < needed)
-		warning(_(rename_limit_advice), varname, needed);
+		fmt = _(rename_limit_advice);
+
+	fflush(out);
+	if (out == stdout)
+		warning(fmt, varname, needed);
+	else
+		fprintf(out, fmt, varname, needed);
 }
 
 static void create_filepairs_for_header_only_notifications(struct diff_options *o)
@@ -6870,7 +6878,7 @@ int diff_result_code(struct diff_options *opt, int status)
 
 	diff_warn_rename_limit("diff.renameLimit",
 			       opt->needed_rename_limit,
-			       opt->degraded_cc_to_c);
+			       opt->degraded_cc_to_c, stdout);
 	if (!opt->flags.exit_with_status &&
 	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
 		return status;
diff --git a/diff.h b/diff.h
index ce9e2cf2e4f..40c5b78fb0a 100644
--- a/diff.h
+++ b/diff.h
@@ -597,7 +597,8 @@ void diffcore_fix_diff_index(void);
 int diff_queue_is_empty(struct diff_options *o);
 void diff_flush(struct diff_options*);
 void diff_free(struct diff_options*);
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out);
 
 /* diff-raw status letters */
 #define DIFF_STATUS_ADDED		'A'
diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..65618048b59 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4305,7 +4305,7 @@ void merge_switch_to_result(struct merge_options *opt,
 
 		/* Also include needed rename limit adjustment now */
 		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
+				       opti->renames.needed_limit, 0, stdout);
 
 		trace2_region_leave("merge", "display messages", opt->repo);
 	}
diff --git a/merge-recursive.c b/merge-recursive.c
index 9ec1e6d043a..d2eeca9fa20 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3738,7 +3738,8 @@ static void merge_finalize(struct merge_options *opt)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->priv->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0,
+				       stdout);
 	FREE_AND_NULL(opt->priv);
 }
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 06/13] merge-ort: split out a separate display_update_messages() function
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (4 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 05/13] diff: allow diff_warn_rename_limit to write somewhere besides stdout Johannes Schindelin via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 07/13] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
                     ` (7 subsequent siblings)
  13 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 77 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 36 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 65618048b59..1ada3198390 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0, stdout);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4273,42 +4312,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
 
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0, stdout);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 07/13] merge-ort: allow update messages to be written to different file stream
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (5 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 06/13] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 08/13] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
                     ` (6 subsequent siblings)
  13 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This modifies the new display_update_messages() function to allow
printing to somewhere other than stdout.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 9 +++++----
 merge-ort.h | 3 ++-
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 1ada3198390..d28d1721d14 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 }
 
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result)
+				   struct merge_result *result,
+				   FILE *stream)
 {
 	struct merge_options_internal *opti = result->priv;
 	struct hashmap_iter iter;
@@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
 	for (i = 0; i < olist.nr; ++i) {
 		struct strbuf *sb = olist.items[i].util;
 
-		printf("%s", sb->buf);
+		strbuf_write(sb, stream);
 	}
 	string_list_clear(&olist, 0);
 
 	/* Also include needed rename limit adjustment now */
 	diff_warn_rename_limit("merge.renamelimit",
-			       opti->renames.needed_limit, 0, stdout);
+			       opti->renames.needed_limit, 0, stream);
 
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
@@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
 	}
 
 	if (display_update_msgs)
-		merge_display_update_messages(opt, result);
+		merge_display_update_messages(opt, result, stdout);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..d643b47cb7c 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
  * so only call this when bypassing merge_switch_to_result().
  */
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result);
+				   struct merge_result *result,
+				   FILE *stream);
 
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 08/13] merge-tree: support including merge messages in output
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (6 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 07/13] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-02-02 21:30     ` Junio C Hamano
  2022-01-29 18:07   ` [PATCH v2 09/13] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
                     ` (5 subsequent siblings)
  13 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 19 ++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 21 +++++++++++++++
 3 files changed, 76 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 569485815a0..42e0f8f6183 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
@@ -35,17 +35,47 @@ merge is distinguished from a trivial merge in that it includes:
 After the merge completes, it will create a new toplevel tree object.
 See `OUTPUT` below for details.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
+
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -69,7 +99,8 @@ be used as a part of a series of steps such as
 
 However, it does not quite fit into the same category of low-level
 plumbing commands since the possibility of merge conflicts give it a
-much higher chance of the command not succeeding.
+much higher chance of the command not succeeding (and NEWTREE containing
+a bunch of stuff other than just a toplevel tree).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d14c9f6e44e..6a556ab1c9c 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -390,6 +390,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -432,18 +433,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result, stdout);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -453,10 +463,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    'w'),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), 't'),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	if (o.mode) {
@@ -468,6 +481,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			usage_with_options(merge_tree_usage, mt_options);
 		o.mode = (argc == 2 ? 'w' : 't');
 	}
+	if (o.mode == 't' && original_argc < argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 66c3eaf2021..e2255711f9c 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -85,4 +85,25 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 09/13] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (7 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 08/13] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-01-29 18:07   ` [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
                     ` (4 subsequent siblings)
  13 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index d28d1721d14..c4ce6027dc4 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index d643b47cb7c..e635a294ea8 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result,
 				   FILE *stream);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (8 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 09/13] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-02-02 21:32     ` Junio C Hamano
                       ` (2 more replies)
  2022-01-29 18:07   ` [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
                     ` (3 subsequent siblings)
  13 siblings, 3 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 42e0f8f6183..160e8f44b62 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -55,6 +55,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -66,6 +67,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 6a556ab1c9c..54dae018203 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -394,7 +397,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -438,6 +442,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result, stdout);
@@ -486,7 +506,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index e2255711f9c..7113d060bc5 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -95,6 +95,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -106,4 +108,13 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (9 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-02-02 21:32     ` Junio C Hamano
  2022-01-29 18:07   ` [PATCH v2 12/13] merge-tree: add a --allow-unrelated-histories flag Elijah Newren via GitGitGadget
                     ` (2 subsequent siblings)
  13 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide an --exclude-modes-oids-stages/-l option for users to exclude
the mode, oid, and stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 30 ++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++++--
 3 files changed, 58 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 160e8f44b62..55bb7bc61c1 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -38,6 +38,11 @@ See `OUTPUT` below for details.
 OPTIONS
 -------
 
+--exclude-oids-and-modes::
+	Instead of writing a list of (mode, oid, stage, path) tuples
+	to output for conflicted files, just provide a list of
+	filenames with conflicts.
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -55,7 +60,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -67,18 +72,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--exclude-oids-and-modes` option is passed, the mode, object, and
+stage will be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -110,6 +120,14 @@ plumbing commands since the possibility of merge conflicts give it a
 much higher chance of the command not succeeding (and NEWTREE containing
 a bunch of stuff other than just a toplevel tree).
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 54dae018203..dc52cd02dce 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -394,6 +394,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int exclude_modes_oids_stages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->exclude_modes_oids_stages)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), 't'),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F('l', "exclude-modes-oids-stages",
+			   &o.exclude_modes_oids_stages,
+			   N_("list conflicted files without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 7113d060bc5..1572f460da0 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -47,6 +47,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(cat .git/AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -86,7 +87,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -109,7 +110,7 @@ test_expect_success 'test conflict notices and such' '
 '
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -117,4 +118,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 12/13] merge-tree: add a --allow-unrelated-histories flag
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (10 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-02-02 21:32     ` Junio C Hamano
  2022-01-29 18:07   ` [PATCH v2 13/13] git-merge-tree.txt: add a section on potentional usage mistakes Elijah Newren via GitGitGadget
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  13 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 55bb7bc61c1..d35710c81d5 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -49,6 +49,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index dc52cd02dce..cca5075d521 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -393,6 +393,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int exclude_modes_oids_stages;
 };
@@ -430,7 +431,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -494,6 +495,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.exclude_modes_oids_stages,
 			   N_("list conflicted files without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 1572f460da0..996bdfaab7d 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -38,7 +38,13 @@ test_expect_success setup '
 	>whatever/empty &&
 	git add numbers greeting whatever/empty &&
 	test_tick &&
-	git commit -m other-modifications
+	git commit -m other-modifications &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Content merge and a few conflicts' '
@@ -139,4 +145,20 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v2 13/13] git-merge-tree.txt: add a section on potentional usage mistakes
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (11 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 12/13] merge-tree: add a --allow-unrelated-histories flag Elijah Newren via GitGitGadget
@ 2022-01-29 18:07   ` Elijah Newren via GitGitGadget
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  13 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-01-29 18:07 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index d35710c81d5..b97ddc58a7a 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -133,6 +133,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 00/12] RFC: In-core git merge-tree ("Server side merges")
  2022-01-29 17:43       ` Elijah Newren
@ 2022-01-31 17:45         ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-01-31 17:45 UTC (permalink / raw)
  To: Christian Couder
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, René Scharfe

On Sat, Jan 29, 2022 at 9:43 AM Elijah Newren <newren@gmail.com> wrote:
>
> Hi Christian,
>
> On Sat, Jan 29, 2022 at 12:18 AM Christian Couder
> <christian.couder@gmail.com> wrote:
> >
> > On Sat, Jan 29, 2022 at 8:04 AM Elijah Newren <newren@gmail.com> wrote:
> > >
> > > On Wed, Jan 26, 2022 at 12:48 AM Christian Couder
> > > <christian.couder@gmail.com> wrote:
[...]
> > > > The reason is
> > > > that I think in many cases when there are conflicts, the conflicts
> > > > will be small and the user will want to see them.
> > >
> > > I'm a little worried about the assumption here that conflict size is
> > > measurable and visible via diffs.  That might be true in some cases,
> > > but a UI written with that assumption is going to be very confusing
> > > when hitting cases where that assumption does not hold.  For example:
> > >
> > >   * What if there is a binary file conflict, or a modify/delete or
> > > rename/delete conflict, or failed-to-merge submodule conflict, or a
> > > file location conflict? (For these, there is no diff relative to the
> > > first parent and hence this conflict would have no diff output for
> > > it)?
> > >   * What if there was a simple file/directory conflict?  A diff would
> > > show a rename (even when neither side did any renames), but not any
> > > conflict markers.
> > >   * What if there was a rename/rename conflict (both sides renamed
> > > same file differently) or a distinct types conflict?  The former
> > > results in three different conflicting files, none of them with
> > > conflict markers, while the latter results in two different
> > > conflicting files both without conflict markers?  Showing individual
> > > per-file diffs totally loses all context here -- it'll show no-diff
> > > for one of the files, and totally new additions for the ones.
> >
> > In those cases we just tell users that they cannot resolve those
> > conflicts in the user interface, see the following doc:
> >
> > https://docs.gitlab.com/ee/user/project/merge_requests/conflicts.html#conflicts-you-can-resolve-in-the-user-interface
>
> So...I think you may have just convinced me that my fears were
> justified and that I should probably NAK any attempt to add diffs to
> the merge-tree command.  I won't jump to conclusions but you've
> provided some pretty strong signal to me against going down that
> route.  The list of limitations in the link you provide do mostly
> avoid the broken cases I listed above, but it enshrines those
> limitations on that webpage as fundamental rather than just as current
> implementation shortcomings.  You may not be able to remove those
> limitations on that webpage without either expunging the diffs from
> the UI or exposing the brokenness of the various cases above.
>
> If you do propose a diff option in the future, come prepared to
> discuss how you'll avoid accidentally leading others down into paths
> with the same fundamental issues, and/or how the above types of
> conflicts might still be meaningfully handled.

Actually, after having a few extra days to think about it, I thought
of something that should have been obvious to me, given my other
in-flight series that this depends upon...

If you used the same trick that remerge-diff does to include the
CONFLICT (and related messages) headers in the diff, then the kinds of
conflicts that are normally either invisible or misleading/confusing
to show via a diff would suddenly have the extra notices needed to
explain them, and make this problem tractable.

Further, it'd only make sense to do the special diff as part of the
merge-tree process, since it has the conflict messages strmap needed
to do this.

And there's not all that much work that would be needed to take
advantage of this, especially since this series already depends upon
the remerge-diff series.

So, maybe this is fine after all.

> Also, the list of limitations you have may not be quite comprehensive
> enough to avoid all problems (though it certainly avoids most of
> them).  Can I ask a couple clarifying questions about your list of
> limitations in that link? :
>
>   * When that page says the file cannot already contain conflict
> markers, is the check performed on the version of the file in the two
> trees being merged, or is the check performed on the 2nd and 3rd index
> stage of the merge result (these are not equivalent checks, even if
> they often give the same answer)?
>   * When that page says the file must already exist in the same path
> on both branches, is the check performed on by checking the path in
> the two trees being merged, or is the check performed on the 2nd and
> 3rd index stage of the merge result (again, these are not equivalent
> checks)?

I am still curious about this either way.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 00/15] In-core git merge-tree ("Server side merges")
  2022-01-29 18:07 ` [PATCH v2 00/13] " Elijah Newren via GitGitGadget
                     ` (12 preceding siblings ...)
  2022-01-29 18:07   ` [PATCH v2 13/13] git-merge-tree.txt: add a section on potentional usage mistakes Elijah Newren via GitGitGadget
@ 2022-02-02  7:34   ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 01/15] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
                       ` (15 more replies)
  13 siblings, 16 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren

Note: Depends on en/remerge-diff (to avoid a small textual conflict). Still
under heavy discussion, though, so no need to pick up yet.

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs.

== Quick Overview ==

 * Patches 1-2: preparatory cleanups
 * Patches 3-4: implement basic real merges
 * Patches 5-9: include informational messages ("CONFLICT" messages and
   such) in output
   * to be honest, patches 5, 6, & 8 may be less relevant since we're now
     including these messages on stdout anyway...
 * Patches 10-13: add ability to include ls-files -u style of info in the
   output
 * Patch 14: support --allow-unrelated-histories
 * Patch 15: augment the manual with potential usage mistakes

== Updates Log ==

Updates since v2 (or v4, if you include the rounds at
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):

 * Improved patches from Dscho for the diff_warn_rename_limit() handling
 * Add a -z option for NUL-terminated conflict info lines (so that filenames
   do not have to be quoted)

Stuff NOT included that reviewers brought up in earlier rounds:

 * Very generic (mode, oid, stage, filename) printing formatting[1]
 * Always printing 3 stages for each filename with conflicts[2] [1]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [2]
   https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/

Updates since v1 (or v3 depending on how you count; thanks to René, Ævar,
Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

Updates since original submission v2 (thanks to Christian, Dscho, Ramsay,
and René for suggestions and comments):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since original submission v1 (thanks to Johannes Altmanninger and
Fabian for suggestions):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (13):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-ort: allow update messages to be written to different file
    stream
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

Johannes Schindelin (2):
  Introduce a variant of the `warning()` function that takes a `FILE *`
  diff: allow diff_warn_rename_limit to write somewhere besides stderr

 Documentation/git-merge-tree.txt | 192 +++++++++++++++++++++++++++--
 builtin/merge-tree.c             | 164 +++++++++++++++++++++++--
 diff.c                           |  13 +-
 diff.h                           |   3 +-
 git-compat-util.h                |   1 +
 git.c                            |   2 +-
 merge-ort.c                      | 109 +++++++++++------
 merge-ort.h                      |  30 +++++
 merge-recursive.c                |   3 +-
 t/t4301-merge-tree-write-tree.sh | 204 +++++++++++++++++++++++++++++++
 usage.c                          |  14 +++
 11 files changed, 667 insertions(+), 68 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v2:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  63f42df21ae =  3:  63f42df21ae merge-tree: add option parsing and initial shell for real merge function
  4:  02c29f920d0 =  4:  02c29f920d0 merge-tree: implement real merges
  -:  ----------- >  5:  290b42846b5 Introduce a variant of the `warning()` function that takes a `FILE *`
  5:  6fb4f4580a5 !  6:  2083fbe9b2e diff: allow diff_warn_rename_limit to write somewhere besides stdout
     @@ Metadata
      Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
      
       ## Commit message ##
     -    diff: allow diff_warn_rename_limit to write somewhere besides stdout
     +    diff: allow diff_warn_rename_limit to write somewhere besides stderr
      
     -    diff_warn_rename_limit() is hardcoded to write to stdout.  Make it
     -    accept an output location parameter to make it more flexible.
     +    diff_warn_rename_limit() is hardcoded to write to stderr.  Make it
     +    accept a file stream parameter to make it more flexible.
      
          Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
          Signed-off-by: Elijah Newren <newren@gmail.com>
     @@ diff.c: static const char rename_limit_advice[] =
      +void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
      +			    FILE *out)
       {
     --	fflush(stdout);
     -+	const char *fmt = NULL;
     -+
     + 	fflush(stdout);
       	if (degraded_cc)
      -		warning(_(degrade_cc_to_c_warning));
     -+		fmt = _(degrade_cc_to_c_warning);
     ++		warning_fp(out, _(degrade_cc_to_c_warning));
       	else if (needed)
      -		warning(_(rename_limit_warning));
     -+		fmt = _(rename_limit_warning);
     ++		warning_fp(out, _(rename_limit_warning));
       	else
       		return;
     ++
     ++
       	if (0 < needed)
      -		warning(_(rename_limit_advice), varname, needed);
     -+		fmt = _(rename_limit_advice);
     -+
     -+	fflush(out);
     -+	if (out == stdout)
     -+		warning(fmt, varname, needed);
     -+	else
     -+		fprintf(out, fmt, varname, needed);
     ++		warning_fp(out, _(rename_limit_advice), varname, needed);
       }
       
       static void create_filepairs_for_header_only_notifications(struct diff_options *o)
     @@ diff.c: int diff_result_code(struct diff_options *opt, int status)
       	diff_warn_rename_limit("diff.renameLimit",
       			       opt->needed_rename_limit,
      -			       opt->degraded_cc_to_c);
     -+			       opt->degraded_cc_to_c, stdout);
     ++			       opt->degraded_cc_to_c, stderr);
       	if (!opt->flags.exit_with_status &&
       	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
       		return status;
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
       		/* Also include needed rename limit adjustment now */
       		diff_warn_rename_limit("merge.renamelimit",
      -				       opti->renames.needed_limit, 0);
     -+				       opti->renames.needed_limit, 0, stdout);
     ++				       opti->renames.needed_limit, 0, stderr);
       
       		trace2_region_leave("merge", "display messages", opt->repo);
       	}
     @@ merge-recursive.c: static void merge_finalize(struct merge_options *opt)
       		diff_warn_rename_limit("merge.renamelimit",
      -				       opt->priv->needed_rename_limit, 0);
      +				       opt->priv->needed_rename_limit, 0,
     -+				       stdout);
     ++				       stderr);
       	FREE_AND_NULL(opt->priv);
       }
       
  6:  28368c03898 !  7:  1be858e6aa6 merge-ort: split out a separate display_update_messages() function
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
      +
      +	/* Also include needed rename limit adjustment now */
      +	diff_warn_rename_limit("merge.renamelimit",
     -+			       opti->renames.needed_limit, 0, stdout);
     ++			       opti->renames.needed_limit, 0, stderr);
      +
      +	trace2_region_leave("merge", "display messages", opt->repo);
      +}
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
      -
      -		/* Also include needed rename limit adjustment now */
      -		diff_warn_rename_limit("merge.renamelimit",
     --				       opti->renames.needed_limit, 0, stdout);
     +-				       opti->renames.needed_limit, 0, stderr);
      -
      -		trace2_region_leave("merge", "display messages", opt->repo);
      -	}
  7:  593d0c00b57 !  8:  04c3bdc44d2 merge-ort: allow update messages to be written to different file stream
     @@ Commit message
          merge-ort: allow update messages to be written to different file stream
      
          This modifies the new display_update_messages() function to allow
     -    printing to somewhere other than stdout.
     +    printing to somewhere other than stdout.  It also consolidates the
     +    location of the diff_warn_rename_limit() message with the rest of the
     +    CONFLICT and other update messages to all go to the same stream.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
     @@ merge-ort.c: void merge_display_update_messages(struct merge_options *opt,
       
       	/* Also include needed rename limit adjustment now */
       	diff_warn_rename_limit("merge.renamelimit",
     --			       opti->renames.needed_limit, 0, stdout);
     +-			       opti->renames.needed_limit, 0, stderr);
      +			       opti->renames.needed_limit, 0, stream);
       
       	trace2_region_leave("merge", "display messages", opt->repo);
  8:  d0d30e92ecd =  9:  c8ed002408d merge-tree: support including merge messages in output
  9:  9c2334ae9f2 = 10:  1c2a3f5ef63 merge-ort: provide a merge_get_conflicted_files() helper function
 10:  243134dc247 = 11:  9c2389eef0e merge-tree: provide a list of which files have conflicts
 11:  c322e4c6938 = 12:  2188a8ca1e7 merge-tree: provide easy access to `ls-files -u` style info
  -:  ----------- > 13:  52339b396fa merge-tree: allow `ls-files -u` style info to be NUL terminated
 12:  25677d5038c ! 14:  c854ecb5f4a merge-tree: add a --allow-unrelated-histories flag
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
       '
       
       test_expect_success 'Content merge and a few conflicts' '
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and modes without messages' '
     - 	test_cmp conflicted-file-info actual
     +@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'NUL terminated conflicted file "lines"' '
     + 	test_cmp expect actual
       '
       
      +test_expect_success 'error out by default for unrelated histories' '
 13:  e7c63425a0e = 15:  bc8591bbb63 git-merge-tree.txt: add a section on potentional usage mistakes

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 01/15] merge-tree: rename merge_trees() to trivial_merge_trees()
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 02/15] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
                       ` (14 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 02/15] merge-tree: move logic for existing merge into new function
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 01/15] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
                       ` (13 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 01/15] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 02/15] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-03  2:05       ` Ævar Arnfjörð Bjarmason
  2022-02-07 22:41       ` Emily Shaffer
  2022-02-02  7:34     ` [PATCH v3 04/15] merge-tree: implement real merges Elijah Newren via GitGitGadget
                       ` (12 subsequent siblings)
  15 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 61 +++++++++++++++++++++++++++++++++++++-------
 git.c                |  2 +-
 2 files changed, 53 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..e98ec8a9f1d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,51 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    'w'),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), 't'),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	if (o.mode) {
+		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
+		if (argc != expected_remaining_argc)
+			usage_with_options(merge_tree_usage, mt_options);
+	} else {
+		if (argc < 2 || argc > 3)
+			usage_with_options(merge_tree_usage, mt_options);
+		o.mode = (argc == 2 ? 'w' : 't');
+	}
+
+	/* Do the relevant type of merge */
+	if (o.mode == 'w')
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (2 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02 21:22       ` Junio C Hamano
  2022-02-04  4:48       ` Josh Steadmon
  2022-02-02  7:34     ` [PATCH v3 05/15] Introduce a variant of the `warning()` function that takes a `FILE *` Johannes Schindelin via GitGitGadget
                       ` (11 subsequent siblings)
  15 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability.

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 71 +++++++++++++++++++++-----
 builtin/merge-tree.c             | 44 +++++++++++++++-
 t/t4301-merge-tree-write-tree.sh | 88 ++++++++++++++++++++++++++++++++
 3 files changed, 190 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..569485815a0 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,73 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The second form is deprecated and supported only for backward
+compatibility.  It will likely be removed in the future, and will not
+be discussed further in this manual.
+
+The first form will merge the two branches, doing a real merge.  A real
+merge is distinguished from a trivial merge in that it includes:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, it will create a new toplevel tree object.
+See `OUTPUT` below for details.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1.
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
+be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
+       git update-ref $BRANCH1 $NEWCOMMIT
+
+However, it does not quite fit into the same category of low-level
+plumbing commands since the possibility of merge conflicts give it a
+much higher chance of the command not succeeding.
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index e98ec8a9f1d..d14c9f6e44e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -2,6 +2,9 @@
 #include "builtin.h"
 #include "tree-walk.h"
 #include "xdiff-interface.h"
+#include "help.h"
+#include "commit-reach.h"
+#include "merge-ort.h"
 #include "object-store.h"
 #include "parse-options.h"
 #include "repository.h"
@@ -392,7 +395,46 @@ struct merge_tree_options {
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2)
 {
-	die(_("real merges are not yet implemented"));
+	struct commit *parent1, *parent2;
+	struct commit_list *common;
+	struct commit_list *merge_bases = NULL;
+	struct commit_list *j;
+	struct merge_options opt;
+	struct merge_result result = { 0 };
+
+	parent1 = get_merge_parent(branch1);
+	if (!parent1)
+		help_unknown_ref(branch1, "merge-tree",
+				 _("not something we can merge"));
+
+	parent2 = get_merge_parent(branch2);
+	if (!parent2)
+		help_unknown_ref(branch2, "merge-tree",
+				 _("not something we can merge"));
+
+	init_merge_options(&opt, the_repository);
+
+	opt.show_rename_progress = 0;
+
+	opt.branch1 = branch1;
+	opt.branch2 = branch2;
+
+	/*
+	 * Get the merge bases, in reverse order; see comment above
+	 * merge_incore_recursive in merge-ort.h
+	 */
+	common = get_merge_bases(parent1, parent2);
+	if (!common)
+		die(_("refusing to merge unrelated histories"));
+	for (j = common; j; j = j->next)
+		commit_list_insert(j->item, &merge_bases);
+
+	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
+	if (result.clean < 0)
+		die(_("failure to merge"));
+	puts(oid_to_hex(&result.tree->object.oid));
+	merge_finalize(&opt, &result);
+	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
new file mode 100755
index 00000000000..66c3eaf2021
--- /dev/null
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -0,0 +1,88 @@
+#!/bin/sh
+
+test_description='git merge-tree --write-tree'
+
+. ./test-lib.sh
+
+# This test is ort-specific
+if test "${GIT_TEST_MERGE_ALGORITHM}" != "ort"
+then
+	skip_all="GIT_TEST_MERGE_ALGORITHM != ort"
+	test_done
+fi
+
+test_expect_success setup '
+	test_write_lines 1 2 3 4 5 >numbers &&
+	echo hello >greeting &&
+	echo foo >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m initial &&
+
+	git branch side1 &&
+	git branch side2 &&
+
+	git checkout side1 &&
+	test_write_lines 1 2 3 4 5 6 >numbers &&
+	echo hi >greeting &&
+	echo bar >whatever &&
+	git add numbers greeting whatever &&
+	test_tick &&
+	git commit -m modify-stuff &&
+
+	git checkout side2 &&
+	test_write_lines 0 1 2 3 4 5 >numbers &&
+	echo yo >greeting &&
+	git rm whatever &&
+	mkdir whatever &&
+	>whatever/empty &&
+	git add numbers greeting whatever/empty &&
+	test_tick &&
+	git commit -m other-modifications
+'
+
+test_expect_success 'Content merge and a few conflicts' '
+	git checkout side1^0 &&
+	test_must_fail git merge side2 &&
+	expected_tree=$(cat .git/AUTO_MERGE) &&
+
+	# We will redo the merge, while we are still in a conflicted state!
+	test_when_finished "git reset --hard" &&
+
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
+	actual_tree=$(head -n 1 RESULT) &&
+
+	# Due to differences of e.g. "HEAD" vs "side1", the results will not
+	# exactly match.  Dig into individual files.
+
+	# Numbers should have three-way merged cleanly
+	test_write_lines 0 1 2 3 4 5 6 >expect &&
+	git show ${actual_tree}:numbers >actual &&
+	test_cmp expect actual &&
+
+	# whatever and whatever~<branch> should have same HASHES
+	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
+	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
+	test_cmp expect actual &&
+
+	# greeting should have a merge conflict
+	git show ${expected_tree}:greeting >tmp &&
+	cat tmp | sed -e s/HEAD/side1/ >expect &&
+	git show ${actual_tree}:greeting >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
+	# Mis-spell with single "s" instead of double "s"
+	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
+
+	grep "error: unknown option.*mesages" expect
+'
+
+test_expect_success 'Barf on too many arguments' '
+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
+
+	grep "^usage: git merge-tree" expect
+'
+
+test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 05/15] Introduce a variant of the `warning()` function that takes a `FILE *`
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (3 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 04/15] merge-tree: implement real merges Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Johannes Schindelin via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 06/15] diff: allow diff_warn_rename_limit to write somewhere besides stderr Johannes Schindelin via GitGitGadget
                       ` (10 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Johannes Schindelin

From: Johannes Schindelin <johannes.schindelin@gmx.de>

We are about to teach `diff_warn_rename_limit()` to write into a file
instead of `stderr`. That function wants to call `warning()` when
writing to `stderr`, though, allowing for the `warn_routine` to be
overridden.

Let's introduce a helper for that.

Note: Since there is currently no need to provide similar functions for
`error()` or `die()`, let alone for the `_errno` variants, we will leave
that to a date when the need for those should arise, if ever.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 git-compat-util.h |  1 +
 usage.c           | 14 ++++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/git-compat-util.h b/git-compat-util.h
index d70ce142861..64ba60e5c71 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,6 +475,7 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
+void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index c7d233b0de9..0bfd2c603c0 100644
--- a/usage.c
+++ b/usage.c
@@ -253,6 +253,20 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
+void warning_fp(FILE *out, const char *warn, ...)
+{
+	va_list params;
+
+	va_start(params, warn);
+	if (out == stderr)
+		warn_routine(warn, params);
+	else {
+		vfprintf(out, warn, params);
+		fputc('\n', out);
+	}
+	va_end(params);
+}
+
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 06/15] diff: allow diff_warn_rename_limit to write somewhere besides stderr
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (4 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 05/15] Introduce a variant of the `warning()` function that takes a `FILE *` Johannes Schindelin via GitGitGadget
@ 2022-02-02  7:34     ` Johannes Schindelin via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 07/15] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
                       ` (9 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Johannes Schindelin via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Johannes Schindelin

From: Johannes Schindelin <Johannes.Schindelin@gmx.de>

diff_warn_rename_limit() is hardcoded to write to stderr.  Make it
accept a file stream parameter to make it more flexible.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 diff.c            | 13 ++++++++-----
 diff.h            |  3 ++-
 merge-ort.c       |  2 +-
 merge-recursive.c |  3 ++-
 4 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/diff.c b/diff.c
index 1bfb01c18ec..28368110147 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,17 +6377,20 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc)
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out)
 {
 	fflush(stdout);
 	if (degraded_cc)
-		warning(_(degrade_cc_to_c_warning));
+		warning_fp(out, _(degrade_cc_to_c_warning));
 	else if (needed)
-		warning(_(rename_limit_warning));
+		warning_fp(out, _(rename_limit_warning));
 	else
 		return;
+
+
 	if (0 < needed)
-		warning(_(rename_limit_advice), varname, needed);
+		warning_fp(out, _(rename_limit_advice), varname, needed);
 }
 
 static void create_filepairs_for_header_only_notifications(struct diff_options *o)
@@ -6870,7 +6873,7 @@ int diff_result_code(struct diff_options *opt, int status)
 
 	diff_warn_rename_limit("diff.renameLimit",
 			       opt->needed_rename_limit,
-			       opt->degraded_cc_to_c);
+			       opt->degraded_cc_to_c, stderr);
 	if (!opt->flags.exit_with_status &&
 	    !(opt->output_format & DIFF_FORMAT_CHECKDIFF))
 		return status;
diff --git a/diff.h b/diff.h
index ce9e2cf2e4f..40c5b78fb0a 100644
--- a/diff.h
+++ b/diff.h
@@ -597,7 +597,8 @@ void diffcore_fix_diff_index(void);
 int diff_queue_is_empty(struct diff_options *o);
 void diff_flush(struct diff_options*);
 void diff_free(struct diff_options*);
-void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc);
+void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
+			    FILE *out);
 
 /* diff-raw status letters */
 #define DIFF_STATUS_ADDED		'A'
diff --git a/merge-ort.c b/merge-ort.c
index 9bf15a01db8..46e72b62880 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4305,7 +4305,7 @@ void merge_switch_to_result(struct merge_options *opt,
 
 		/* Also include needed rename limit adjustment now */
 		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0);
+				       opti->renames.needed_limit, 0, stderr);
 
 		trace2_region_leave("merge", "display messages", opt->repo);
 	}
diff --git a/merge-recursive.c b/merge-recursive.c
index 9ec1e6d043a..01ca82773cc 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -3738,7 +3738,8 @@ static void merge_finalize(struct merge_options *opt)
 		strbuf_release(&opt->obuf);
 	if (show(opt, 2))
 		diff_warn_rename_limit("merge.renamelimit",
-				       opt->priv->needed_rename_limit, 0);
+				       opt->priv->needed_rename_limit, 0,
+				       stderr);
 	FREE_AND_NULL(opt->priv);
 }
 
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 07/15] merge-ort: split out a separate display_update_messages() function
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (5 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 06/15] diff: allow diff_warn_rename_limit to write somewhere besides stderr Johannes Schindelin via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
                       ` (8 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This patch includes no new code; it simply moves a bunch of lines into a
new function.  As such, there are no functional changes.  This is just a
preparatory step to allow the printed messages to be handled differently
by other callers, such as in `git merge-tree --write-tree`.

(Patch best viewed with
     --color-moved --color-moved-ws=allow-indentation-change
 to see that it is a simple code movement.)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 77 ++++++++++++++++++++++++++++-------------------------
 merge-ort.h |  8 ++++++
 2 files changed, 49 insertions(+), 36 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 46e72b62880..82d2faf5bf9 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4235,6 +4235,45 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 	return errs;
 }
 
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result)
+{
+	struct merge_options_internal *opti = result->priv;
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct string_list olist = STRING_LIST_INIT_NODUP;
+	int i;
+
+	if (opt->record_conflict_msgs_as_headers)
+		BUG("Either display conflict messages or record them as headers, not both");
+
+	trace2_region_enter("merge", "display messages", opt->repo);
+
+	/* Hack to pre-allocate olist to the desired size */
+	ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
+		   olist.alloc);
+
+	/* Put every entry from output into olist, then sort */
+	strmap_for_each_entry(&opti->output, &iter, e) {
+		string_list_append(&olist, e->key)->util = e->value;
+	}
+	string_list_sort(&olist);
+
+	/* Iterate over the items, printing them */
+	for (i = 0; i < olist.nr; ++i) {
+		struct strbuf *sb = olist.items[i].util;
+
+		printf("%s", sb->buf);
+	}
+	string_list_clear(&olist, 0);
+
+	/* Also include needed rename limit adjustment now */
+	diff_warn_rename_limit("merge.renamelimit",
+			       opti->renames.needed_limit, 0, stderr);
+
+	trace2_region_leave("merge", "display messages", opt->repo);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
@@ -4273,42 +4312,8 @@ void merge_switch_to_result(struct merge_options *opt,
 		trace2_region_leave("merge", "write_auto_merge", opt->repo);
 	}
 
-	if (display_update_msgs) {
-		struct merge_options_internal *opti = result->priv;
-		struct hashmap_iter iter;
-		struct strmap_entry *e;
-		struct string_list olist = STRING_LIST_INIT_NODUP;
-		int i;
-
-		if (opt->record_conflict_msgs_as_headers)
-			BUG("Either display conflict messages or record them as headers, not both");
-
-		trace2_region_enter("merge", "display messages", opt->repo);
-
-		/* Hack to pre-allocate olist to the desired size */
-		ALLOC_GROW(olist.items, strmap_get_size(&opti->output),
-			   olist.alloc);
-
-		/* Put every entry from output into olist, then sort */
-		strmap_for_each_entry(&opti->output, &iter, e) {
-			string_list_append(&olist, e->key)->util = e->value;
-		}
-		string_list_sort(&olist);
-
-		/* Iterate over the items, printing them */
-		for (i = 0; i < olist.nr; ++i) {
-			struct strbuf *sb = olist.items[i].util;
-
-			printf("%s", sb->buf);
-		}
-		string_list_clear(&olist, 0);
-
-		/* Also include needed rename limit adjustment now */
-		diff_warn_rename_limit("merge.renamelimit",
-				       opti->renames.needed_limit, 0, stderr);
-
-		trace2_region_leave("merge", "display messages", opt->repo);
-	}
+	if (display_update_msgs)
+		merge_display_update_messages(opt, result);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index fe599b87868..e5aec45b18f 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -80,6 +80,14 @@ void merge_switch_to_result(struct merge_options *opt,
 			    int update_worktree_and_index,
 			    int display_update_msgs);
 
+/*
+ * Display messages about conflicts and which files were 3-way merged.
+ * Automatically called by merge_switch_to_result() with stream == stdout,
+ * so only call this when bypassing merge_switch_to_result().
+ */
+void merge_display_update_messages(struct merge_options *opt,
+				   struct merge_result *result);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (6 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 07/15] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-03  1:48       ` Ævar Arnfjörð Bjarmason
  2022-02-02  7:34     ` [PATCH v3 09/15] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
                       ` (7 subsequent siblings)
  15 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This modifies the new display_update_messages() function to allow
printing to somewhere other than stdout.  It also consolidates the
location of the diff_warn_rename_limit() message with the rest of the
CONFLICT and other update messages to all go to the same stream.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 9 +++++----
 merge-ort.h | 3 ++-
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/merge-ort.c b/merge-ort.c
index 82d2faf5bf9..d28d1721d14 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
 }
 
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result)
+				   struct merge_result *result,
+				   FILE *stream)
 {
 	struct merge_options_internal *opti = result->priv;
 	struct hashmap_iter iter;
@@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
 	for (i = 0; i < olist.nr; ++i) {
 		struct strbuf *sb = olist.items[i].util;
 
-		printf("%s", sb->buf);
+		strbuf_write(sb, stream);
 	}
 	string_list_clear(&olist, 0);
 
 	/* Also include needed rename limit adjustment now */
 	diff_warn_rename_limit("merge.renamelimit",
-			       opti->renames.needed_limit, 0, stderr);
+			       opti->renames.needed_limit, 0, stream);
 
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
@@ -4313,7 +4314,7 @@ void merge_switch_to_result(struct merge_options *opt,
 	}
 
 	if (display_update_msgs)
-		merge_display_update_messages(opt, result);
+		merge_display_update_messages(opt, result, stdout);
 
 	merge_finalize(opt, result);
 }
diff --git a/merge-ort.h b/merge-ort.h
index e5aec45b18f..d643b47cb7c 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -86,7 +86,8 @@ void merge_switch_to_result(struct merge_options *opt,
  * so only call this when bypassing merge_switch_to_result().
  */
 void merge_display_update_messages(struct merge_options *opt,
-				   struct merge_result *result);
+				   struct merge_result *result,
+				   FILE *stream);
 
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 09/15] merge-tree: support including merge messages in output
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (7 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 10/15] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
                       ` (6 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

When running `git merge-tree --write-tree`, we previously would only
return an exit status reflecting the cleanness of a merge, and print out
the toplevel tree of the resulting merge.  Merges also have
informational messages, such as:
  * "Auto-merging <PATH>"
  * "CONFLICT (content): ..."
  * "CONFLICT (file/directory)"
  * etc.
In fact, when non-content conflicts occur (such as file/directory,
modify/delete, add/add with differing modes, rename/rename (1to2),
etc.), these informational messages may be the only notification the
user gets since these conflicts are not representable in the contents
of the file.

Add a --[no-]messages option so that callers can request these messages
be included at the end of the output.  Include such messages by default
when there are conflicts, and omit them by default when the merge is
clean.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 45 +++++++++++++++++++++++++++-----
 builtin/merge-tree.c             | 19 ++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 21 +++++++++++++++
 3 files changed, 76 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 569485815a0..42e0f8f6183 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -9,7 +9,7 @@ git-merge-tree - Perform merge without touching index or working tree
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--write-tree] [<options>] <branch1> <branch2>
 'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
@@ -35,17 +35,47 @@ merge is distinguished from a trivial merge in that it includes:
 After the merge completes, it will create a new toplevel tree object.
 See `OUTPUT` below for details.
 
+OPTIONS
+-------
+
+--[no-]messages::
+	Write any informational messages such as "Auto-merging <path>"
+	or CONFLICT notices to the end of stdout.  If unspecified, the
+	default is to include these messages if there are merge
+	conflicts, and to omit them otherwise.
+
 OUTPUT
 ------
 
-For either a successful or conflicted merge, the output from
-git-merge-tree is simply one line:
+By default, for a successful merge, the output from git-merge-tree is
+simply one line:
+
+	<OID of toplevel tree>
+
+Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Informational messages>
+
+These are discussed individually below.
+
+OID of toplevel tree
+~~~~~~~~~~~~~~~~~~~~
+
+This is a tree object that represents what would be checked out in the
+working tree at the end of `git merge`.  If there were conflicts, then
+files within this tree may have embedded conflict markers.
+
+Informational messages
+~~~~~~~~~~~~~~~~~~~~~~
+
+This always starts with a blank line to separate it from the previous
+section, and then has free-form messages about the merge, such as:
 
-The printed tree object corresponds to what would be checked out in
-the working tree at the end of `git merge`, and thus may have files
-with conflict markers in them.
+  * "Auto-merging <file>"
+  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
+  * "Failed to merge submodule <submodule> (<reason>)"
+  * "Warning: cannot merge binary files: <filename>"
 
 EXIT STATUS
 -----------
@@ -69,7 +99,8 @@ be used as a part of a series of steps such as
 
 However, it does not quite fit into the same category of low-level
 plumbing commands since the possibility of merge conflicts give it a
-much higher chance of the command not succeeding.
+much higher chance of the command not succeeding (and NEWTREE containing
+a bunch of stuff other than just a toplevel tree).
 
 GIT
 ---
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index d14c9f6e44e..6a556ab1c9c 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -390,6 +390,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int show_messages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -432,18 +433,27 @@ static int real_merge(struct merge_tree_options *o,
 	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
 	if (result.clean < 0)
 		die(_("failure to merge"));
+
+	if (o->show_messages == -1)
+		o->show_messages = !result.clean;
+
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (o->show_messages) {
+		printf("\n");
+		merge_display_update_messages(&opt, &result, stdout);
+	}
 	merge_finalize(&opt, &result);
 	return !result.clean; /* result.clean < 0 handled above */
 }
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { 0 };
+	struct merge_tree_options o = { .show_messages = -1 };
 	int expected_remaining_argc;
+	int original_argc;
 
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
 		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
 		NULL
 	};
@@ -453,10 +463,13 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    'w'),
 		OPT_CMDMODE(0, "trivial-merge", &o.mode,
 			    N_("do a trivial merge only"), 't'),
+		OPT_BOOL(0, "messages", &o.show_messages,
+			 N_("also show informational/conflict messages")),
 		OPT_END()
 	};
 
 	/* Parse arguments */
+	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
 	if (o.mode) {
@@ -468,6 +481,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			usage_with_options(merge_tree_usage, mt_options);
 		o.mode = (argc == 2 ? 'w' : 't');
 	}
+	if (o.mode == 't' && original_argc < argc)
+		die(_("--trivial-merge is incompatible with all other options"));
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 66c3eaf2021..e2255711f9c 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -85,4 +85,25 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+test_expect_success 'test conflict notices and such' '
+	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "numbers" should merge cleanly
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	cat <<-EOF >expect &&
+	HASH
+
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	Auto-merging numbers
+	CONFLICT (file/directory): directory in the way of whatever from side1; moving it to whatever~side1 instead.
+	CONFLICT (modify/delete): whatever~side1 deleted in side2 and modified in side1.  Version side1 of whatever~side1 left in tree.
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 10/15] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (8 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 09/15] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 11/15] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
                       ` (5 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

After a merge, this function allows the user to extract the same
information that would be printed by `ls-files -u`, which means
files with their mode, oid, and stage.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 merge-ort.c | 31 +++++++++++++++++++++++++++++++
 merge-ort.h | 21 +++++++++++++++++++++
 2 files changed, 52 insertions(+)

diff --git a/merge-ort.c b/merge-ort.c
index d28d1721d14..c4ce6027dc4 100644
--- a/merge-ort.c
+++ b/merge-ort.c
@@ -4275,6 +4275,37 @@ void merge_display_update_messages(struct merge_options *opt,
 	trace2_region_leave("merge", "display messages", opt->repo);
 }
 
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files)
+{
+	struct hashmap_iter iter;
+	struct strmap_entry *e;
+	struct merge_options_internal *opti = result->priv;
+
+	strmap_for_each_entry(&opti->conflicted, &iter, e) {
+		const char *path = e->key;
+		struct conflict_info *ci = e->value;
+		int i;
+
+		VERIFY_CI(ci);
+
+		for (i = MERGE_BASE; i <= MERGE_SIDE2; i++) {
+			struct stage_info *si;
+
+			if (!(ci->filemask & (1ul << i)))
+				continue;
+
+			si = xmalloc(sizeof(*si));
+			si->stage = i+1;
+			si->mode = ci->stages[i].mode;
+			oidcpy(&si->oid, &ci->stages[i].oid);
+			string_list_append(conflicted_files, path)->util = si;
+		}
+	}
+	/* string_list_sort() uses a stable sort, so we're good */
+	string_list_sort(conflicted_files);
+}
+
 void merge_switch_to_result(struct merge_options *opt,
 			    struct tree *head,
 			    struct merge_result *result,
diff --git a/merge-ort.h b/merge-ort.h
index d643b47cb7c..e635a294ea8 100644
--- a/merge-ort.h
+++ b/merge-ort.h
@@ -2,6 +2,7 @@
 #define MERGE_ORT_H
 
 #include "merge-recursive.h"
+#include "hash.h"
 
 struct commit;
 struct tree;
@@ -89,6 +90,26 @@ void merge_display_update_messages(struct merge_options *opt,
 				   struct merge_result *result,
 				   FILE *stream);
 
+struct stage_info {
+	struct object_id oid;
+	int mode;
+	int stage;
+};
+
+/*
+ * Provide a list of path -> {struct stage_info*} mappings for
+ * all conflicted files.  Note that each path could appear up to three
+ * times in the list, corresponding to 3 different stage entries.  In short,
+ * this basically provides the info that would be printed by `ls-files -u`.
+ *
+ * result should have been populated by a call to
+ * one of the merge_incore_[non]recursive() functions.
+ *
+ * conflicted_files should be empty before calling this function.
+ */
+void merge_get_conflicted_files(struct merge_result *result,
+				struct string_list *conflicted_files);
+
 /* Do needed cleanup when not calling merge_switch_to_result() */
 void merge_finalize(struct merge_options *opt,
 		    struct merge_result *result);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 11/15] merge-tree: provide a list of which files have conflicts
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (9 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 10/15] merge-ort: provide a merge_get_conflicted_files() helper function Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
                       ` (4 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Callers of `git merge-tree --write-tree` will often want to know which
files had conflicts.  While they could potentially attempt to parse the
CONFLICT notices printed, those messages are not meant to be machine
readable.  Provide a simpler mechanism of just printing the files (in
the same format as `git ls-files` with quoting, but restricted to
unmerged files) in the output before the free-form messages.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  8 ++++++++
 builtin/merge-tree.c             | 24 ++++++++++++++++++++++--
 t/t4301-merge-tree-write-tree.sh | 11 +++++++++++
 3 files changed, 41 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 42e0f8f6183..160e8f44b62 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -55,6 +55,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
+	<Conflicted file list>
 	<Informational messages>
 
 These are discussed individually below.
@@ -66,6 +67,13 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
+Conflicted file list
+~~~~~~~~~~~~~~~~~~~~
+
+This is a sequence of lines containing a filename on each line, quoted
+as explained for the configuration variable `core.quotePath` (see
+linkgit:git-config[1]).
+
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 6a556ab1c9c..54dae018203 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -11,6 +11,9 @@
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
+#include "quote.h"
+
+static int line_termination = '\n';
 
 struct merge_list {
 	struct merge_list *next;
@@ -394,7 +397,8 @@ struct merge_tree_options {
 };
 
 static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2)
+		      const char *branch1, const char *branch2,
+		      const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
@@ -438,6 +442,22 @@ static int real_merge(struct merge_tree_options *o,
 		o->show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
+	if (!result.clean) {
+		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
+		const char *last = NULL;
+		int i;
+
+		merge_get_conflicted_files(&result, &conflicted_files);
+		for (i = 0; i < conflicted_files.nr; i++) {
+			const char *name = conflicted_files.items[i].string;
+			if (last && !strcmp(last, name))
+				continue;
+			write_name_quoted_relative(
+				name, prefix, stdout, line_termination);
+			last = name;
+		}
+		string_list_clear(&conflicted_files, 1);
+	}
 	if (o->show_messages) {
 		printf("\n");
 		merge_display_update_messages(&opt, &result, stdout);
@@ -486,7 +506,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
-		return real_merge(&o, argv[0], argv[1]);
+		return real_merge(&o, argv[0], argv[1], prefix);
 	else
 		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index e2255711f9c..7113d060bc5 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -95,6 +95,8 @@ test_expect_success 'test conflict notices and such' '
 	#   "whatever" has *both* a modify/delete and a file/directory conflict
 	cat <<-EOF >expect &&
 	HASH
+	greeting
+	whatever~side1
 
 	Auto-merging greeting
 	CONFLICT (content): Merge conflict in greeting
@@ -106,4 +108,13 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Just the conflicted files without the messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	test_write_lines HASH greeting whatever~side1 >expect &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (10 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 11/15] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02 23:55       ` Ævar Arnfjörð Bjarmason
  2022-02-02  7:34     ` [PATCH v3 13/15] merge-tree: allow `ls-files -u` style info to be NUL terminated Elijah Newren via GitGitGadget
                       ` (3 subsequent siblings)
  15 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Much like `git merge` updates the index with information of the form
    (mode, oid, stage, name)
provide this output for conflicted files for merge-tree as well.
Provide an --exclude-modes-oids-stages/-l option for users to exclude
the mode, oid, and stage and only get the list of conflicted filenames.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 30 ++++++++++++++++++++++++------
 builtin/merge-tree.c             | 11 ++++++++++-
 t/t4301-merge-tree-write-tree.sh | 26 ++++++++++++++++++++++++--
 3 files changed, 58 insertions(+), 9 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 160e8f44b62..55bb7bc61c1 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -38,6 +38,11 @@ See `OUTPUT` below for details.
 OPTIONS
 -------
 
+--exclude-oids-and-modes::
+	Instead of writing a list of (mode, oid, stage, path) tuples
+	to output for conflicted files, just provide a list of
+	filenames with conflicts.
+
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
 	or CONFLICT notices to the end of stdout.  If unspecified, the
@@ -55,7 +60,7 @@ simply one line:
 Whereas for a conflicted merge, the output is by default of the form:
 
 	<OID of toplevel tree>
-	<Conflicted file list>
+	<Conflicted file info>
 	<Informational messages>
 
 These are discussed individually below.
@@ -67,18 +72,23 @@ This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
 files within this tree may have embedded conflict markers.
 
-Conflicted file list
+Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
 
-This is a sequence of lines containing a filename on each line, quoted
-as explained for the configuration variable `core.quotePath` (see
-linkgit:git-config[1]).
+This is a sequence of lines with the format
+
+	<mode> <object> <stage> <filename>
+
+The filename will be quoted as explained for the configuration
+variable `core.quotePath` (see linkgit:git-config[1]).  However, if
+the `--exclude-oids-and-modes` option is passed, the mode, object, and
+stage will be omitted.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
 This always starts with a blank line to separate it from the previous
-section, and then has free-form messages about the merge, such as:
+sections, and then has free-form messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
@@ -110,6 +120,14 @@ plumbing commands since the possibility of merge conflicts give it a
 much higher chance of the command not succeeding (and NEWTREE containing
 a bunch of stuff other than just a toplevel tree).
 
+git-merge-tree was written to provide users with the same information
+that they'd have access to if using `git merge`:
+  * what would be written to the working tree (the <OID of toplevel tree>)
+  * the higher order stages that would be written to the index (the
+    <Conflicted file info>)
+  * any messages that would have been printed to stdout (the <Informational
+    messages>)
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 54dae018203..dc52cd02dce 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -394,6 +394,7 @@ static int trivial_merge(const char *base,
 struct merge_tree_options {
 	int mode;
 	int show_messages;
+	int exclude_modes_oids_stages;
 };
 
 static int real_merge(struct merge_tree_options *o,
@@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
 		merge_get_conflicted_files(&result, &conflicted_files);
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
-			if (last && !strcmp(last, name))
+			struct stage_info *c = conflicted_files.items[i].util;
+			if (!o->exclude_modes_oids_stages)
+				printf("%06o %s %d\t",
+				       c->mode, oid_to_hex(&c->oid), c->stage);
+			else if (last && !strcmp(last, name))
 				continue;
 			write_name_quoted_relative(
 				name, prefix, stdout, line_termination);
@@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), 't'),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_BOOL_F('l', "exclude-modes-oids-stages",
+			   &o.exclude_modes_oids_stages,
+			   N_("list conflicted files without modes/oids/stages"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 7113d060bc5..1572f460da0 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -47,6 +47,7 @@ test_expect_success 'Content merge and a few conflicts' '
 	expected_tree=$(cat .git/AUTO_MERGE) &&
 
 	# We will redo the merge, while we are still in a conflicted state!
+	git ls-files -u >conflicted-file-info &&
 	test_when_finished "git reset --hard" &&
 
 	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
@@ -86,7 +87,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -109,7 +110,7 @@ test_expect_success 'test conflict notices and such' '
 '
 
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&
@@ -117,4 +118,25 @@ test_expect_success 'Just the conflicted files without the messages' '
 	test_cmp expect actual
 '
 
+test_expect_success 'Check conflicted oids and modes without messages' '
+	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Compare the basic output format
+	q_to_tab >expect <<-\EOF &&
+	HASH
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~side1
+	100644 HASH 2Qwhatever~side1
+	EOF
+
+	test_cmp expect actual &&
+
+	# Check the actual hashes against the `ls-files -u` output too
+	tail -n +2 out | sed -e s/side1/HEAD/ >actual &&
+	test_cmp conflicted-file-info actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 13/15] merge-tree: allow `ls-files -u` style info to be NUL terminated
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (11 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 14/15] merge-tree: add a --allow-unrelated-histories flag Elijah Newren via GitGitGadget
                       ` (2 subsequent siblings)
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Much as `git ls-files` has a -z option, let's add one to merge-tree so
that the conflict-info section can be NUL terminated (and avoid quoting
of unusual filenames).

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 21 +++++++++++++----
 builtin/merge-tree.c             |  4 +++-
 t/t4301-merge-tree-write-tree.sh | 40 ++++++++++++++++++++++++++++++++
 3 files changed, 60 insertions(+), 5 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 55bb7bc61c1..02f766716f9 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -38,6 +38,12 @@ See `OUTPUT` below for details.
 OPTIONS
 -------
 
+-z::
+	Do not quote filenames in the <Conflicted file info> section,
+	and end each filename with a NUL character rather than
+	newline.  Also begin the messages section with a NUL character
+	instead of a newline.  See OUTPUT below for more information.
+
 --exclude-oids-and-modes::
 	Instead of writing a list of (mode, oid, stage, path) tuples
 	to output for conflicted files, just provide a list of
@@ -70,7 +76,8 @@ OID of toplevel tree
 
 This is a tree object that represents what would be checked out in the
 working tree at the end of `git merge`.  If there were conflicts, then
-files within this tree may have embedded conflict markers.
+files within this tree may have embedded conflict markers.  This section
+is always followed by a newline.
 
 Conflicted file info
 ~~~~~~~~~~~~~~~~~~~~
@@ -82,19 +89,25 @@ This is a sequence of lines with the format
 The filename will be quoted as explained for the configuration
 variable `core.quotePath` (see linkgit:git-config[1]).  However, if
 the `--exclude-oids-and-modes` option is passed, the mode, object, and
-stage will be omitted.
+stage will be omitted.  If `-z` is passed, the "lines" are terminated
+by a NUL character instead of a newline character.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
 
-This always starts with a blank line to separate it from the previous
-sections, and then has free-form messages about the merge, such as:
+This always starts with a blank line (or NUL if `-z` is passed) to
+separate it from the previous sections, and then has free-form
+messages about the merge, such as:
 
   * "Auto-merging <file>"
   * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
   * "Failed to merge submodule <submodule> (<reason>)"
   * "Warning: cannot merge binary files: <filename>"
 
+Note that these free-form messages will never have a NUL character
+in or between them, even if -z is passed.  It is simply a large block
+of text taking up the remainder of the output.
+
 EXIT STATUS
 -----------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index dc52cd02dce..7e55f0fa301 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -464,7 +464,7 @@ static int real_merge(struct merge_tree_options *o,
 		string_list_clear(&conflicted_files, 1);
 	}
 	if (o->show_messages) {
-		printf("\n");
+		putchar(line_termination);
 		merge_display_update_messages(&opt, &result, stdout);
 	}
 	merge_finalize(&opt, &result);
@@ -490,6 +490,8 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			    N_("do a trivial merge only"), 't'),
 		OPT_BOOL(0, "messages", &o.show_messages,
 			 N_("also show informational/conflict messages")),
+		OPT_SET_INT('z', NULL, &line_termination,
+			    N_("separate paths with the NUL character"), '\0'),
 		OPT_BOOL_F('l', "exclude-modes-oids-stages",
 			   &o.exclude_modes_oids_stages,
 			   N_("list conflicted files without modes/oids/stages"),
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 1572f460da0..f89d87c26b7 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -139,4 +139,44 @@ test_expect_success 'Check conflicted oids and modes without messages' '
 	test_cmp conflicted-file-info actual
 '
 
+test_expect_success 'NUL terminated conflicted file "lines"' '
+	git checkout -b tweak1 side1 &&
+	test_write_lines zero 1 2 3 4 5 6 >numbers &&
+	git add numbers &&
+	git mv numbers "Αυτά μου φαίνονται κινέζικα" &&
+	git commit -m "Renamed numbers" &&
+
+	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
+
+	# Expected results:
+	#   "greeting" should merge with conflicts
+	#   "whatever" has *both* a modify/delete and a file/directory conflict
+	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
+	echo HASH >expect &&
+
+	q_to_tab <<-EOF | lf_to_nul >>expect &&
+	100644 HASH 1Qgreeting
+	100644 HASH 2Qgreeting
+	100644 HASH 3Qgreeting
+	100644 HASH 1Qwhatever~tweak1
+	100644 HASH 2Qwhatever~tweak1
+	100644 HASH 1QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 2QΑυτά μου φαίνονται κινέζικα
+	100644 HASH 3QΑυτά μου φαίνονται κινέζικα
+
+	EOF
+
+	cat <<-EOF >>expect &&
+	Auto-merging greeting
+	CONFLICT (content): Merge conflict in greeting
+	CONFLICT (file/directory): directory in the way of whatever from tweak1; moving it to whatever~tweak1 instead.
+	CONFLICT (modify/delete): whatever~tweak1 deleted in side2 and modified in tweak1.  Version tweak1 of whatever~tweak1 left in tree.
+	Auto-merging Αυτά μου φαίνονται κινέζικα
+	CONFLICT (content): Merge conflict in Αυτά μου φαίνονται κινέζικα
+	EOF
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 14/15] merge-tree: add a --allow-unrelated-histories flag
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (12 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 13/15] merge-tree: allow `ls-files -u` style info to be NUL terminated Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-02  7:34     ` [PATCH v3 15/15] git-merge-tree.txt: add a section on potentional usage mistakes Elijah Newren via GitGitGadget
  2022-02-12 20:34     ` [PATCH v4 00/12] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Folks may want to merge histories that have no common ancestry; provide
a flag with the same name as used by `git merge` to allow this.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  5 +++++
 builtin/merge-tree.c             |  7 ++++++-
 t/t4301-merge-tree-write-tree.sh | 24 +++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 02f766716f9..e6a9ff2768b 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -55,6 +55,11 @@ OPTIONS
 	default is to include these messages if there are merge
 	conflicts, and to omit them otherwise.
 
+--allow-unrelated-histories::
+	merge-tree will by default error out if the two branches specified
+	share no common history.  This flag can be given to override that
+	check and make the merge proceed anyway.
+
 OUTPUT
 ------
 
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 7e55f0fa301..58c0ddc5a32 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -393,6 +393,7 @@ static int trivial_merge(const char *base,
 
 struct merge_tree_options {
 	int mode;
+	int allow_unrelated_histories;
 	int show_messages;
 	int exclude_modes_oids_stages;
 };
@@ -430,7 +431,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common)
+	if (!common && !o->allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -496,6 +497,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			   &o.exclude_modes_oids_stages,
 			   N_("list conflicted files without modes/oids/stages"),
 			   PARSE_OPT_NONEG),
+		OPT_BOOL_F(0, "allow-unrelated-histories",
+			   &o.allow_unrelated_histories,
+			   N_("allow merging unrelated histories"),
+			   PARSE_OPT_NONEG),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index f89d87c26b7..4de089d976d 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -38,7 +38,13 @@ test_expect_success setup '
 	>whatever/empty &&
 	git add numbers greeting whatever/empty &&
 	test_tick &&
-	git commit -m other-modifications
+	git commit -m other-modifications &&
+
+	git switch --orphan unrelated &&
+	>something-else &&
+	git add something-else &&
+	test_tick &&
+	git commit -m first-commit
 '
 
 test_expect_success 'Content merge and a few conflicts' '
@@ -179,4 +185,20 @@ test_expect_success 'NUL terminated conflicted file "lines"' '
 	test_cmp expect actual
 '
 
+test_expect_success 'error out by default for unrelated histories' '
+	test_expect_code 128 git merge-tree --write-tree side1 unrelated 2>error &&
+
+	grep "refusing to merge unrelated histories" error
+'
+
+test_expect_success 'can override merge of unrelated histories' '
+	git merge-tree --write-tree --allow-unrelated-histories side1 unrelated >tree &&
+	TREE=$(cat tree) &&
+
+	git rev-parse side1:numbers side1:greeting side1:whatever unrelated:something-else >expect &&
+	git rev-parse $TREE:numbers $TREE:greeting $TREE:whatever $TREE:something-else >actual &&
+
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v3 15/15] git-merge-tree.txt: add a section on potentional usage mistakes
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (13 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 14/15] merge-tree: add a --allow-unrelated-histories flag Elijah Newren via GitGitGadget
@ 2022-02-02  7:34     ` Elijah Newren via GitGitGadget
  2022-02-12 20:34     ` [PATCH v4 00/12] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  15 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-02  7:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index e6a9ff2768b..6a2ed475106 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -146,6 +146,52 @@ that they'd have access to if using `git merge`:
   * any messages that would have been printed to stdout (the <Informational
     messages>)
 
+MISTAKES TO AVOID
+-----------------
+
+Do NOT look through the resulting toplevel tree to try to find which
+files conflict; parse the <Conflicted file info> section instead.  Not
+only would parsing an entire tree be horrendously slow in large
+repositories, there are numerous types of conflicts not representable by
+conflict markers (modify/delete, mode conflict, binary file changed on
+both sides, file/directory conflicts, various rename conflict
+permutations, etc.)
+
+Do NOT interpret an empty <Conflicted file info> list as a clean merge;
+check the exit status.  A merge can have conflicts without having
+individual files conflict (there are a few types of directory rename
+conflicts that fall into this category, and others might also be added
+in the future).
+
+Do NOT attempt to guess or make the user guess the conflict types from
+the <Conflicted file info> list.  The information there is insufficient
+to do so.  For example: Rename/rename(1to2) conflicts (both sides
+renamed the same file differently) will result in three different file
+having higher order stages (but each only has one higher order stage),
+with no way (short of the <Informational messages> section) to determine
+which three files are related.  File/directory conflicts also result in
+a file with exactly one higher order stage.
+Possibly-involved-in-directory-rename conflicts (when
+"merge.directoryRenames" is unset or set to "conflicts") also result in
+a file with exactly one higher order stage.  In all cases, the
+<Informational messages> section has the necessary info, though it is
+not designed to be machine parseable.
+
+Do NOT assume all filenames listed in the <Informational messages>
+section had conflicts.  Messages can be included for files that have no
+conflicts, such as "Auto-merging <file>".
+
+AVOID taking the OIDS from the <Conflicted file info> and re-merging
+them to present the conflicts to the user.  This will lose information.
+Instead, look up the version of the file found within the <OID of
+toplevel tree> and show that instead.  In particular, the latter will
+have conflict markers annotated with the original branch/commit being
+merged and, if renames were involved, the original filename.  While you
+could include the original branch/commit in the conflict marker
+annotations when re-merging, the original filename is not available from
+the <Conflicted file info> and thus you would be losing information that
+might help the user resolve the conflict.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-02  7:34     ` [PATCH v3 04/15] merge-tree: implement real merges Elijah Newren via GitGitGadget
@ 2022-02-02 21:22       ` Junio C Hamano
  2022-02-02 21:56         ` Elijah Newren
  2022-02-04  4:48       ` Josh Steadmon
  1 sibling, 1 reply; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:22 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> @@ -392,7 +395,46 @@ struct merge_tree_options {
>  static int real_merge(struct merge_tree_options *o,
>  		      const char *branch1, const char *branch2)
>  {
> -	die(_("real merges are not yet implemented"));
> +	struct commit *parent1, *parent2;
> +	struct commit_list *common;
> +	struct commit_list *merge_bases = NULL;
> +	struct commit_list *j;
> +	struct merge_options opt;
> +	struct merge_result result = { 0 };
> +
> +	parent1 = get_merge_parent(branch1);
> +	if (!parent1)
> +		help_unknown_ref(branch1, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	parent2 = get_merge_parent(branch2);
> +	if (!parent2)
> +		help_unknown_ref(branch2, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	init_merge_options(&opt, the_repository);
> +
> +	opt.show_rename_progress = 0;
> +
> +	opt.branch1 = branch1;
> +	opt.branch2 = branch2;
> +
> +	/*
> +	 * Get the merge bases, in reverse order; see comment above
> +	 * merge_incore_recursive in merge-ort.h
> +	 */
> +	common = get_merge_bases(parent1, parent2);
> +	if (!common)
> +		die(_("refusing to merge unrelated histories"));

It appears to me that "merge-tree" in this mode, with the above
code, cannot be used as a workhorse to implement server-side
cherry-pick (or revert), which needs to allow the user to specify an
arbitrary "common ancestor", instead of computing on its own.

To replay the change made by commit A on top of commit X (i.e.
"cherry-pick A on X"), we have to be able to say "compute the
three-way merge between A and X, pretending as if A^ were their
common ancestor".  The story is the same for revert---we compute
three-way merge between A^ and X, pretending as if A were their
common ancestor.

The above interface into this function, sadly, does not seem to
allow such a request, unless I am missing something.

And if I am correct, it is a shame---after all, the point of the
merge-trees command is to take three trees and run a three-way
merge, and not being able to merge three "trees" and require
"commits" makes this mode much less useful than its potential.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 03/13] merge-tree: add option parsing and initial shell for real merge function
  2022-01-29 18:07   ` [PATCH v2 03/13] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
@ 2022-02-02 21:30     ` Junio C Hamano
  0 siblings, 0 replies; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:30 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +struct merge_tree_options {
> +	int mode;
> +};

> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_CMDMODE(0, "write-tree", &o.mode,
> +			    N_("do a real merge instead of a trivial merge"),
> +			    'w'),

Given the length of the second line in the usage[] array, it would
make more sense to have "'w')," on the same line to make it match
better with the following one.

> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
> +			    N_("do a trivial merge only"), 't'),
> +		OPT_END()
> +	};
> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +	if (o.mode) {
> +		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.mode = (argc == 2 ? 'w' : 't');
> +	}

We are not planning to have tons of different command modes, but the
above looks quite brittle in assuming that 'w' and 't' are the only
ones, and not having an easy way to extend the part without major
rewrite when the assumption will have to be broken.  I wonder:

	switch (o.cmd_mode) {
        default:
		BUG("unexpected cmdmode %c", o.cmd_mode);
	case 0:
		switch (argc) {
		default:
			usage_with_options(merge_tree_usage, mt_options);
		case 2: 
			o.cmd_mode = 'w';
			break;
		case 3:
			o.cmd_mode = 't';
			break;
		}
                expected_remaining_argc = argc;
		break;
	case 'w':
		expected_remaining_argc = 2;
                break;
	case 't':
		expected_remaining_argc = 3;
 		break;
	}

        if (argc != expected_remaining_argc)
		usage_with_options(merge_tree_usage, mt_options);

even though it is a tad longer with more boilerplate, is easlier to
manage.

> +	/* Do the relevant type of merge */
> +	if (o.mode == 'w')
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }
> diff --git a/git.c b/git.c
> index 5ff21be21f3..6090a1289db 100644
> --- a/git.c
> +++ b/git.c
> @@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
>  	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
>  	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
> -	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
> +	{ "merge-tree", cmd_merge_tree, RUN_SETUP },

This affects git_support_parseopt_helper function in the completion
script, but it by itself is very unlikely to break existing tests on
the completion, as the bit only affects how the command line options
are completed, and the command didn't have any command line options
to be tested before this series ;-)

>  	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
>  	{ "mktree", cmd_mktree, RUN_SETUP },
>  	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 04/13] merge-tree: implement real merges
  2022-01-29 18:07   ` [PATCH v2 04/13] merge-tree: implement real merges Elijah Newren via GitGitGadget
@ 2022-02-02 21:30     ` Junio C Hamano
  2022-02-02 22:00       ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:30 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Elijah Newren <newren@gmail.com>
>
> This adds the ability to perform real merges rather than just trivial
> merges (meaning handling three way content merges, recursive ancestor
> consolidation, renames, proper directory/file conflict handling, and so
> forth).  However, unlike `git merge`, the working tree and index are
> left alone and no branch is updated.
>
> The only output is:
>   - the toplevel resulting tree printed on stdout
>   - exit status of 0 (clean), 1 (conflicts present), anything else
>     (merge could not be performed; unknown if clean or conflicted)
>
> This output is meant to be used by some higher level script, perhaps in
> a sequence of steps like this:
>
>    NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
>    test $? -eq 0 || die "There were conflicts..."
>    NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
>    git update-ref $BRANCH1 $NEWCOMMIT

It is unclear what NEWTREE has, if anything meaningful, when the
command exited with non-zero status.  Let's read on.

> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 58731c19422..569485815a0 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -3,26 +3,73 @@ git-merge-tree(1)
>  
>  NAME
>  ----
> -git-merge-tree - Show three-way merge without touching index
> +git-merge-tree - Perform merge without touching index or working tree

OK.  It is interesting that both apply equally to either command
mode ;-)

> +Performs a merge, but does not make any new commits and does not read
> +from or write to either the working tree or index.
> +
> +The second form is deprecated and supported only for backward
> +compatibility.  It will likely be removed in the future, and will not
> +be discussed further in this manual.

This, especially the deletion of the original description on what
trivial merge does, may be premature, especially if it is still
"supported for backward compatibility".

> +The first form will merge the two branches, doing a real merge.  A real
> +merge is distinguished from a trivial merge in that it includes:

I think this is worth keeping, simply because I do not think you
should entirely omit description on the trivial one.

But if we were to remove the description on the trivial one, then it
is totally meaningless to label the following as "... distinguished
from a trivial merge in that".  The list of things the real merge
command mode does (below) is of course worth having.

> +  * three way content merges of individual files
> +  * rename detection
> +  * proper directory/file conflict handling
> +  * recursive ancestor consolidation (i.e. when there is more than one
> +    merge base, creating a virtual merge base by merging the merge bases)
> +  * etc.
> +
> +After the merge completes, it will create a new toplevel tree object.
> +See `OUTPUT` below for details.
> +
> +OUTPUT
> +------
> +
> +For either a successful or conflicted merge, the output from
> +git-merge-tree is simply one line:
> +
> +	<OID of toplevel tree>
> +
> +The printed tree object corresponds to what would be checked out in
> +the working tree at the end of `git merge`, and thus may have files
> +with conflict markers in them.

So we would leave cruft in the object store, but that is very much
on purpose, and the expectation is that the user would commit-tree
the tree object and reference it with a ref soon enough before
garbage collection prunes them, just like how write-tree is meant to
be used.  OK.

> +EXIT STATUS
> +-----------
> +
> +For a successful, non-conflicted merge, the exit status is 0.  When the
> +merge has conflicts, the exit status is 1.  If the merge is not able to
> +complete (or start) due to some kind of error, the exit status is
> +something other than 0 or 1.

And the output given to the standard output stream, when the command
exits with status higher than 1, is...?  "unspecified" is of course
an acceptable answer, of course, but I am wondering if it is worth
spelling out in the doc.

> +USAGE NOTES
> +-----------
> +
> +git-merge-tree was written to be low-level plumbing, similar to
> +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could

A notable omission in the above list is 'write-tree'.

> +be used as a part of a series of steps such as
> +
> +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> +       test $? -eq 0 || die "There were conflicts..."
> +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> +       git update-ref $BRANCH1 $NEWCOMMIT
> +
> +However, it does not quite fit into the same category of low-level
> +plumbing commands since the possibility of merge conflicts give it a
> +much higher chance of the command not succeeding.

I am not sure if that is a fair categorization.  It is a fine
building block at the lowest level of the tool hierarchy.  The
primary thing that differentiates plumbing from Porcelain is that
the former is a better fit for scripting: doing one thing and one
thing well with minimum and stable UI.  The complexity of that one
thing it does has much less to do with the categorization, and the
rate at which users may use the command in a failure-inducing
situation has nothing to do with it.  The write-tree plumbing
command will reliably fail with 100% chance when the index used as
its input is unmerged.

> @@ -392,7 +395,46 @@ struct merge_tree_options {
>  static int real_merge(struct merge_tree_options *o,
>  		      const char *branch1, const char *branch2)
>  {
> -	die(_("real merges are not yet implemented"));
> +	struct commit *parent1, *parent2;
> +	struct commit_list *common;
> +	struct commit_list *merge_bases = NULL;
> +	struct commit_list *j;
> +	struct merge_options opt;
> +	struct merge_result result = { 0 };
> +
> +	parent1 = get_merge_parent(branch1);
> +	if (!parent1)
> +		help_unknown_ref(branch1, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	parent2 = get_merge_parent(branch2);
> +	if (!parent2)
> +		help_unknown_ref(branch2, "merge-tree",
> +				 _("not something we can merge"));
> +
> +	init_merge_options(&opt, the_repository);
> +
> +	opt.show_rename_progress = 0;
> +
> +	opt.branch1 = branch1;
> +	opt.branch2 = branch2;
> +
> +	/*
> +	 * Get the merge bases, in reverse order; see comment above
> +	 * merge_incore_recursive in merge-ort.h
> +	 */
> +	common = get_merge_bases(parent1, parent2);
> +	if (!common)
> +		die(_("refusing to merge unrelated histories"));
> +	for (j = common; j; j = j->next)
> +		commit_list_insert(j->item, &merge_bases);
> +
> +	merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> +	if (result.clean < 0)
> +		die(_("failure to merge"));
> +	puts(oid_to_hex(&result.tree->object.oid));
> +	merge_finalize(&opt, &result);
> +	return !result.clean; /* result.clean < 0 handled above */
>  }

The implementation is rather straight-forward, if you know how to
drive the merge_incore_recursive() helper ;-)

> +test_expect_success setup '
> +	test_write_lines 1 2 3 4 5 >numbers &&
> +	echo hello >greeting &&
> +	echo foo >whatever &&
> +	git add numbers greeting whatever &&
> +	test_tick &&
> +	git commit -m initial &&
> +
> +	git branch side1 &&
> +	git branch side2 &&
> +
> +	git checkout side1 &&
> +	test_write_lines 1 2 3 4 5 6 >numbers &&
> +	echo hi >greeting &&
> +	echo bar >whatever &&
> +	git add numbers greeting whatever &&
> +	test_tick &&
> +	git commit -m modify-stuff &&
> +
> +	git checkout side2 &&
> +	test_write_lines 0 1 2 3 4 5 >numbers &&
> +	echo yo >greeting &&
> +	git rm whatever &&
> +	mkdir whatever &&
> +	>whatever/empty &&
> +	git add numbers greeting whatever/empty &&
> +	test_tick &&
> +	git commit -m other-modifications
> +'
> +
> +test_expect_success 'Content merge and a few conflicts' '
> +	git checkout side1^0 &&
> +	test_must_fail git merge side2 &&
> +	expected_tree=$(cat .git/AUTO_MERGE) &&
> +
> +	# We will redo the merge, while we are still in a conflicted state!
> +	test_when_finished "git reset --hard" &&
> +
> +	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
> +	actual_tree=$(head -n 1 RESULT) &&
> +
> +	# Due to differences of e.g. "HEAD" vs "side1", the results will not
> +	# exactly match.  Dig into individual files.
> +
> +	# Numbers should have three-way merged cleanly
> +	test_write_lines 0 1 2 3 4 5 6 >expect &&
> +	git show ${actual_tree}:numbers >actual &&
> +	test_cmp expect actual &&
> +
> +	# whatever and whatever~<branch> should have same HASHES
> +	git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
> +	git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
> +	test_cmp expect actual &&
> +
> +	# greeting should have a merge conflict
> +	git show ${expected_tree}:greeting >tmp &&
> +	cat tmp | sed -e s/HEAD/side1/ >expect &&
> +	git show ${actual_tree}:greeting >actual &&
> +	test_cmp expect actual
> +'

It is somewhat sad that we need to reivent merge test cases over and
over, instead of easily reuse an existing one by replacing

	git checkout one &&
	git merge two

with

	git checkout one &&
	T=$(git merge-tree HEAD two) &&
	C=$(git commit-tree $T -p HEAD -p two) &&
	git reset --hard $C

;-)

> +> +test_expect_success 'Barf on misspelled option, with exit code other than 0 or 1' '
> +	# Mis-spell with single "s" instead of double "s"
> +	test_expect_code 129 git merge-tree --write-tree --mesages FOOBAR side1 side2 2>expect &&
> +
> +	grep "error: unknown option.*mesages" expect
> +'
> +
> +test_expect_success 'Barf on too many arguments' '
> +	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
> +
> +	grep "^usage: git merge-tree" expect
> +'


^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 08/13] merge-tree: support including merge messages in output
  2022-01-29 18:07   ` [PATCH v2 08/13] merge-tree: support including merge messages in output Elijah Newren via GitGitGadget
@ 2022-02-02 21:30     ` Junio C Hamano
  2022-02-02 23:09       ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:30 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +By default, for a successful merge, the output from git-merge-tree is
> +simply one line:
> +
> +	<OID of toplevel tree>
> +
> +Whereas for a conflicted merge, the output is by default of the form:
>  
>  	<OID of toplevel tree>
> +	<Informational messages>

Sounds useful.  This made me wonder, as the only shuffling of the
output destination in the past few steps were to send the output to
some "FILE *", how you send the findings you make while coming up
with the result _after_ the result.  It turns out that the ORT
machinery already buffers these findings in a strbuf per path, so
there is no trouble doing so ;-)

It still makes me wonder how the "send rename warnings to the
standard output stream, instead of the standard error stream" change
interacts with this change, though.  That needs to be done way
before you finish computing the result, and it does not seem to be
buffered in-core, like per-path conflict information messages.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts
  2022-01-29 18:07   ` [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
@ 2022-02-02 21:32     ` Junio C Hamano
  2022-02-02 21:32     ` Junio C Hamano
  2022-02-03 23:55     ` Junio C Hamano
  2 siblings, 0 replies; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:32 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).

Makes sense.  Ideally things like this should be discoverable by
inspecting the tree object shown as the result of the (conflicted)
merge, but since the design of the output is to show only a single
tree, there is nowhere to store such an extra piece of information
per path (grepping for markers in blobs of course does not count).

I guess an alternative to show four trees when conflicted instead of
one (i.e. the primary tree may either contain only the cleanly
merged paths _or_ also blobs with conflict markers for conflicted
paths; the three other trees record three stages that would be in
the index, if we were performing the same merge using the index),
but a machine-parseable list of paths is fine.

> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;

OK.  The iteration used here makes casual readers wonder why the
helper doesn't make paths unique, but the string list item holds
in its util pointer a pointer to a structure with <stage, mode, oid>
tuple, so it is natural to make the consumer, who wants uniquified
list, responsible for deduping, like this loop.

> +		}
> +		string_list_clear(&conflicted_files, 1);

And the stage-info structure associated with these paths are
deallocated with this call.  Good.

> +	}


^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts
  2022-01-29 18:07   ` [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
  2022-02-02 21:32     ` Junio C Hamano
@ 2022-02-02 21:32     ` Junio C Hamano
  2022-02-03 23:55     ` Junio C Hamano
  2 siblings, 0 replies; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:32 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).

Makes sense.  Ideally things like this should be discoverable by
inspecting the tree object shown as the result of the (conflicted)
merge, but since the design of the output is to show only a single
tree, there is nowhere to store such an extra piece of information
per path (grepping for markers in blobs of course does not count).

I guess an alternative to show four trees when conflicted instead of
one (i.e. the primary tree may either contain only the cleanly
merged paths _or_ also blobs with conflict markers for conflicted
paths; the three other trees record three stages that would be in
the index, if we were performing the same merge using the index),
but a machine-parseable list of paths is fine.

> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;

OK.  The iteration used here makes casual readers wonder why the
helper doesn't make paths unique, but the string list item holds
in its util pointer a pointer to a structure with <stage, mode, oid>
tuple, so it is natural to make the consumer, who wants uniquified
list, responsible for deduping, like this loop.

> +		}
> +		string_list_clear(&conflicted_files, 1);

And the stage-info structure associated with these paths are
deallocated with this call.  Good.

> +	}


^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info
  2022-01-29 18:07   ` [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
@ 2022-02-02 21:32     ` Junio C Hamano
  2022-02-02 23:18       ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:32 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
>  		merge_get_conflicted_files(&result, &conflicted_files);
>  		for (i = 0; i < conflicted_files.nr; i++) {
>  			const char *name = conflicted_files.items[i].string;
> -			if (last && !strcmp(last, name))
> +			struct stage_info *c = conflicted_files.items[i].util;
> +			if (!o->exclude_modes_oids_stages)
> +				printf("%06o %s %d\t",
> +				       c->mode, oid_to_hex(&c->oid), c->stage);
> +			else if (last && !strcmp(last, name))
>  				continue;
>  			write_name_quoted_relative(
>  				name, prefix, stdout, line_termination);

OK.  The addition (and disabling of the deduping) is quite trivial.
We do not even have to worry about line termination since the extra
pieces of info are prepended to the pathname.  Nice.

> @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			    N_("do a trivial merge only"), 't'),
>  		OPT_BOOL(0, "messages", &o.show_messages,
>  			 N_("also show informational/conflict messages")),
> +		OPT_BOOL_F('l', "exclude-modes-oids-stages",
> +			   &o.exclude_modes_oids_stages,
> +			   N_("list conflicted files without modes/oids/stages"),
> +			   PARSE_OPT_NONEG),

Why does "-l" give shorter output than without it?  "-l" strongly
hints a longer output than without, at least to me.  Just wondering
if this will not become a source of confusion to future scripting
users.

>  		OPT_END()
>  	};
>  

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 12/13] merge-tree: add a --allow-unrelated-histories flag
  2022-01-29 18:07   ` [PATCH v2 12/13] merge-tree: add a --allow-unrelated-histories flag Elijah Newren via GitGitGadget
@ 2022-02-02 21:32     ` Junio C Hamano
  0 siblings, 0 replies; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 21:32 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> @@ -430,7 +431,7 @@ static int real_merge(struct merge_tree_options *o,
>  	 * merge_incore_recursive in merge-ort.h
>  	 */
>  	common = get_merge_bases(parent1, parent2);
> -	if (!common)
> +	if (!common && !o->allow_unrelated_histories)
>  		die(_("refusing to merge unrelated histories"));
>  	for (j = common; j; j = j->next)
>  		commit_list_insert(j->item, &merge_bases);

Curious.  This step _adds_ an "--allow" option from the command
line, but we actually did not have to die() when seeing that there
is no common ancestor before this step.  The end result is OK either
way.

> @@ -494,6 +495,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  			   &o.exclude_modes_oids_stages,
>  			   N_("list conflicted files without modes/oids/stages"),
>  			   PARSE_OPT_NONEG),
> +		OPT_BOOL_F(0, "allow-unrelated-histories",
> +			   &o.allow_unrelated_histories,
> +			   N_("allow merging unrelated histories"),
> +			   PARSE_OPT_NONEG),
>  		OPT_END()
>  	};

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-02 21:22       ` Junio C Hamano
@ 2022-02-02 21:56         ` Elijah Newren
  2022-02-02 22:01           ` Junio C Hamano
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-02 21:56 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Johannes Sixt

On Wed, Feb 2, 2022 at 1:22 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > @@ -392,7 +395,46 @@ struct merge_tree_options {
> >  static int real_merge(struct merge_tree_options *o,
> >                     const char *branch1, const char *branch2)
> >  {
> > -     die(_("real merges are not yet implemented"));
> > +     struct commit *parent1, *parent2;
> > +     struct commit_list *common;
> > +     struct commit_list *merge_bases = NULL;
> > +     struct commit_list *j;
> > +     struct merge_options opt;
> > +     struct merge_result result = { 0 };
> > +
> > +     parent1 = get_merge_parent(branch1);
> > +     if (!parent1)
> > +             help_unknown_ref(branch1, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     parent2 = get_merge_parent(branch2);
> > +     if (!parent2)
> > +             help_unknown_ref(branch2, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     init_merge_options(&opt, the_repository);
> > +
> > +     opt.show_rename_progress = 0;
> > +
> > +     opt.branch1 = branch1;
> > +     opt.branch2 = branch2;
> > +
> > +     /*
> > +      * Get the merge bases, in reverse order; see comment above
> > +      * merge_incore_recursive in merge-ort.h
> > +      */
> > +     common = get_merge_bases(parent1, parent2);
> > +     if (!common)
> > +             die(_("refusing to merge unrelated histories"));
>
> It appears to me that "merge-tree" in this mode, with the above
> code, cannot be used as a workhorse to implement server-side
> cherry-pick (or revert), which needs to allow the user to specify an
> arbitrary "common ancestor", instead of computing on its own.
>
> To replay the change made by commit A on top of commit X (i.e.
> "cherry-pick A on X"), we have to be able to say "compute the
> three-way merge between A and X, pretending as if A^ were their
> common ancestor".  The story is the same for revert---we compute
> three-way merge between A^ and X, pretending as if A were their
> common ancestor.
>
> The above interface into this function, sadly, does not seem to
> allow such a request, unless I am missing something.
>
> And if I am correct, it is a shame---after all, the point of the
> merge-trees command is to take three trees and run a three-way
> merge, and not being able to merge three "trees" and require
> "commits" makes this mode much less useful than its potential.

Yes, you are reading right.  I think the cherry-pick/rebase
replacement actually deserves a separate command from what merges
should use; replaying a sequence of commits just has a number of UI
differences and abilities that I think pull it in a different
direction.  I didn't want to wait and submit everything all at once
(this series is long enough), and I figured that providing the in-core
equivalent to `git merge` was a simpler first step worth submitting
first before later providing the in-core equivalents to `git
rebase/git cherry-pick`.

(Also, I'm a bit wary of providing a command meant to just do a single
three-way merge with a defined merge-base, because that seems to be a
path towards returning to a scripted rebase.  Having a way to do only
a single such special merge is fine but we should avoid encouraging
people to go down that route.)

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 04/13] merge-tree: implement real merges
  2022-02-02 21:30     ` Junio C Hamano
@ 2022-02-02 22:00       ` Elijah Newren
  2022-02-21  8:40         ` Johannes Schindelin
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-02 22:00 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Johannes Sixt

On Wed, Feb 2, 2022 at 1:30 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > This adds the ability to perform real merges rather than just trivial
> > merges (meaning handling three way content merges, recursive ancestor
> > consolidation, renames, proper directory/file conflict handling, and so
> > forth).  However, unlike `git merge`, the working tree and index are
> > left alone and no branch is updated.
> >
> > The only output is:
> >   - the toplevel resulting tree printed on stdout
> >   - exit status of 0 (clean), 1 (conflicts present), anything else
> >     (merge could not be performed; unknown if clean or conflicted)
> >
> > This output is meant to be used by some higher level script, perhaps in
> > a sequence of steps like this:
> >
> >    NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> >    test $? -eq 0 || die "There were conflicts..."
> >    NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> >    git update-ref $BRANCH1 $NEWCOMMIT
>
> It is unclear what NEWTREE has, if anything meaningful, when the
> command exited with non-zero status.  Let's read on.
>
> > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > index 58731c19422..569485815a0 100644
> > --- a/Documentation/git-merge-tree.txt
> > +++ b/Documentation/git-merge-tree.txt
> > @@ -3,26 +3,73 @@ git-merge-tree(1)
> >
> >  NAME
> >  ----
> > -git-merge-tree - Show three-way merge without touching index
> > +git-merge-tree - Perform merge without touching index or working tree
>
> OK.  It is interesting that both apply equally to either command
> mode ;-)
>
> > +Performs a merge, but does not make any new commits and does not read
> > +from or write to either the working tree or index.
> > +
> > +The second form is deprecated and supported only for backward
> > +compatibility.  It will likely be removed in the future, and will not
> > +be discussed further in this manual.
>
> This, especially the deletion of the original description on what
> trivial merge does, may be premature, especially if it is still
> "supported for backward compatibility".

I actually extended it, but Dscho suggested removing it entirely --
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2201251804250.2121@tvgsbejvaqbjf.bet/.
I can restore it; does that paragraph look good to you (you can see
the full thing even if it's split by Dscho's commentary).

> > +The first form will merge the two branches, doing a real merge.  A real
> > +merge is distinguished from a trivial merge in that it includes:
>
> I think this is worth keeping, simply because I do not think you
> should entirely omit description on the trivial one.
>
> But if we were to remove the description on the trivial one, then it
> is totally meaningless to label the following as "... distinguished
> from a trivial merge in that".  The list of things the real merge
> command mode does (below) is of course worth having.
>
> > +  * three way content merges of individual files
> > +  * rename detection
> > +  * proper directory/file conflict handling
> > +  * recursive ancestor consolidation (i.e. when there is more than one
> > +    merge base, creating a virtual merge base by merging the merge bases)
> > +  * etc.
> > +
> > +After the merge completes, it will create a new toplevel tree object.
> > +See `OUTPUT` below for details.
> > +
> > +OUTPUT
> > +------
> > +
> > +For either a successful or conflicted merge, the output from
> > +git-merge-tree is simply one line:
> > +
> > +     <OID of toplevel tree>
> > +
> > +The printed tree object corresponds to what would be checked out in
> > +the working tree at the end of `git merge`, and thus may have files
> > +with conflict markers in them.
>
> So we would leave cruft in the object store, but that is very much
> on purpose, and the expectation is that the user would commit-tree
> the tree object and reference it with a ref soon enough before
> garbage collection prunes them, just like how write-tree is meant to
> be used.  OK.
>
> > +EXIT STATUS
> > +-----------
> > +
> > +For a successful, non-conflicted merge, the exit status is 0.  When the
> > +merge has conflicts, the exit status is 1.  If the merge is not able to
> > +complete (or start) due to some kind of error, the exit status is
> > +something other than 0 or 1.
>
> And the output given to the standard output stream, when the command
> exits with status higher than 1, is...?  "unspecified" is of course
> an acceptable answer, of course, but I am wondering if it is worth
> spelling out in the doc.

Yeah, "unspecified" -- whatever random error the user triggered
(couldn't parse arguments, the refs you gave don't exist, the repo you
are running from isn't a repo, disk is full, etc.)

> > +USAGE NOTES
> > +-----------
> > +
> > +git-merge-tree was written to be low-level plumbing, similar to
> > +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
>
> A notable omission in the above list is 'write-tree'.

Ooh, yeah, I should add that one.

> > +be used as a part of a series of steps such as
> > +
> > +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> > +       test $? -eq 0 || die "There were conflicts..."
> > +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> > +       git update-ref $BRANCH1 $NEWCOMMIT
> > +
> > +However, it does not quite fit into the same category of low-level
> > +plumbing commands since the possibility of merge conflicts give it a
> > +much higher chance of the command not succeeding.
>
> I am not sure if that is a fair categorization.  It is a fine
> building block at the lowest level of the tool hierarchy.  The
> primary thing that differentiates plumbing from Porcelain is that
> the former is a better fit for scripting: doing one thing and one
> thing well with minimum and stable UI.  The complexity of that one
> thing it does has much less to do with the categorization, and the
> rate at which users may use the command in a failure-inducing
> situation has nothing to do with it.  The write-tree plumbing
> command will reliably fail with 100% chance when the index used as
> its input is unmerged.
>
> > @@ -392,7 +395,46 @@ struct merge_tree_options {
> >  static int real_merge(struct merge_tree_options *o,
> >                     const char *branch1, const char *branch2)
> >  {
> > -     die(_("real merges are not yet implemented"));
> > +     struct commit *parent1, *parent2;
> > +     struct commit_list *common;
> > +     struct commit_list *merge_bases = NULL;
> > +     struct commit_list *j;
> > +     struct merge_options opt;
> > +     struct merge_result result = { 0 };
> > +
> > +     parent1 = get_merge_parent(branch1);
> > +     if (!parent1)
> > +             help_unknown_ref(branch1, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     parent2 = get_merge_parent(branch2);
> > +     if (!parent2)
> > +             help_unknown_ref(branch2, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     init_merge_options(&opt, the_repository);
> > +
> > +     opt.show_rename_progress = 0;
> > +
> > +     opt.branch1 = branch1;
> > +     opt.branch2 = branch2;
> > +
> > +     /*
> > +      * Get the merge bases, in reverse order; see comment above
> > +      * merge_incore_recursive in merge-ort.h
> > +      */
> > +     common = get_merge_bases(parent1, parent2);
> > +     if (!common)
> > +             die(_("refusing to merge unrelated histories"));
> > +     for (j = common; j; j = j->next)
> > +             commit_list_insert(j->item, &merge_bases);
> > +
> > +     merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > +     if (result.clean < 0)
> > +             die(_("failure to merge"));
> > +     puts(oid_to_hex(&result.tree->object.oid));
> > +     merge_finalize(&opt, &result);
> > +     return !result.clean; /* result.clean < 0 handled above */
> >  }
>
> The implementation is rather straight-forward, if you know how to
> drive the merge_incore_recursive() helper ;-)

:-)

>
> > +test_expect_success setup '
> > +     test_write_lines 1 2 3 4 5 >numbers &&
> > +     echo hello >greeting &&
> > +     echo foo >whatever &&
> > +     git add numbers greeting whatever &&
> > +     test_tick &&
> > +     git commit -m initial &&
> > +
> > +     git branch side1 &&
> > +     git branch side2 &&
> > +
> > +     git checkout side1 &&
> > +     test_write_lines 1 2 3 4 5 6 >numbers &&
> > +     echo hi >greeting &&
> > +     echo bar >whatever &&
> > +     git add numbers greeting whatever &&
> > +     test_tick &&
> > +     git commit -m modify-stuff &&
> > +
> > +     git checkout side2 &&
> > +     test_write_lines 0 1 2 3 4 5 >numbers &&
> > +     echo yo >greeting &&
> > +     git rm whatever &&
> > +     mkdir whatever &&
> > +     >whatever/empty &&
> > +     git add numbers greeting whatever/empty &&
> > +     test_tick &&
> > +     git commit -m other-modifications
> > +'
> > +
> > +test_expect_success 'Content merge and a few conflicts' '
> > +     git checkout side1^0 &&
> > +     test_must_fail git merge side2 &&
> > +     expected_tree=$(cat .git/AUTO_MERGE) &&
> > +
> > +     # We will redo the merge, while we are still in a conflicted state!
> > +     test_when_finished "git reset --hard" &&
> > +
> > +     test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
> > +     actual_tree=$(head -n 1 RESULT) &&
> > +
> > +     # Due to differences of e.g. "HEAD" vs "side1", the results will not
> > +     # exactly match.  Dig into individual files.
> > +
> > +     # Numbers should have three-way merged cleanly
> > +     test_write_lines 0 1 2 3 4 5 6 >expect &&
> > +     git show ${actual_tree}:numbers >actual &&
> > +     test_cmp expect actual &&
> > +
> > +     # whatever and whatever~<branch> should have same HASHES
> > +     git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
> > +     git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
> > +     test_cmp expect actual &&
> > +
> > +     # greeting should have a merge conflict
> > +     git show ${expected_tree}:greeting >tmp &&
> > +     cat tmp | sed -e s/HEAD/side1/ >expect &&
> > +     git show ${actual_tree}:greeting >actual &&
> > +     test_cmp expect actual
> > +'
>
> It is somewhat sad that we need to reivent merge test cases over and
> over, instead of easily reuse an existing one by replacing
>
>         git checkout one &&
>         git merge two
>
> with
>
>         git checkout one &&
>         T=$(git merge-tree HEAD two) &&
>         C=$(git commit-tree $T -p HEAD -p two) &&
>         git reset --hard $C
>
> ;-)

Sorry...I'm afraid I'm not following.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-02 21:56         ` Elijah Newren
@ 2022-02-02 22:01           ` Junio C Hamano
  2022-02-03  0:18             ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Junio C Hamano @ 2022-02-02 22:01 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Johannes Sixt

Elijah Newren <newren@gmail.com> writes:

> Yes, you are reading right.  I think the cherry-pick/rebase
> replacement actually deserves a separate command from what merges
> should use; replaying a sequence of commits just has a number of UI
> differences and abilities that I think pull it in a different
> direction.

I completely disagree.  Each individual step in a sequence of
replaying commits in order (or in reverse order) should be
scriptable as a single merge-tree that takes "apply the change to go
from A^ to A on X".  Sequencing and placing UI around it is a job
for the script that drives merge-tree.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 08/13] merge-tree: support including merge messages in output
  2022-02-02 21:30     ` Junio C Hamano
@ 2022-02-02 23:09       ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-02-02 23:09 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Johannes Sixt

On Wed, Feb 2, 2022 at 1:30 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > +By default, for a successful merge, the output from git-merge-tree is
> > +simply one line:
> > +
> > +     <OID of toplevel tree>
> > +
> > +Whereas for a conflicted merge, the output is by default of the form:
> >
> >       <OID of toplevel tree>
> > +     <Informational messages>
>
> Sounds useful.  This made me wonder, as the only shuffling of the
> output destination in the past few steps were to send the output to
> some "FILE *", how you send the findings you make while coming up
> with the result _after_ the result.  It turns out that the ORT
> machinery already buffers these findings in a strbuf per path, so
> there is no trouble doing so ;-)

:-)

> It still makes me wonder how the "send rename warnings to the
> standard output stream, instead of the standard error stream" change
> interacts with this change, though.  That needs to be done way
> before you finish computing the result, and it does not seem to be
> buffered in-core, like per-path conflict information messages.

Actually, that's also stashed away in detect_regular_renames() from merge-ort.c:

    if (diff_opts.needed_rename_limit > renames->needed_limit)
        renames->needed_limit = diff_opts.needed_rename_limit;

and then the call to diff_warn_rename_limit() is deferred until after
all the informational messages are printed.  Also, this series
modifies diff_warn_rename_limit() later to allow it to send that
message to a different stream.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info
  2022-02-02 21:32     ` Junio C Hamano
@ 2022-02-02 23:18       ` Elijah Newren
  2022-02-03  1:08         ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-02 23:18 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Johannes Sixt

On Wed, Feb 2, 2022 at 1:32 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
> >               merge_get_conflicted_files(&result, &conflicted_files);
> >               for (i = 0; i < conflicted_files.nr; i++) {
> >                       const char *name = conflicted_files.items[i].string;
> > -                     if (last && !strcmp(last, name))
> > +                     struct stage_info *c = conflicted_files.items[i].util;
> > +                     if (!o->exclude_modes_oids_stages)
> > +                             printf("%06o %s %d\t",
> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
> > +                     else if (last && !strcmp(last, name))
> >                               continue;
> >                       write_name_quoted_relative(
> >                               name, prefix, stdout, line_termination);
>
> OK.  The addition (and disabling of the deduping) is quite trivial.
> We do not even have to worry about line termination since the extra
> pieces of info are prepended to the pathname.  Nice.
>
> > @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >                           N_("do a trivial merge only"), 't'),
> >               OPT_BOOL(0, "messages", &o.show_messages,
> >                        N_("also show informational/conflict messages")),
> > +             OPT_BOOL_F('l', "exclude-modes-oids-stages",
> > +                        &o.exclude_modes_oids_stages,
> > +                        N_("list conflicted files without modes/oids/stages"),
> > +                        PARSE_OPT_NONEG),
>
> Why does "-l" give shorter output than without it?  "-l" strongly
> hints a longer output than without, at least to me.  Just wondering
> if this will not become a source of confusion to future scripting
> users.

Here's another example where I was struggling with naming.  Something
like ls-tree's `--name-only` would have been nice, but I was worried
it'd be confusing since it only affected the conflicted info section
and does not suppress the printing of the toplevel tree or the
informational messages sections.  And the name
--exclude-modes-oids-stages was long enough that I wanted a short flag
for it, and just used the first letter of the description ("list
conflicted files...").  I'm happy to change either the long or the
short name for this flag if anyone has suggestions.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info
  2022-02-02  7:34     ` [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info Elijah Newren via GitGitGadget
@ 2022-02-02 23:55       ` Ævar Arnfjörð Bjarmason
  2022-02-03  5:19         ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-02 23:55 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt, Elijah Newren


On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Much like `git merge` updates the index with information of the form
>     (mode, oid, stage, name)
> provide this output for conflicted files for merge-tree as well.
> Provide an --exclude-modes-oids-stages/-l option for users to exclude

This.

> +--exclude-oids-and-modes::

No longer matches this.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-02 22:01           ` Junio C Hamano
@ 2022-02-03  0:18             ` Elijah Newren
  2022-02-03 10:42               ` Johannes Altmanninger
  2022-02-21 18:55               ` Junio C Hamano
  0 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren @ 2022-02-03  0:18 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Johannes Sixt

On Wed, Feb 2, 2022 at 2:01 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Elijah Newren <newren@gmail.com> writes:
>
> > Yes, you are reading right.  I think the cherry-pick/rebase
> > replacement actually deserves a separate command from what merges
> > should use; replaying a sequence of commits just has a number of UI
> > differences and abilities that I think pull it in a different
> > direction.
>
> I completely disagree.  Each individual step in a sequence of
> replaying commits in order (or in reverse order) should be
> scriptable as a single merge-tree that takes "apply the change to go
> from A^ to A on X".  Sequencing and placing UI around it is a job
> for the script that drives merge-tree.

Adding such an ability to merge-tree would be trivial -- it basically
involves just two things: (1) accepting one extra argument, and (2)
calling merge_incore_nonrecursive() instead of
merge_incore_recursive().

However, I think forking a subprocess for every merge of a series of
commits is a completely unreasonable overhead, so even if we provide
such an option to merge-tree, I still want a separate plumbing-ish
tool that does non-worktree/non-index replaying of commits which is
not written as a driver of merge-tree.  That other tool should just
call merge_incore_nonrecursive() directly.  And such a tool, since it
should handle an arbitrary number of commits, should certainly be able
to handle just one commit.  From that angle, it feels like adding
another mode to merge-tree would just be a partial duplication of the
other tool.

However, if the other tool doesn't obviate the need for this
additional mode (perhaps it ends up being forced to be too
porcelain-ish insteading of plumbing-ish?), or folks really just want
another merge-tree mode, I'm happy to add one along with the tool I
submit later.  Does that sound reasonable to you, or is there
something you're still objecting to that I've missed?

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info
  2022-02-02 23:18       ` Elijah Newren
@ 2022-02-03  1:08         ` Ævar Arnfjörð Bjarmason
  2022-02-03  8:39           ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03  1:08 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Junio C Hamano, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt


On Wed, Feb 02 2022, Elijah Newren wrote:

> On Wed, Feb 2, 2022 at 1:32 PM Junio C Hamano <gitster@pobox.com> wrote:
>>
>> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>> > @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
>> >               merge_get_conflicted_files(&result, &conflicted_files);
>> >               for (i = 0; i < conflicted_files.nr; i++) {
>> >                       const char *name = conflicted_files.items[i].string;
>> > -                     if (last && !strcmp(last, name))
>> > +                     struct stage_info *c = conflicted_files.items[i].util;
>> > +                     if (!o->exclude_modes_oids_stages)
>> > +                             printf("%06o %s %d\t",
>> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
>> > +                     else if (last && !strcmp(last, name))
>> >                               continue;
>> >                       write_name_quoted_relative(
>> >                               name, prefix, stdout, line_termination);
>>
>> OK.  The addition (and disabling of the deduping) is quite trivial.
>> We do not even have to worry about line termination since the extra
>> pieces of info are prepended to the pathname.  Nice.
>>
>> > @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>> >                           N_("do a trivial merge only"), 't'),
>> >               OPT_BOOL(0, "messages", &o.show_messages,
>> >                        N_("also show informational/conflict messages")),
>> > +             OPT_BOOL_F('l', "exclude-modes-oids-stages",
>> > +                        &o.exclude_modes_oids_stages,
>> > +                        N_("list conflicted files without modes/oids/stages"),
>> > +                        PARSE_OPT_NONEG),
>>
>> Why does "-l" give shorter output than without it?  "-l" strongly
>> hints a longer output than without, at least to me.  Just wondering
>> if this will not become a source of confusion to future scripting
>> users.
>
> Here's another example where I was struggling with naming.  Something
> like ls-tree's `--name-only` would have been nice, but I was worried
> it'd be confusing since it only affected the conflicted info section
> and does not suppress the printing of the toplevel tree or the
> informational messages sections.  And the name
> --exclude-modes-oids-stages was long enough that I wanted a short flag
> for it, and just used the first letter of the description ("list
> conflicted files...").  I'm happy to change either the long or the
> short name for this flag if anyone has suggestions.

There's always sidestepping it by replacing it with a --format :)

Anyway, I'd mentioned that in an earlier review in
<220124.864k5tigto.gmgdl@evledraar.gmail.com>. FWIW here's an experiment
to do that that I polished up (mostly copied from the ls-tree WIP code
I'd written already).

I don't know if it will ever be useful, or if you think it's
worthwhile/simpler, but in either case I think in doing this I spotted
the following issues or otherwise noted inconsistencies in the pre-image:

   The docs say that "<stage> <path>" is SP-separated, but it's
   actually TAB-separated, the rest is SP-separated.

 * That you de-dupe --exclude-modes-oids-stages is a bit of a hidden feature,
   but argubly initiative. Should it by optional? In any case my formatting
   experiment makes it optional, since it then needs to be generalized to de-dupe
   after we've formatted.

 * Perhaps we should support --abbrev as ls-tree does? The below diff shows
   it's easy enough.

 * The dance you have with sed-ing out the hash in the tests could be made much
   easier with "sed 1d <out >actual" and --no-messages for some existing tests.

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 6a2ed475106..e906d1dc9bf 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -44,10 +44,9 @@ OPTIONS
 	newline.  Also begin the messages section with a NUL character
 	instead of a newline.  See OUTPUT below for more information.
 
---exclude-oids-and-modes::
-	Instead of writing a list of (mode, oid, stage, path) tuples
-	to output for conflicted files, just provide a list of
-	filenames with conflicts.
+--conflict-format::
+	Override the default "%(objectmode) %(objectname)
+	%(stage)%x09%(path)" format.
 
 --[no-]messages::
 	Write any informational messages such as "Auto-merging <path>"
@@ -89,13 +88,13 @@ Conflicted file info
 
 This is a sequence of lines with the format
 
-	<mode> <object> <stage> <filename>
+	%(objectmode) %(objectname) %(stage)%x09%(path)
 
 The filename will be quoted as explained for the configuration
-variable `core.quotePath` (see linkgit:git-config[1]).  However, if
-the `--exclude-oids-and-modes` option is passed, the mode, object, and
-stage will be omitted.  If `-z` is passed, the "lines" are terminated
-by a NUL character instead of a newline character.
+variable `core.quotePath` (see linkgit:git-config[1]).
+
+If `-z` is passed, the "lines" are terminated by a NUL character
+instead of a newline character.
 
 Informational messages
 ~~~~~~~~~~~~~~~~~~~~~~
diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 58c0ddc5a32..14fed95a8ce 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -395,9 +395,64 @@ struct merge_tree_options {
 	int mode;
 	int allow_unrelated_histories;
 	int show_messages;
-	int exclude_modes_oids_stages;
+	const char *conflict_format;
+	int unique_conflicts;
+	int abbrev;
 };
 
+struct expand_conflict_data {
+	const char *prefix;
+	struct string_list_item *item;
+	struct strbuf *scratch;
+	int abbrev;
+	struct strbuf *sb_tmp;
+};
+static size_t expand_conflict_format(struct strbuf *sb,
+				     const char *start,
+				     void *context)
+{
+	struct expand_conflict_data *data = context;
+	struct string_list_item *item = data->item;
+	struct stage_info *info = item->util;
+	const char *end;
+	const char *p;
+	size_t len;
+
+	len = strbuf_expand_literal_cb(sb, start, NULL);
+	if (len)
+		return len;
+
+	if (*start != '(')
+		die(_("bad format as of '%s'"), start);
+	end = strchr(start + 1, ')');
+	if (!end)
+		die(_("format element '%s' does not end in ')'"), start);
+	len = end - start + 1;
+
+	if (skip_prefix(start, "(objectmode)", &p)) {
+		strbuf_addf(sb, "%06o", info->mode);
+	} else if (skip_prefix(start, "(objectname)", &p)) {
+		strbuf_addstr(sb, find_unique_abbrev(&info->oid, data->abbrev));
+	} else if (skip_prefix(start, "(stage)", &p)) {
+		strbuf_addf(sb, "%d", info->stage);
+	} else if (skip_prefix(start, "(path)", &p)) {
+		const char *name = item->string;
+
+		if (data->prefix)
+			name = relative_path(name, data->prefix, data->scratch);
+		strbuf_addstr(sb, name);
+
+		strbuf_reset(data->sb_tmp);
+		/* The relative_path() function resets "scratch" */
+
+	} else {
+		unsigned int errlen = (unsigned long)len;
+		die(_("bad format specifier %%%.*s"), errlen, start);
+	}
+
+	return len;
+}
+
 static int real_merge(struct merge_tree_options *o,
 		      const char *branch1, const char *branch2,
 		      const char *prefix)
@@ -446,23 +501,43 @@ static int real_merge(struct merge_tree_options *o,
 	puts(oid_to_hex(&result.tree->object.oid));
 	if (!result.clean) {
 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
-		const char *last = NULL;
-		int i;
+		struct string_list_item *item;
+		char *last = NULL;
+		struct strbuf sb = STRBUF_INIT;
+		struct strbuf tmp = STRBUF_INIT;
 
 		merge_get_conflicted_files(&result, &conflicted_files);
-		for (i = 0; i < conflicted_files.nr; i++) {
-			const char *name = conflicted_files.items[i].string;
-			struct stage_info *c = conflicted_files.items[i].util;
-			if (!o->exclude_modes_oids_stages)
-				printf("%06o %s %d\t",
-				       c->mode, oid_to_hex(&c->oid), c->stage);
-			else if (last && !strcmp(last, name))
+		for_each_string_list_item(item, &conflicted_files) {
+			struct expand_conflict_data ctx = {
+				.prefix = prefix,
+				.item = item,
+				.abbrev = o->abbrev,
+				.scratch = &sb,
+				.sb_tmp = &tmp,
+			};
+
+			strbuf_expand(&sb, o->conflict_format, expand_conflict_format, &ctx);
+			strbuf_addch(&sb, line_termination);
+
+			if (o->unique_conflicts && last && !strcmp(last, sb.buf)) {
+				free(last);
+				last = strbuf_detach(&sb, NULL);
 				continue;
-			write_name_quoted_relative(
-				name, prefix, stdout, line_termination);
-			last = name;
+			}
+
+			fwrite(sb.buf, sb.len, 1, stdout);
+
+			if (o->unique_conflicts) {
+				free(last);
+				last = strbuf_detach(&sb, NULL);
+			} else {
+				strbuf_reset(&sb);
+			}
 		}
 		string_list_clear(&conflicted_files, 1);
+		strbuf_release(&sb);
+		strbuf_release(&tmp);
+		free(last);
 	}
 	if (o->show_messages) {
 		putchar(line_termination);
@@ -474,7 +549,11 @@ static int real_merge(struct merge_tree_options *o,
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { .show_messages = -1 };
+	struct merge_tree_options o = {
+		.show_messages = -1,
+		.conflict_format = "%(objectmode) %(objectname) %(stage)%x09%(path)",
+		.unique_conflicts = 1,
+	};
 	int expected_remaining_argc;
 	int original_argc;
 
@@ -493,14 +572,15 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			 N_("also show informational/conflict messages")),
 		OPT_SET_INT('z', NULL, &line_termination,
 			    N_("separate paths with the NUL character"), '\0'),
-		OPT_BOOL_F('l', "exclude-modes-oids-stages",
-			   &o.exclude_modes_oids_stages,
-			   N_("list conflicted files without modes/oids/stages"),
-			   PARSE_OPT_NONEG),
+		OPT_STRING(0, "conflict-format", &o.conflict_format, N_("format"),
+			   N_("specify a custom format to use for conflicted files")),
+		OPT_BOOL(0, "unique-conflicts", &o.unique_conflicts,
+			 N_("omit duplicate --conflict-format lines")),
 		OPT_BOOL_F(0, "allow-unrelated-histories",
 			   &o.allow_unrelated_histories,
 			   N_("allow merging unrelated histories"),
 			   PARSE_OPT_NONEG),
+		OPT__ABBREV(&o.abbrev),
 		OPT_END()
 	};
 
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 4de089d976d..e6354b2d284 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -93,7 +93,7 @@ test_expect_success 'Barf on too many arguments' '
 '
 
 test_expect_success 'test conflict notices and such' '
-	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	# Expected results:
@@ -115,8 +115,35 @@ test_expect_success 'test conflict notices and such' '
 	test_cmp expect actual
 '
 
+test_expect_success 'merge-tree --unique-conflicts is the default' '
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out &&
+	sed 1d <out >actual &&
+	cat >expect <<-\EOF &&
+	greeting
+	whatever~side1
+	EOF
+	test_cmp expect actual &&
+
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out2 &&
+	sed 1d <out2 >actual2 &&
+	test_cmp actual actual2
+'
+
+test_expect_success 'merge-tree --no-unique-conflicts' '
+	test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-unique-conflicts --no-messages side1 side2 >out &&
+	sed 1d <out >actual &&
+	cat >expect <<-\EOF &&
+	greeting
+	greeting
+	greeting
+	whatever~side1
+	whatever~side1
+	EOF
+	test_cmp expect actual
+'
+
 test_expect_success 'Just the conflicted files without the messages' '
-	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
+	test_expect_code 1 git merge-tree --write-tree --no-messages --conflict-format="%(path)" side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
 
 	test_write_lines HASH greeting whatever~side1 >expect &&

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream
  2022-02-02  7:34     ` [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream Elijah Newren via GitGitGadget
@ 2022-02-03  1:48       ` Ævar Arnfjörð Bjarmason
  2022-02-03  9:12         ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03  1:48 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt, Elijah Newren


On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> This modifies the new display_update_messages() function to allow
> printing to somewhere other than stdout.  It also consolidates the
> location of the diff_warn_rename_limit() message with the rest of the
> CONFLICT and other update messages to all go to the same stream.
>
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  merge-ort.c | 9 +++++----
>  merge-ort.h | 3 ++-
>  2 files changed, 7 insertions(+), 5 deletions(-)
>
> diff --git a/merge-ort.c b/merge-ort.c
> index 82d2faf5bf9..d28d1721d14 100644
> --- a/merge-ort.c
> +++ b/merge-ort.c
> @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
>  }
>  
>  void merge_display_update_messages(struct merge_options *opt,
> -				   struct merge_result *result)
> +				   struct merge_result *result,
> +				   FILE *stream)
>  {
>  	struct merge_options_internal *opti = result->priv;
>  	struct hashmap_iter iter;
> @@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
>  	for (i = 0; i < olist.nr; ++i) {
>  		struct strbuf *sb = olist.items[i].util;
>  
> -		printf("%s", sb->buf);
> +		strbuf_write(sb, stream);
>  	}
>  	string_list_clear(&olist, 0);
>  
>  	/* Also include needed rename limit adjustment now */
>  	diff_warn_rename_limit("merge.renamelimit",
> -			       opti->renames.needed_limit, 0, stderr);
> +			       opti->renames.needed_limit, 0, stream);

At the tip of this series I tried to s/stream/stderr/g this, and
t4301-merge-tree-write-tree.sh passes, doesn't this warning_fp() special
behavior need a test somewhere?

I assumed that warning_fp() would be using vreportf() in usage.c, but
it's not, it's just falling back to the equivalent of fprintf(out, ...),
no? I don't really see why 05/15 and parts of 06/15 & this are needed
over a much simpler simple helper macro like the below. applied on top
of this series.

I would get it if the point was to actually use the full usage.c
machinery, but we're just either calling warning(), or printing a
formatted string to a file FILE *. There's no need to go through usage.c
for that, and adding an API to it that behaves like this new
warning_fp() is really confusing.

I.e. an API in usage.c that allowed warning to a given FD would be
trying to replace the "2" in the write_in_full() call in vreportf(), I
would think.

diff --git a/diff.c b/diff.c
index 28368110147..4cf67e93dea 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
+#define warning_fp(out, fmt, ...) do { \
+	if (out == stderr) \
+		warning(fmt, __VA_ARGS__); \
+	else \
+		fprintf(out, fmt, __VA_ARGS__); \
+} while (0)
+
 void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
 			    FILE *out)
 {
 	fflush(stdout);
 	if (degraded_cc)
-		warning_fp(out, _(degrade_cc_to_c_warning));
+		warning_fp(out, _(degrade_cc_to_c_warning), NULL);
 	else if (needed)
-		warning_fp(out, _(rename_limit_warning));
+		warning_fp(out, _(rename_limit_warning), NULL);
 	else
 		return;
 
diff --git a/git-compat-util.h b/git-compat-util.h
index 64ba60e5c71..d70ce142861 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
-void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index 0bfd2c603c0..c7d233b0de9 100644
--- a/usage.c
+++ b/usage.c
@@ -253,20 +253,6 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
-void warning_fp(FILE *out, const char *warn, ...)
-{
-	va_list params;
-
-	va_start(params, warn);
-	if (out == stderr)
-		warn_routine(warn, params);
-	else {
-		vfprintf(out, warn, params);
-		fputc('\n', out);
-	}
-	va_end(params);
-}
-
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-02  7:34     ` [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
@ 2022-02-03  2:05       ` Ævar Arnfjörð Bjarmason
  2022-02-03  9:04         ` Elijah Newren
  2022-02-07 22:41       ` Emily Shaffer
  1 sibling, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03  2:05 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt, Elijah Newren


On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:

> From: Elijah Newren <newren@gmail.com>
>
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.

Maybe that ship has sailed, but just my 0.02: I thought this whole thing
was much less confusing with your initial merge-tree-ort proposal at
https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
I.e. the end-state of merge-tree.c is that you end up reading largely
unrelated code (various static functions only used by one side or
another).

But maybe that's all water under the bridge etc, however...

>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_CMDMODE(0, "write-tree", &o.mode,
> +			    N_("do a real merge instead of a trivial merge"),
> +			    'w'),
> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
> +			    N_("do a trivial merge only"), 't'),
> +		OPT_END()
> +	};
> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +	if (o.mode) {
> +		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.mode = (argc == 2 ? 'w' : 't');
> +	}

Do we really need to make this interface more special-casey by
auto-guessing based on argc what argument you want? I.e. instead of
usage like:

	N_("git merge-tree [--write-tree] <branch1> <branch2>"),
	N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),

Wouldn't it be simpler to just have the equivalent of:

	# old
        git merge-tree ...
        # new
        git merge-tree --new-thing ...

And not have to look at ... to figure out if we're dispatching to the
new or old thing.


^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 12/15] merge-tree: provide easy access to `ls-files -u` style info
  2022-02-02 23:55       ` Ævar Arnfjörð Bjarmason
@ 2022-02-03  5:19         ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-02-03  5:19 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Wed, Feb 2, 2022 at 5:07 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Much like `git merge` updates the index with information of the form
> >     (mode, oid, stage, name)
> > provide this output for conflicted files for merge-tree as well.
> > Provide an --exclude-modes-oids-stages/-l option for users to exclude
>
> This.
>
> > +--exclude-oids-and-modes::
>
> No longer matches this.

Thanks; good catch.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 11/13] merge-tree: provide easy access to `ls-files -u` style info
  2022-02-03  1:08         ` Ævar Arnfjörð Bjarmason
@ 2022-02-03  8:39           ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-02-03  8:39 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Junio C Hamano, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Wed, Feb 2, 2022 at 5:22 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren wrote:
>
> > On Wed, Feb 2, 2022 at 1:32 PM Junio C Hamano <gitster@pobox.com> wrote:
> >>
> >> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
> >>
> >> > @@ -450,7 +451,11 @@ static int real_merge(struct merge_tree_options *o,
> >> >               merge_get_conflicted_files(&result, &conflicted_files);
> >> >               for (i = 0; i < conflicted_files.nr; i++) {
> >> >                       const char *name = conflicted_files.items[i].string;
> >> > -                     if (last && !strcmp(last, name))
> >> > +                     struct stage_info *c = conflicted_files.items[i].util;
> >> > +                     if (!o->exclude_modes_oids_stages)
> >> > +                             printf("%06o %s %d\t",
> >> > +                                    c->mode, oid_to_hex(&c->oid), c->stage);
> >> > +                     else if (last && !strcmp(last, name))
> >> >                               continue;
> >> >                       write_name_quoted_relative(
> >> >                               name, prefix, stdout, line_termination);
> >>
> >> OK.  The addition (and disabling of the deduping) is quite trivial.
> >> We do not even have to worry about line termination since the extra
> >> pieces of info are prepended to the pathname.  Nice.
> >>
> >> > @@ -485,6 +490,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >> >                           N_("do a trivial merge only"), 't'),
> >> >               OPT_BOOL(0, "messages", &o.show_messages,
> >> >                        N_("also show informational/conflict messages")),
> >> > +             OPT_BOOL_F('l', "exclude-modes-oids-stages",
> >> > +                        &o.exclude_modes_oids_stages,
> >> > +                        N_("list conflicted files without modes/oids/stages"),
> >> > +                        PARSE_OPT_NONEG),
> >>
> >> Why does "-l" give shorter output than without it?  "-l" strongly
> >> hints a longer output than without, at least to me.  Just wondering
> >> if this will not become a source of confusion to future scripting
> >> users.
> >
> > Here's another example where I was struggling with naming.  Something
> > like ls-tree's `--name-only` would have been nice, but I was worried
> > it'd be confusing since it only affected the conflicted info section
> > and does not suppress the printing of the toplevel tree or the
> > informational messages sections.  And the name
> > --exclude-modes-oids-stages was long enough that I wanted a short flag
> > for it, and just used the first letter of the description ("list
> > conflicted files...").  I'm happy to change either the long or the
> > short name for this flag if anyone has suggestions.
>
> There's always sidestepping it by replacing it with a --format :)

Another solution that occurred to me, and I was _really_ close to
doing it for v3, was to just flat drop this patch entirely and not
include any such option.  But...

  * "Which files had conflicts?" seems like such an obvious question
  * I've used `git ls-files -u | awk {print\$4} | uniq` a lot in the
past after `git merge` (Or `git rebase`) to get this info (yeah, it
turns out `git diff --name-only --diff-filter=U` is 4 fewer
characters)
  * "display the list of files where conflicts were present in the web
UI" was listed as an early usecase[1]

[1] https://lore.kernel.org/git/YYlqpuzv+bmZaFzz@nand.local/

So it seemed like making that question easy to answer was worthwhile.

> Anyway, I'd mentioned that in an earlier review in
> <220124.864k5tigto.gmgdl@evledraar.gmail.com>. FWIW here's an experiment
> to do that that I polished up (mostly copied from the ls-tree WIP code
> I'd written already).
>
> I don't know if it will ever be useful, or if you think it's
> worthwhile/simpler, but in either case I think in doing this I spotted
> the following issues or otherwise noted inconsistencies in the pre-image:
>
>    The docs say that "<stage> <path>" is SP-separated, but it's
>    actually TAB-separated, the rest is SP-separated.

Yeah, good catch.  However, it doesn't actually say they are
SP-separated; it's ambiguous about the spacing.  Which probably isn't
a good thing, but it was kind of copied from the ls-files manual:

"""
       git ls-files just outputs the filenames unless --stage is specified in
       which case it outputs:

           [<tag> ]<mode> <object> <stage> <file>
"""

(which also uses a tab between <stage> and <file> and a space
otherwise, but the output above may lead you to believe otherwise.)

>  * That you de-dupe --exclude-modes-oids-stages is a bit of a hidden feature,
>    but argubly initiative. Should it by optional? In any case my formatting
>    experiment makes it optional, since it then needs to be generalized to de-dupe
>    after we've formatted.

I think without de-duping the flag isn't helpful enough to bother
implementing.  Requiring two flags also seems painful, given the
common case scenario.

I hope I'm not coming across as dismissive.  I think eventually adding
a --format and --dedupe (the combination of which might be implied by
whatever flag is used now) might be useful additions.  Maybe --abbrev
too...eventually.  But I'm worried that it's distracting from focusing
on usecases.  In particular, I'm worried it leads to "well, script
writers technically can get what they want because we provided
everything" rather than focusing on making the most common things easy
to get, and then extending the command for flexibility as needed
later.

I'd really rather that early versions _just_ focus on actual usecases
as far as UI is concerned (and thus I was really happy to see Dscho
and Taylor concentrate on that side; I think Christian might have been
talking about that angle some but it was hard to differentiate from
the "merge-tree on steroids" spitballing).  While I want to be careful
to avoid preventing UI flexibility, I think building it in from the
beginning tends to lead to a design that is less usable.  (e.g. the
possible loss of de-duping that would naturally have arisen from
looking at things from the other angle.)  It's just a bias I have.

>  * Perhaps we should support --abbrev as ls-tree does? The below diff shows
>    it's easy enough.

This one is less problematic to me, but I'd still rather that the UI
side of things focused on the usecases for early versions.

>  * The dance you have with sed-ing out the hash in the tests could be made much
>    easier with "sed 1d <out >actual" and --no-messages for some existing tests.

Ignoring the first line is semantically different than verifying it
looks like a hash.  It also only works on the first line, and hashes
appear in multiple places, so you'd need a variety of different sed
commands for different parts of the output, which doesn't seem any
easier at all to me; I think using the same replacement everywhere is
simpler.  But perhaps I should turn it into a shell function that I
use in each case.

> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 6a2ed475106..e906d1dc9bf 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -44,10 +44,9 @@ OPTIONS
>         newline.  Also begin the messages section with a NUL character
>         instead of a newline.  See OUTPUT below for more information.
>
> ---exclude-oids-and-modes::
> -       Instead of writing a list of (mode, oid, stage, path) tuples
> -       to output for conflicted files, just provide a list of
> -       filenames with conflicts.
> +--conflict-format::
> +       Override the default "%(objectmode) %(objectname)
> +       %(stage)%x09%(path)" format.
>
>  --[no-]messages::
>         Write any informational messages such as "Auto-merging <path>"
> @@ -89,13 +88,13 @@ Conflicted file info
>
>  This is a sequence of lines with the format
>
> -       <mode> <object> <stage> <filename>
> +       %(objectmode) %(objectname) %(stage)%x09%(path)
>
>  The filename will be quoted as explained for the configuration
> -variable `core.quotePath` (see linkgit:git-config[1]).  However, if
> -the `--exclude-oids-and-modes` option is passed, the mode, object, and
> -stage will be omitted.  If `-z` is passed, the "lines" are terminated
> -by a NUL character instead of a newline character.
> +variable `core.quotePath` (see linkgit:git-config[1]).
> +
> +If `-z` is passed, the "lines" are terminated by a NUL character
> +instead of a newline character.
>
>  Informational messages
>  ~~~~~~~~~~~~~~~~~~~~~~
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 58c0ddc5a32..14fed95a8ce 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -395,9 +395,64 @@ struct merge_tree_options {
>         int mode;
>         int allow_unrelated_histories;
>         int show_messages;
> -       int exclude_modes_oids_stages;
> +       const char *conflict_format;
> +       int unique_conflicts;
> +       int abbrev;
>  };
>
> +struct expand_conflict_data {
> +       const char *prefix;
> +       struct string_list_item *item;
> +       struct strbuf *scratch;
> +       int abbrev;
> +       struct strbuf *sb_tmp;
> +};
> +static size_t expand_conflict_format(struct strbuf *sb,
> +                                    const char *start,
> +                                    void *context)
> +{
> +       struct expand_conflict_data *data = context;
> +       struct string_list_item *item = data->item;
> +       struct stage_info *info = item->util;
> +       const char *end;
> +       const char *p;
> +       size_t len;
> +
> +       len = strbuf_expand_literal_cb(sb, start, NULL);
> +       if (len)
> +               return len;
> +
> +       if (*start != '(')
> +               die(_("bad format as of '%s'"), start);
> +       end = strchr(start + 1, ')');
> +       if (!end)
> +               die(_("format element '%s' does not end in ')'"), start);
> +       len = end - start + 1;
> +
> +       if (skip_prefix(start, "(objectmode)", &p)) {
> +               strbuf_addf(sb, "%06o", info->mode);
> +       } else if (skip_prefix(start, "(objectname)", &p)) {
> +               strbuf_addstr(sb, find_unique_abbrev(&info->oid, data->abbrev));
> +       } else if (skip_prefix(start, "(stage)", &p)) {
> +               strbuf_addf(sb, "%d", info->stage);
> +       } else if (skip_prefix(start, "(path)", &p)) {
> +               const char *name = item->string;
> +
> +               if (data->prefix)
> +                       name = relative_path(name, data->prefix, data->scratch);
> +               strbuf_addstr(sb, name);
> +
> +               strbuf_reset(data->sb_tmp);
> +               /* The relative_path() function resets "scratch" */
> +
> +       } else {
> +               unsigned int errlen = (unsigned long)len;
> +               die(_("bad format specifier %%%.*s"), errlen, start);
> +       }
> +
> +       return len;
> +}
> +
>  static int real_merge(struct merge_tree_options *o,
>                       const char *branch1, const char *branch2,
>                       const char *prefix)
> @@ -446,23 +501,43 @@ static int real_merge(struct merge_tree_options *o,
>         puts(oid_to_hex(&result.tree->object.oid));
>         if (!result.clean) {
>                 struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
> -               const char *last = NULL;
> -               int i;
> +               struct string_list_item *item;
> +               char *last = NULL;
> +               struct strbuf sb = STRBUF_INIT;
> +               struct strbuf tmp = STRBUF_INIT;
>
>                 merge_get_conflicted_files(&result, &conflicted_files);
> -               for (i = 0; i < conflicted_files.nr; i++) {
> -                       const char *name = conflicted_files.items[i].string;
> -                       struct stage_info *c = conflicted_files.items[i].util;
> -                       if (!o->exclude_modes_oids_stages)
> -                               printf("%06o %s %d\t",
> -                                      c->mode, oid_to_hex(&c->oid), c->stage);
> -                       else if (last && !strcmp(last, name))
> +               for_each_string_list_item(item, &conflicted_files) {
> +                       struct expand_conflict_data ctx = {
> +                               .prefix = prefix,
> +                               .item = item,
> +                               .abbrev = o->abbrev,
> +                               .scratch = &sb,
> +                               .sb_tmp = &tmp,
> +                       };
> +
> +                       strbuf_expand(&sb, o->conflict_format, expand_conflict_format, &ctx);
> +                       strbuf_addch(&sb, line_termination);
> +
> +                       if (o->unique_conflicts && last && !strcmp(last, sb.buf)) {
> +                               free(last);
> +                               last = strbuf_detach(&sb, NULL);
>                                 continue;
> -                       write_name_quoted_relative(
> -                               name, prefix, stdout, line_termination);
> -                       last = name;
> +                       }
> +
> +                       fwrite(sb.buf, sb.len, 1, stdout);
> +
> +                       if (o->unique_conflicts) {
> +                               free(last);
> +                               last = strbuf_detach(&sb, NULL);
> +                       } else {
> +                               strbuf_reset(&sb);
> +                       }
>                 }
>                 string_list_clear(&conflicted_files, 1);
> +               strbuf_release(&sb);
> +               strbuf_release(&tmp);
> +               free(last);
>         }
>         if (o->show_messages) {
>                 putchar(line_termination);
> @@ -474,7 +549,11 @@ static int real_merge(struct merge_tree_options *o,
>
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -       struct merge_tree_options o = { .show_messages = -1 };
> +       struct merge_tree_options o = {
> +               .show_messages = -1,
> +               .conflict_format = "%(objectmode) %(objectname) %(stage)%x09%(path)",
> +               .unique_conflicts = 1,
> +       };
>         int expected_remaining_argc;
>         int original_argc;
>
> @@ -493,14 +572,15 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>                          N_("also show informational/conflict messages")),
>                 OPT_SET_INT('z', NULL, &line_termination,
>                             N_("separate paths with the NUL character"), '\0'),
> -               OPT_BOOL_F('l', "exclude-modes-oids-stages",
> -                          &o.exclude_modes_oids_stages,
> -                          N_("list conflicted files without modes/oids/stages"),
> -                          PARSE_OPT_NONEG),
> +               OPT_STRING(0, "conflict-format", &o.conflict_format, N_("format"),
> +                          N_("specify a custom format to use for conflicted files")),
> +               OPT_BOOL(0, "unique-conflicts", &o.unique_conflicts,
> +                        N_("omit duplicate --conflict-format lines")),

The latter of which you didn't include in the manual?  Also,
unique_conflicts seems like something that is trivial to understand
from the coding perspective, but probably require quite a bit more
explanation from the manual.  For example, if objectname is included
in the format, unique-conflicts is essentially a no-op.  And that's
the default...so, you'd probably have to spend time in the manual
explaining under what circumstances it's useful.  I'm also not sure if
a user who wanted (mode, path) would want unique_conflicts to default
to 1; it may be something only meaningful in the particular case of
"just give me conflicted filenames".

>                 OPT_BOOL_F(0, "allow-unrelated-histories",
>                            &o.allow_unrelated_histories,
>                            N_("allow merging unrelated histories"),
>                            PARSE_OPT_NONEG),
> +               OPT__ABBREV(&o.abbrev),
>                 OPT_END()
>         };
>
> diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
> index 4de089d976d..e6354b2d284 100755
> --- a/t/t4301-merge-tree-write-tree.sh
> +++ b/t/t4301-merge-tree-write-tree.sh
> @@ -93,7 +93,7 @@ test_expect_success 'Barf on too many arguments' '
>  '
>
>  test_expect_success 'test conflict notices and such' '
> -       test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" side1 side2 >out &&
>         sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
>
>         # Expected results:
> @@ -115,8 +115,35 @@ test_expect_success 'test conflict notices and such' '
>         test_cmp expect actual
>  '
>
> +test_expect_success 'merge-tree --unique-conflicts is the default' '
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out &&
> +       sed 1d <out >actual &&
> +       cat >expect <<-\EOF &&
> +       greeting
> +       whatever~side1
> +       EOF
> +       test_cmp expect actual &&
> +
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-messages side1 side2 >out2 &&
> +       sed 1d <out2 >actual2 &&
> +       test_cmp actual actual2
> +'
> +
> +test_expect_success 'merge-tree --no-unique-conflicts' '
> +       test_expect_code 1 git merge-tree --write-tree --conflict-format="%(path)" --no-unique-conflicts --no-messages side1 side2 >out &&
> +       sed 1d <out >actual &&
> +       cat >expect <<-\EOF &&
> +       greeting
> +       greeting
> +       greeting
> +       whatever~side1
> +       whatever~side1
> +       EOF
> +       test_cmp expect actual
> +'
> +
>  test_expect_success 'Just the conflicted files without the messages' '
> -       test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
> +       test_expect_code 1 git merge-tree --write-tree --no-messages --conflict-format="%(path)" side1 side2 >out &&
>         sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
>
>         test_write_lines HASH greeting whatever~side1 >expect &&

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-03  2:05       ` Ævar Arnfjörð Bjarmason
@ 2022-02-03  9:04         ` Elijah Newren
  2022-02-03  9:22           ` Elijah Newren
  2022-02-03 10:26           ` Ævar Arnfjörð Bjarmason
  0 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren @ 2022-02-03  9:04 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > Let merge-tree accept a `--write-tree` parameter for choosing real
> > merges instead of trivial merges, and accept an optional
> > `--trivial-merge` option to get the traditional behavior.  Note that
> > these accept different numbers of arguments, though, so these names
> > need not actually be used.
>
> Maybe that ship has sailed, but just my 0.02: I thought this whole thing
> was much less confusing with your initial merge-tree-ort proposal at
> https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
> I.e. the end-state of merge-tree.c is that you end up reading largely
> unrelated code (various static functions only used by one side or
> another).

Christian's merge-tree-ort proposal?

> But maybe that's all water under the bridge etc, however...
>
> >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> >  {
> > -     if (argc != 4)
> > -             usage(merge_tree_usage);
> > -     return trivial_merge(argc, argv);
> > +     struct merge_tree_options o = { 0 };
> > +     int expected_remaining_argc;
> > +
> > +     const char * const merge_tree_usage[] = {
> > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> > +             NULL
> > +     };
> > +     struct option mt_options[] = {
> > +             OPT_CMDMODE(0, "write-tree", &o.mode,
> > +                         N_("do a real merge instead of a trivial merge"),
> > +                         'w'),
> > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
> > +                         N_("do a trivial merge only"), 't'),
> > +             OPT_END()
> > +     };
> > +
> > +     /* Parse arguments */
> > +     argc = parse_options(argc, argv, prefix, mt_options,
> > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> > +     if (o.mode) {
> > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> > +             if (argc != expected_remaining_argc)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +     } else {
> > +             if (argc < 2 || argc > 3)
> > +                     usage_with_options(merge_tree_usage, mt_options);
> > +             o.mode = (argc == 2 ? 'w' : 't');
> > +     }
>
> Do we really need to make this interface more special-casey by
> auto-guessing based on argc what argument you want? I.e. instead of
> usage like:
>
>         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>
> Wouldn't it be simpler to just have the equivalent of:
>
>         # old
>         git merge-tree ...
>         # new
>         git merge-tree --new-thing ...
>
> And not have to look at ... to figure out if we're dispatching to the
> new or old thing.

You seem to be focusing on code simplicity?  Sure, that'd be simpler
code, it'd just be a less useful feature.

I think passing --write-tree all the time would be an annoyance.  I
don't see why anyone would ever use the other mode.  However, for as
long as both exist in the manual, it makes the manual easier to
explain to users, and example testcases more self-documenting by
having the flag there.  That's the sole purpose of the flag.

I'm never going to actually use it when I invoke it from the command
line.  And I suspect most would leave it off.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream
  2022-02-03  1:48       ` Ævar Arnfjörð Bjarmason
@ 2022-02-03  9:12         ` Elijah Newren
  2022-02-03 10:01           ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-03  9:12 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > This modifies the new display_update_messages() function to allow
> > printing to somewhere other than stdout.  It also consolidates the
> > location of the diff_warn_rename_limit() message with the rest of the
> > CONFLICT and other update messages to all go to the same stream.
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  merge-ort.c | 9 +++++----
> >  merge-ort.h | 3 ++-
> >  2 files changed, 7 insertions(+), 5 deletions(-)
> >
> > diff --git a/merge-ort.c b/merge-ort.c
> > index 82d2faf5bf9..d28d1721d14 100644
> > --- a/merge-ort.c
> > +++ b/merge-ort.c
> > @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
> >  }
> >
> >  void merge_display_update_messages(struct merge_options *opt,
> > -                                struct merge_result *result)
> > +                                struct merge_result *result,
> > +                                FILE *stream)
> >  {
> >       struct merge_options_internal *opti = result->priv;
> >       struct hashmap_iter iter;
> > @@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
> >       for (i = 0; i < olist.nr; ++i) {
> >               struct strbuf *sb = olist.items[i].util;
> >
> > -             printf("%s", sb->buf);
> > +             strbuf_write(sb, stream);
> >       }
> >       string_list_clear(&olist, 0);
> >
> >       /* Also include needed rename limit adjustment now */
> >       diff_warn_rename_limit("merge.renamelimit",
> > -                            opti->renames.needed_limit, 0, stderr);
> > +                            opti->renames.needed_limit, 0, stream);
>
> At the tip of this series I tried to s/stream/stderr/g this, and
> t4301-merge-tree-write-tree.sh passes, doesn't this warning_fp() special
> behavior need a test somewhere?

That's a fair point, but...this gets back to my cover letter comments
about patches 5, 6, and 8.  They implement a code feature that seems
useful in general...but which Dscho and Christian didn't like using in
this particular command; they just wanted all output on stdout.

So, it's hard to add a test, because there's no code anywhere that
exercises it in this series anymore.  I originally wanted this feature
in my remerge-diff series, but the idea of conflict headers made me
punt it to this series.  I wanted it for this series, but Dscho and
Christian didn't.  I could have punted again, but decided the
underlying want kept coming up and decided to not excise it --
especially since Dscho was helping improve it.  And Junio commented
that he liked the idea[1].

[1] https://lore.kernel.org/git/xmqqh79hx8g1.fsf@gitster.g/

But yeah, it does leave it feeling slightly odd that we implemented a
feature that nothing is currently using.  Maybe these 3 should be
split off into their own series?  Still wouldn't have a test yet,
though.

> I assumed that warning_fp() would be using vreportf() in usage.c, but
> it's not, it's just falling back to the equivalent of fprintf(out, ...),
> no? I don't really see why 05/15 and parts of 06/15 & this are needed
> over a much simpler simple helper macro like the below. applied on top
> of this series.

That macro is simple?  I thought I basically understood Dscho's code,
but looking at what you did with diff_warn_rename_limit(), I think I'm
lost.

> I would get it if the point was to actually use the full usage.c
> machinery, but we're just either calling warning(), or printing a
> formatted string to a file FILE *. There's no need to go through usage.c
> for that, and adding an API to it that behaves like this new
> warning_fp() is really confusing.

Because the formatted string being printed to the file won't have the
same "warning: " prefix that is normally added to stuff in usage?

That's a fair point; that does have a bit of a consistency problem.
And I'd rather the messages were consistent regardless of where they
are printed.

> I.e. an API in usage.c that allowed warning to a given FD would be
> trying to replace the "2" in the write_in_full() call in vreportf(), I
> would think.

Hmm, makes sense.

> diff --git a/diff.c b/diff.c
> index 28368110147..4cf67e93dea 100644
> --- a/diff.c
> +++ b/diff.c
> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
>  N_("you may want to set your %s variable to at least "
>     "%d and retry the command.");
>
> +#define warning_fp(out, fmt, ...) do { \
> +       if (out == stderr) \
> +               warning(fmt, __VA_ARGS__); \
> +       else \
> +               fprintf(out, fmt, __VA_ARGS__); \
> +} while (0)
> +
>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
>                             FILE *out)
>  {
>         fflush(stdout);
>         if (degraded_cc)
> -               warning_fp(out, _(degrade_cc_to_c_warning));
> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
>         else if (needed)
> -               warning_fp(out, _(rename_limit_warning));
> +               warning_fp(out, _(rename_limit_warning), NULL);

Why do the only callers have a NULL parameter here?  Is this one of
those va_list/va_args things I never bothered to properly learn?

>         else
>                 return;
>
> diff --git a/git-compat-util.h b/git-compat-util.h
> index 64ba60e5c71..d70ce142861 100644
> --- a/git-compat-util.h
> +++ b/git-compat-util.h
> @@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
>  int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
>  void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
>  void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
> -void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
>
>  #ifndef NO_OPENSSL
>  #ifdef APPLE_COMMON_CRYPTO
> diff --git a/usage.c b/usage.c
> index 0bfd2c603c0..c7d233b0de9 100644
> --- a/usage.c
> +++ b/usage.c
> @@ -253,20 +253,6 @@ void warning(const char *warn, ...)
>         va_end(params);
>  }
>
> -void warning_fp(FILE *out, const char *warn, ...)
> -{
> -       va_list params;
> -
> -       va_start(params, warn);
> -       if (out == stderr)
> -               warn_routine(warn, params);
> -       else {
> -               vfprintf(out, warn, params);
> -               fputc('\n', out);
> -       }
> -       va_end(params);
> -}
> -
>  /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
>  int BUG_exit_code;

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-03  9:04         ` Elijah Newren
@ 2022-02-03  9:22           ` Elijah Newren
  2022-02-03  9:45             ` Ævar Arnfjörð Bjarmason
  2022-02-03 10:26           ` Ævar Arnfjörð Bjarmason
  1 sibling, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-03  9:22 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Thu, Feb 3, 2022 at 1:04 AM Elijah Newren <newren@gmail.com> wrote:
>
> On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >
> > On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
> >
> > > From: Elijah Newren <newren@gmail.com>
> > >
> > > Let merge-tree accept a `--write-tree` parameter for choosing real
> > > merges instead of trivial merges, and accept an optional
> > > `--trivial-merge` option to get the traditional behavior.  Note that
> > > these accept different numbers of arguments, though, so these names
> > > need not actually be used.
> >
> > Maybe that ship has sailed, but just my 0.02: I thought this whole thing
> > was much less confusing with your initial merge-tree-ort proposal at
> > https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
> > I.e. the end-state of merge-tree.c is that you end up reading largely
> > unrelated code (various static functions only used by one side or
> > another).
>
> Christian's merge-tree-ort proposal?
>
> > But maybe that's all water under the bridge etc, however...
> >
> > >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
> > >  {
> > > -     if (argc != 4)
> > > -             usage(merge_tree_usage);
> > > -     return trivial_merge(argc, argv);
> > > +     struct merge_tree_options o = { 0 };
> > > +     int expected_remaining_argc;
> > > +
> > > +     const char * const merge_tree_usage[] = {
> > > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> > > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> > > +             NULL
> > > +     };
> > > +     struct option mt_options[] = {
> > > +             OPT_CMDMODE(0, "write-tree", &o.mode,
> > > +                         N_("do a real merge instead of a trivial merge"),
> > > +                         'w'),
> > > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
> > > +                         N_("do a trivial merge only"), 't'),
> > > +             OPT_END()
> > > +     };
> > > +
> > > +     /* Parse arguments */
> > > +     argc = parse_options(argc, argv, prefix, mt_options,
> > > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> > > +     if (o.mode) {
> > > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> > > +             if (argc != expected_remaining_argc)
> > > +                     usage_with_options(merge_tree_usage, mt_options);
> > > +     } else {
> > > +             if (argc < 2 || argc > 3)
> > > +                     usage_with_options(merge_tree_usage, mt_options);
> > > +             o.mode = (argc == 2 ? 'w' : 't');
> > > +     }
> >
> > Do we really need to make this interface more special-casey by
> > auto-guessing based on argc what argument you want? I.e. instead of
> > usage like:
> >
> >         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> >         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> >
> > Wouldn't it be simpler to just have the equivalent of:
> >
> >         # old
> >         git merge-tree ...
> >         # new
> >         git merge-tree --new-thing ...
> >
> > And not have to look at ... to figure out if we're dispatching to the
> > new or old thing.
>
> You seem to be focusing on code simplicity?  Sure, that'd be simpler
> code, it'd just be a less useful feature.
>
> I think passing --write-tree all the time would be an annoyance.  I
> don't see why anyone would ever use the other mode.  However, for as
> long as both exist in the manual, it makes the manual easier to
> explain to users, and example testcases more self-documenting by
> having the flag there.  That's the sole purpose of the flag.
>
> I'm never going to actually use it when I invoke it from the command
> line.  And I suspect most would leave it off.

...also, even if we did require the `--write-tree` flag, we'd still
have to look at argc.  Since the option parsing handles both modes,
someone could leave off --write-tree, but include a bunch of other
options that only make sense with --write-tree.  Individually checking
the setting of every extra flag along with write_tree is a royal pain
and I don't want to repeat that for each new option added.  Simply
checking argc allows you to provide an error message if the user does
that.

(And I think it's sad that in Git we often forgot to warn and notify
users of options that are only functional with certain other
arguments; it makes it harder for users to figure out, and has in the
past even made it harder for other developers to figure out what was
meant and how things are to be used.  I think I've seen multiple Git
devs be confused over ls-files --directory and --no-empty-directory
options, assuming they'd do something sensible for tracked files, when
in fact those arguments are simply ignored because they are only
modifiers for how untracked files are treated.)

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-03  9:22           ` Elijah Newren
@ 2022-02-03  9:45             ` Ævar Arnfjörð Bjarmason
  2022-02-03 16:20               ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03  9:45 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt


On Thu, Feb 03 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 1:04 AM Elijah Newren <newren@gmail.com> wrote:
>>
>> On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> >
>> > On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>> >
>> > > From: Elijah Newren <newren@gmail.com>
>> > >
>> > > Let merge-tree accept a `--write-tree` parameter for choosing real
>> > > merges instead of trivial merges, and accept an optional
>> > > `--trivial-merge` option to get the traditional behavior.  Note that
>> > > these accept different numbers of arguments, though, so these names
>> > > need not actually be used.
>> >
>> > Maybe that ship has sailed, but just my 0.02: I thought this whole thing
>> > was much less confusing with your initial merge-tree-ort proposal at
>> > https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
>> > I.e. the end-state of merge-tree.c is that you end up reading largely
>> > unrelated code (various static functions only used by one side or
>> > another).
>>
>> Christian's merge-tree-ort proposal?
>>
>> > But maybe that's all water under the bridge etc, however...
>> >
>> > >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>> > >  {
>> > > -     if (argc != 4)
>> > > -             usage(merge_tree_usage);
>> > > -     return trivial_merge(argc, argv);
>> > > +     struct merge_tree_options o = { 0 };
>> > > +     int expected_remaining_argc;
>> > > +
>> > > +     const char * const merge_tree_usage[] = {
>> > > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> > > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> > > +             NULL
>> > > +     };
>> > > +     struct option mt_options[] = {
>> > > +             OPT_CMDMODE(0, "write-tree", &o.mode,
>> > > +                         N_("do a real merge instead of a trivial merge"),
>> > > +                         'w'),
>> > > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
>> > > +                         N_("do a trivial merge only"), 't'),
>> > > +             OPT_END()
>> > > +     };
>> > > +
>> > > +     /* Parse arguments */
>> > > +     argc = parse_options(argc, argv, prefix, mt_options,
>> > > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
>> > > +     if (o.mode) {
>> > > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
>> > > +             if (argc != expected_remaining_argc)
>> > > +                     usage_with_options(merge_tree_usage, mt_options);
>> > > +     } else {
>> > > +             if (argc < 2 || argc > 3)
>> > > +                     usage_with_options(merge_tree_usage, mt_options);
>> > > +             o.mode = (argc == 2 ? 'w' : 't');
>> > > +     }
>> >
>> > Do we really need to make this interface more special-casey by
>> > auto-guessing based on argc what argument you want? I.e. instead of
>> > usage like:
>> >
>> >         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> >         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> >
>> > Wouldn't it be simpler to just have the equivalent of:
>> >
>> >         # old
>> >         git merge-tree ...
>> >         # new
>> >         git merge-tree --new-thing ...
>> >
>> > And not have to look at ... to figure out if we're dispatching to the
>> > new or old thing.
>>
>> You seem to be focusing on code simplicity?  Sure, that'd be simpler
>> code, it'd just be a less useful feature.
>>
>> I think passing --write-tree all the time would be an annoyance.  I
>> don't see why anyone would ever use the other mode.  However, for as
>> long as both exist in the manual, it makes the manual easier to
>> explain to users, and example testcases more self-documenting by
>> having the flag there.  That's the sole purpose of the flag.
>>
>> I'm never going to actually use it when I invoke it from the command
>> line.  And I suspect most would leave it off.
>
> ...also, even if we did require the `--write-tree` flag, we'd still
> have to look at argc.  Since the option parsing handles both modes,
> someone could leave off --write-tree, but include a bunch of other
> options that only make sense with --write-tree.  Individually checking
> the setting of every extra flag along with write_tree is a royal pain
> and I don't want to repeat that for each new option added.  Simply
> checking argc allows you to provide an error message if the user does
> that.
>
> (And I think it's sad that in Git we often forgot to warn and notify
> users of options that are only functional with certain other
> arguments; it makes it harder for users to figure out, and has in the
> past even made it harder for other developers to figure out what was
> meant and how things are to be used.  I think I've seen multiple Git
> devs be confused over ls-files --directory and --no-empty-directory
> options, assuming they'd do something sensible for tracked files, when
> in fact those arguments are simply ignored because they are only
> modifiers for how untracked files are treated.)

There's a much simpler way to do what you're trying to do here which is
to only parse --write-tree, and as soon as you have that pass off two
one function or the other, and have those functions call
parse_options().

This is how builtin/{bundle,stash,commit-graph}.c etc. solve the same
problem. It avoids having to the sort of check you did in 09/15:

	+	if (o.mode == 't' && original_argc < argc)
	+		die(_("--trivial-merge is incompatible with all other options"));

The builtin/submodule--helper.c then does it with a first argument that
--looks-like-an-option, but is really the same sort of sub-command
selector.

Which, at least for me brings this back to liking your merge-tree-ort
(or merge-tree-ng, merge-tree-new or whatever) better. I.e. both for the
code & manual we effectively have two completely different commands here
anyway. Splitting them up would make both the code simpler, and also
what we have to explain to users.


^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream
  2022-02-03  9:12         ` Elijah Newren
@ 2022-02-03 10:01           ` Ævar Arnfjörð Bjarmason
  2022-02-03 16:09             ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03 10:01 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt


On Thu, Feb 03 2022, Elijah Newren wrote:

> On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>>
>> > From: Elijah Newren <newren@gmail.com>
>> >
>> > This modifies the new display_update_messages() function to allow
>> > printing to somewhere other than stdout.  It also consolidates the
>> > location of the diff_warn_rename_limit() message with the rest of the
>> > CONFLICT and other update messages to all go to the same stream.
>> >
>> > Signed-off-by: Elijah Newren <newren@gmail.com>
>> > ---
>> >  merge-ort.c | 9 +++++----
>> >  merge-ort.h | 3 ++-
>> >  2 files changed, 7 insertions(+), 5 deletions(-)
>> >
>> > diff --git a/merge-ort.c b/merge-ort.c
>> > index 82d2faf5bf9..d28d1721d14 100644
>> > --- a/merge-ort.c
>> > +++ b/merge-ort.c
>> > @@ -4236,7 +4236,8 @@ static int record_conflicted_index_entries(struct merge_options *opt)
>> >  }
>> >
>> >  void merge_display_update_messages(struct merge_options *opt,
>> > -                                struct merge_result *result)
>> > +                                struct merge_result *result,
>> > +                                FILE *stream)
>> >  {
>> >       struct merge_options_internal *opti = result->priv;
>> >       struct hashmap_iter iter;
>> > @@ -4263,13 +4264,13 @@ void merge_display_update_messages(struct merge_options *opt,
>> >       for (i = 0; i < olist.nr; ++i) {
>> >               struct strbuf *sb = olist.items[i].util;
>> >
>> > -             printf("%s", sb->buf);
>> > +             strbuf_write(sb, stream);
>> >       }
>> >       string_list_clear(&olist, 0);
>> >
>> >       /* Also include needed rename limit adjustment now */
>> >       diff_warn_rename_limit("merge.renamelimit",
>> > -                            opti->renames.needed_limit, 0, stderr);
>> > +                            opti->renames.needed_limit, 0, stream);
>>
>> At the tip of this series I tried to s/stream/stderr/g this, and
>> t4301-merge-tree-write-tree.sh passes, doesn't this warning_fp() special
>> behavior need a test somewhere?
>
> That's a fair point, but...this gets back to my cover letter comments
> about patches 5, 6, and 8.  They implement a code feature that seems
> useful in general...but which Dscho and Christian didn't like using in
> this particular command; they just wanted all output on stdout.
>
> So, it's hard to add a test, because there's no code anywhere that
> exercises it in this series anymore.  I originally wanted this feature
> in my remerge-diff series, but the idea of conflict headers made me
> punt it to this series.  I wanted it for this series, but Dscho and
> Christian didn't.  I could have punted again, but decided the
> underlying want kept coming up and decided to not excise it --
> especially since Dscho was helping improve it.  And Junio commented
> that he liked the idea[1].
>
> [1] https://lore.kernel.org/git/xmqqh79hx8g1.fsf@gitster.g/
>
> But yeah, it does leave it feeling slightly odd that we implemented a
> feature that nothing is currently using.  Maybe these 3 should be
> split off into their own series?  Still wouldn't have a test yet,
> though.
>
>> I assumed that warning_fp() would be using vreportf() in usage.c, but
>> it's not, it's just falling back to the equivalent of fprintf(out, ...),
>> no? I don't really see why 05/15 and parts of 06/15 & this are needed
>> over a much simpler simple helper macro like the below. applied on top
>> of this series.
>
> That macro is simple?  I thought I basically understood Dscho's code,
> but looking at what you did with diff_warn_rename_limit(), I think I'm
> lost.

I guess that's a matter of taste, yeah it's a bit of macro soup if
you're not familiar with it. FWIW (sans bug I noted below) it's the
macro soup we already use for other functions in usage.c.

>> I would get it if the point was to actually use the full usage.c
>> machinery, but we're just either calling warning(), or printing a
>> formatted string to a file FILE *. There's no need to go through usage.c
>> for that, and adding an API to it that behaves like this new
>> warning_fp() is really confusing.
>
> Because the formatted string being printed to the file won't have the
> same "warning: " prefix that is normally added to stuff in usage?

But the pre-image doesn't add that either. We're just calling
vfprintf(), not our own vreportf().

> That's a fair point; that does have a bit of a consistency problem.
> And I'd rather the messages were consistent regardless of where they
> are printed.

I think that makes sense, that's why I added die_message() recently. If
you meant to print a "warning: " prefix I think it would also be fine in
this case to just do it inline. See prior art at:

    git grep '"(fatal|error|warning):' -- '*.c'

>> I.e. an API in usage.c that allowed warning to a given FD would be
>> trying to replace the "2" in the write_in_full() call in vreportf(), I
>> would think.
>
> Hmm, makes sense.

The reason I'm barking up this particular tree is that I've got some
upcoming patches for usage.c (the C99-only macro series):
https://lore.kernel.org/git/RFC-cover-00.21-00000000000-20211115T220831Z-avarab@gmail.com/

It would need to deal with anything in the API. In this case there's not
much to deal with, since it's really not at all using the rest of
usage.c, it's just a "or to stderr".

>> diff --git a/diff.c b/diff.c
>> index 28368110147..4cf67e93dea 100644
>> --- a/diff.c
>> +++ b/diff.c
>> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
>>  N_("you may want to set your %s variable to at least "
>>     "%d and retry the command.");
>>
>> +#define warning_fp(out, fmt, ...) do { \
>> +       if (out == stderr) \
>> +               warning(fmt, __VA_ARGS__); \
>> +       else \
>> +               fprintf(out, fmt, __VA_ARGS__); \
>> +} while (0)
>> +
>>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
>>                             FILE *out)
>>  {
>>         fflush(stdout);
>>         if (degraded_cc)
>> -               warning_fp(out, _(degrade_cc_to_c_warning));
>> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
>>         else if (needed)
>> -               warning_fp(out, _(rename_limit_warning));
>> +               warning_fp(out, _(rename_limit_warning), NULL);
>
> Why do the only callers have a NULL parameter here?  Is this one of
> those va_list/va_args things I never bothered to properly learn?

That's wrong (I blame tiredness last night),an actual working version is
produced below. Clang accepted my broken code, but gcc rightly yells
about it:

diff --git a/diff.c b/diff.c
index 28368110147..a2bc2595533 100644
--- a/diff.c
+++ b/diff.c
@@ -6377,6 +6377,13 @@ static const char rename_limit_advice[] =
 N_("you may want to set your %s variable to at least "
    "%d and retry the command.");
 
+#define warning_fp(out, ...) do { \
+	if (out == stderr) \
+		warning(__VA_ARGS__); \
+	else \
+		fprintf(out, __VA_ARGS__); \
+} while (0)
+
 void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
 			    FILE *out)
 {
diff --git a/git-compat-util.h b/git-compat-util.h
index 64ba60e5c71..d70ce142861 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
-void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index 0bfd2c603c0..c7d233b0de9 100644
--- a/usage.c
+++ b/usage.c
@@ -253,20 +253,6 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
-void warning_fp(FILE *out, const char *warn, ...)
-{
-	va_list params;
-
-	va_start(params, warn);
-	if (out == stderr)
-		warn_routine(warn, params);
-	else {
-		vfprintf(out, warn, params);
-		fputc('\n', out);
-	}
-	va_end(params);
-}
-
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 
I do think you'd probably prefer the non-macro version, which is pretty
much just going back to this:
https://lore.kernel.org/git/6fb4f4580a581b2e43bc4b8deaa3d2d2bf4a8756.1643479633.git.gitgitgadget@gmail.com/

diff --git a/diff.c b/diff.c
index 28368110147..21c9561f546 100644
--- a/diff.c
+++ b/diff.c
@@ -6380,17 +6380,28 @@ N_("you may want to set your %s variable to at least "
 void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
 			    FILE *out)
 {
+	const char *msg;
+
 	fflush(stdout);
 	if (degraded_cc)
-		warning_fp(out, _(degrade_cc_to_c_warning));
+		msg = _(degrade_cc_to_c_warning);
 	else if (needed)
-		warning_fp(out, _(rename_limit_warning));
+		msg = _(rename_limit_warning);
 	else
 		return;
 
+	if (out == stderr)
+		warning("%s", msg);
+	else
+		fprintf(stderr, "%s", msg);
 
-	if (0 < needed)
-		warning_fp(out, _(rename_limit_advice), varname, needed);
+	if (0 >= needed)
+		return;
+
+	if (out == stderr)
+		warning(_(rename_limit_advice), varname, needed);
+	else
+		fprintf(stderr, _(rename_limit_advice), varname, needed);
 }
 
 static void create_filepairs_for_header_only_notifications(struct diff_options *o)
diff --git a/git-compat-util.h b/git-compat-util.h
index 64ba60e5c71..d70ce142861 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -475,7 +475,6 @@ int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 int error_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 void warning_errno(const char *err, ...) __attribute__((format (printf, 1, 2)));
-void warning_fp(FILE *out, const char *warn, ...) __attribute__((format (printf, 2, 3)));
 
 #ifndef NO_OPENSSL
 #ifdef APPLE_COMMON_CRYPTO
diff --git a/usage.c b/usage.c
index 0bfd2c603c0..c7d233b0de9 100644
--- a/usage.c
+++ b/usage.c
@@ -253,20 +253,6 @@ void warning(const char *warn, ...)
 	va_end(params);
 }
 
-void warning_fp(FILE *out, const char *warn, ...)
-{
-	va_list params;
-
-	va_start(params, warn);
-	if (out == stderr)
-		warn_routine(warn, params);
-	else {
-		vfprintf(out, warn, params);
-		fputc('\n', out);
-	}
-	va_end(params);
-}
-
 /* Only set this, ever, from t/helper/, when verifying that bugs are caught. */
 int BUG_exit_code;
 
Note that both your pre-image, my macro version and Johannes's
linked-to-above are technically buggy in that they treat a
non-formatting format as a formatting format. I.e. we should use
warning("%s", msg) in that case, not warning(msg).

See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
usage, 2021-07-13) for a similar bug/fix.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-03  9:04         ` Elijah Newren
  2022-02-03  9:22           ` Elijah Newren
@ 2022-02-03 10:26           ` Ævar Arnfjörð Bjarmason
  1 sibling, 0 replies; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03 10:26 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt


On Thu, Feb 03 2022, Elijah Newren wrote:

> On Wed, Feb 2, 2022 at 6:09 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Wed, Feb 02 2022, Elijah Newren via GitGitGadget wrote:
>>
>> > From: Elijah Newren <newren@gmail.com>
>> >
>> > Let merge-tree accept a `--write-tree` parameter for choosing real
>> > merges instead of trivial merges, and accept an optional
>> > `--trivial-merge` option to get the traditional behavior.  Note that
>> > these accept different numbers of arguments, though, so these names
>> > need not actually be used.
>>
>> Maybe that ship has sailed, but just my 0.02: I thought this whole thing
>> was much less confusing with your initial merge-tree-ort proposal at
>> https://lore.kernel.org/git/CABPp-BEeBpJoU4yXdfA6vRAYVAUbd2gRhEV6j4VEqoqcu=FGSw@mail.gmail.com/;
>> I.e. the end-state of merge-tree.c is that you end up reading largely
>> unrelated code (various static functions only used by one side or
>> another).
>
> Christian's merge-tree-ort proposal?

Yes. I'm clearly getting the chronology wrong here. Sorry.

>> But maybe that's all water under the bridge etc, however...
>>
>> >  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>> >  {
>> > -     if (argc != 4)
>> > -             usage(merge_tree_usage);
>> > -     return trivial_merge(argc, argv);
>> > +     struct merge_tree_options o = { 0 };
>> > +     int expected_remaining_argc;
>> > +
>> > +     const char * const merge_tree_usage[] = {
>> > +             N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> > +             N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> > +             NULL
>> > +     };
>> > +     struct option mt_options[] = {
>> > +             OPT_CMDMODE(0, "write-tree", &o.mode,
>> > +                         N_("do a real merge instead of a trivial merge"),
>> > +                         'w'),
>> > +             OPT_CMDMODE(0, "trivial-merge", &o.mode,
>> > +                         N_("do a trivial merge only"), 't'),
>> > +             OPT_END()
>> > +     };
>> > +
>> > +     /* Parse arguments */
>> > +     argc = parse_options(argc, argv, prefix, mt_options,
>> > +                          merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
>> > +     if (o.mode) {
>> > +             expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
>> > +             if (argc != expected_remaining_argc)
>> > +                     usage_with_options(merge_tree_usage, mt_options);
>> > +     } else {
>> > +             if (argc < 2 || argc > 3)
>> > +                     usage_with_options(merge_tree_usage, mt_options);
>> > +             o.mode = (argc == 2 ? 'w' : 't');
>> > +     }
>>
>> Do we really need to make this interface more special-casey by
>> auto-guessing based on argc what argument you want? I.e. instead of
>> usage like:
>>
>>         N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>>         N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>>
>> Wouldn't it be simpler to just have the equivalent of:
>>
>>         # old
>>         git merge-tree ...
>>         # new
>>         git merge-tree --new-thing ...
>>
>> And not have to look at ... to figure out if we're dispatching to the
>> new or old thing.
>
> You seem to be focusing on code simplicity?  Sure, that'd be simpler
> code, it'd just be a less useful feature.
>
> I think passing --write-tree all the time would be an annoyance.  I
> don't see why anyone would ever use the other mode.  However, for as
> long as both exist in the manual, it makes the manual easier to
> explain to users, and example testcases more self-documenting by
> having the flag there.  That's the sole purpose of the flag.
>
> I'm never going to actually use it when I invoke it from the command
> line.  And I suspect most would leave it off.

I think I mostly covered this in
https://lore.kernel.org/git/220203.86wnic5lba.gmgdl@evledraar.gmail.com/
in the side-thread.

I'm not opposed to them being in the same command, it just seems like
more work for those reasons in every way.

E.g. the current manual page doesn't really describe the format the
--trivial-merge emits, so we could do with a section on that, but then
it would say "only for this option, not the other" etc.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-03  0:18             ` Elijah Newren
@ 2022-02-03 10:42               ` Johannes Altmanninger
  2022-02-03 16:54                 ` Elijah Newren
  2022-02-03 20:05                 ` Junio C Hamano
  2022-02-21 18:55               ` Junio C Hamano
  1 sibling, 2 replies; 215+ messages in thread
From: Johannes Altmanninger @ 2022-02-03 10:42 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Junio C Hamano, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Ramsay Jones, Johannes Schindelin,
	Christian Couder, René Scharfe,
	Ævar Arnfjörð Bjarmason, Johannes Sixt

On Wed, Feb 02, 2022 at 04:18:39PM -0800, Elijah Newren wrote:
> On Wed, Feb 2, 2022 at 2:01 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Elijah Newren <newren@gmail.com> writes:
> >
> > > Yes, you are reading right.  I think the cherry-pick/rebase
> > > replacement actually deserves a separate command from what merges
> > > should use; replaying a sequence of commits just has a number of UI
> > > differences and abilities that I think pull it in a different
> > > direction.
> >
> > I completely disagree.  Each individual step in a sequence of
> > replaying commits in order (or in reverse order) should be
> > scriptable as a single merge-tree that takes "apply the change to go
> > from A^ to A on X".  Sequencing and placing UI around it is a job
> > for the script that drives merge-tree.
> 
> Adding such an ability to merge-tree would be trivial -- it basically
> involves just two things: (1) accepting one extra argument, and (2)
> calling merge_incore_nonrecursive() instead of
> merge_incore_recursive().
> 
> However, I think forking a subprocess for every merge of a series of
> commits is a completely unreasonable overhead, so even if we provide
> such an option to merge-tree, I still want a separate plumbing-ish
> tool that does non-worktree/non-index replaying of commits which is
> not written as a driver of merge-tree.  That other tool should just
> call merge_incore_nonrecursive() directly.  And such a tool, since it
> should handle an arbitrary number of commits, should certainly be able
> to handle just one commit.  From that angle, it feels like adding
> another mode to merge-tree would just be a partial duplication of the
> other tool.

I wonder how the UI of a tool that does non-worktree/non-index cherry-picks
will look like.  I'd expect it to produce the same output as merge-tree,
except cherry-pick should probably output a commit OID, not a tree.

Maybe we want a unified command that produces commits from any sequence of
merge/cherry-pick/revert/reword steps. The obvious UI would use something
like the rebase-todo list as input.  For example:

	$ echo '
	pick commit1
	reword commit2	# edit commit message in $GIT_EDITOR
	merge commit3 -m "log message"
	' | git create-commit commit0
	<OID of final commit>

we start from commit0 and apply steps one-by-one. Obviously, one unsolved
problem is how to pass parameters like commit messages if no editor should
be invoked (my sketch uses -m).
If any of the steps fails when merging merge, then we get the tree with
conflicts

	$ echo '
	pick commit1
	pick commit2
	pick commit-that-does-not-apply
	' | git create-commit commit0
	<OID of commit after step 2>
	<OID of toplevel tree after failed merge>
	<Conflicted file info>
	<Informational messages>

Replaying a series of commits might look like this:

	$ echo 'pick commit1 ^commit0' | git create-commit new-base

I'm concluding that this is a difficult UI problem, and having a merge-tree
command that accepts a "common ancestor" parameter could make it easier
to experiment.  Of course that depends on who is experimenting.

> 
> However, if the other tool doesn't obviate the need for this
> additional mode (perhaps it ends up being forced to be too
> porcelain-ish insteading of plumbing-ish?), or folks really just want
> another merge-tree mode, I'm happy to add one along with the tool I
> submit later.  Does that sound reasonable to you, or is there
> something you're still objecting to that I've missed?

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream
  2022-02-03 10:01           ` Ævar Arnfjörð Bjarmason
@ 2022-02-03 16:09             ` Elijah Newren
  2022-02-03 16:19               ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-03 16:09 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Thu, Feb 3, 2022 at 2:26 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
> > On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
[...]
> >> I would get it if the point was to actually use the full usage.c
> >> machinery, but we're just either calling warning(), or printing a
> >> formatted string to a file FILE *. There's no need to go through usage.c
> >> for that, and adding an API to it that behaves like this new
> >> warning_fp() is really confusing.
> >
> > Because the formatted string being printed to the file won't have the
> > same "warning: " prefix that is normally added to stuff in usage?
>
> But the pre-image doesn't add that either. We're just calling
> vfprintf(), not our own vreportf().

Right, I'm saying that I thought you were reporting the original patch
as buggy because it doesn't produce the same message when given a
different stream; it'll omit the "warning: " prefix.  And I was
agreeing that it was buggy for those reasons.

Or was there a different reason you didn't like that function being in usage.c?

> > That's a fair point; that does have a bit of a consistency problem.
> > And I'd rather the messages were consistent regardless of where they
> > are printed.
>
> I think that makes sense, that's why I added die_message() recently. If
> you meant to print a "warning: " prefix I think it would also be fine in
> this case to just do it inline. See prior art at:
>
>     git grep '"(fatal|error|warning):' -- '*.c'

So, making diff_warn_rename_limit() stop using warning(), and just
always directly writing out and including "warning:" in its message?

I'm wondering if that might cause problems if there are any existing
callers of diff_warn_rename_limit() that might also be using
set_warn_routine() (e.g. perhaps apply.c?).  Of course, those callers
probably couldn't handle anything other than the default stream.
Hmm...

> >> diff --git a/diff.c b/diff.c
> >> index 28368110147..4cf67e93dea 100644
> >> --- a/diff.c
> >> +++ b/diff.c
> >> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
> >>  N_("you may want to set your %s variable to at least "
> >>     "%d and retry the command.");
> >>
> >> +#define warning_fp(out, fmt, ...) do { \
> >> +       if (out == stderr) \
> >> +               warning(fmt, __VA_ARGS__); \
> >> +       else \
> >> +               fprintf(out, fmt, __VA_ARGS__); \
> >> +} while (0)
> >> +
> >>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> >>                             FILE *out)
> >>  {
> >>         fflush(stdout);
> >>         if (degraded_cc)
> >> -               warning_fp(out, _(degrade_cc_to_c_warning));
> >> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
> >>         else if (needed)
> >> -               warning_fp(out, _(rename_limit_warning));
> >> +               warning_fp(out, _(rename_limit_warning), NULL);
> >
> > Why do the only callers have a NULL parameter here?  Is this one of
> > those va_list/va_args things I never bothered to properly learn?
>
> That's wrong (I blame tiredness last night),an actual working version is
> produced below. Clang accepted my broken code, but gcc rightly yells
> about it:

Well, seeing the new code makes me feel better as it makes more sense
to me now.  ;-)

> Note that both your pre-image, my macro version and Johannes's
> linked-to-above are technically buggy in that they treat a
> non-formatting format as a formatting format. I.e. we should use
> warning("%s", msg) in that case, not warning(msg).
>
> See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
> usage, 2021-07-13) for a similar bug/fix.

Good point.


Man, what a can of worms this all is.  Maybe I really should just drop
patches 5, 6, and 8 for now...

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream
  2022-02-03 16:09             ` Elijah Newren
@ 2022-02-03 16:19               ` Ævar Arnfjörð Bjarmason
  2022-02-03 17:00                 ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03 16:19 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt


On Thu, Feb 03 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 2:26 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Thu, Feb 03 2022, Elijah Newren wrote:
>>
>> > On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> [...]
>> >> I would get it if the point was to actually use the full usage.c
>> >> machinery, but we're just either calling warning(), or printing a
>> >> formatted string to a file FILE *. There's no need to go through usage.c
>> >> for that, and adding an API to it that behaves like this new
>> >> warning_fp() is really confusing.
>> >
>> > Because the formatted string being printed to the file won't have the
>> > same "warning: " prefix that is normally added to stuff in usage?
>>
>> But the pre-image doesn't add that either. We're just calling
>> vfprintf(), not our own vreportf().
>
> Right, I'm saying that I thought you were reporting the original patch
> as buggy because it doesn't produce the same message when given a
> different stream; it'll omit the "warning: " prefix.  And I was
> agreeing that it was buggy for those reasons.
>
> Or was there a different reason you didn't like that function being in usage.c?

Maybe it was accidentally a bug report :) But no, I was just observing
that it was odd that it was in usage.c when it seemingly had almost
nothing to do with what that API accomplishes.

But maybe the underlying issue is that the "warning: " part is missing
here. But I didn't mean to report that/missed it.

>> > That's a fair point; that does have a bit of a consistency problem.
>> > And I'd rather the messages were consistent regardless of where they
>> > are printed.
>>
>> I think that makes sense, that's why I added die_message() recently. If
>> you meant to print a "warning: " prefix I think it would also be fine in
>> this case to just do it inline. See prior art at:
>>
>>     git grep '"(fatal|error|warning):' -- '*.c'
>
> So, making diff_warn_rename_limit() stop using warning(), and just
> always directly writing out and including "warning:" in its message?
>
> I'm wondering if that might cause problems if there are any existing
> callers of diff_warn_rename_limit() that might also be using
> set_warn_routine() (e.g. perhaps apply.c?).  Of course, those callers
> probably couldn't handle anything other than the default stream.
> Hmm...

Using set_warn_routine() is the "right" way to do it currently, and with
or without a "warning: " prefix the current API use is "wrong" if the
purpose is to have it behave nicely with the pluggable usage.c API.

But of course that may not be the goal at all, i.e. I think here we've
probably stopped caring about usage.c's formatting, logging
etc. entirely, and are just emitting a string.

Just like serve.c emits "E <msg>" or whatever (and not with error()).

>> >> diff --git a/diff.c b/diff.c
>> >> index 28368110147..4cf67e93dea 100644
>> >> --- a/diff.c
>> >> +++ b/diff.c
>> >> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
>> >>  N_("you may want to set your %s variable to at least "
>> >>     "%d and retry the command.");
>> >>
>> >> +#define warning_fp(out, fmt, ...) do { \
>> >> +       if (out == stderr) \
>> >> +               warning(fmt, __VA_ARGS__); \
>> >> +       else \
>> >> +               fprintf(out, fmt, __VA_ARGS__); \
>> >> +} while (0)
>> >> +
>> >>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
>> >>                             FILE *out)
>> >>  {
>> >>         fflush(stdout);
>> >>         if (degraded_cc)
>> >> -               warning_fp(out, _(degrade_cc_to_c_warning));
>> >> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
>> >>         else if (needed)
>> >> -               warning_fp(out, _(rename_limit_warning));
>> >> +               warning_fp(out, _(rename_limit_warning), NULL);
>> >
>> > Why do the only callers have a NULL parameter here?  Is this one of
>> > those va_list/va_args things I never bothered to properly learn?
>>
>> That's wrong (I blame tiredness last night),an actual working version is
>> produced below. Clang accepted my broken code, but gcc rightly yells
>> about it:
>
> Well, seeing the new code makes me feel better as it makes more sense
> to me now.  ;-)
>
>> Note that both your pre-image, my macro version and Johannes's
>> linked-to-above are technically buggy in that they treat a
>> non-formatting format as a formatting format. I.e. we should use
>> warning("%s", msg) in that case, not warning(msg).
>>
>> See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
>> usage, 2021-07-13) for a similar bug/fix.
>
> Good point.
>
> Man, what a can of worms this all is.  Maybe I really should just drop
> patches 5, 6, and 8 for now...

Yeah, I really think it's worth it to just sprinkle a tiny bit of
if/else (or a macro) here and print to stderr inline or not. We can make
some use of some usage.c when there's good reason to do so, but this bit
just seems like a needless digression.

I hope all of this has helped somewhat ...

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-03  9:45             ` Ævar Arnfjörð Bjarmason
@ 2022-02-03 16:20               ` Elijah Newren
  2022-02-03 17:15                 ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-03 16:20 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Thu, Feb 3, 2022 at 1:52 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
[...]
> > ...also, even if we did require the `--write-tree` flag, we'd still
> > have to look at argc.  Since the option parsing handles both modes,
> > someone could leave off --write-tree, but include a bunch of other
> > options that only make sense with --write-tree.  Individually checking
> > the setting of every extra flag along with write_tree is a royal pain
> > and I don't want to repeat that for each new option added.  Simply
> > checking argc allows you to provide an error message if the user does
> > that.
> >
> > (And I think it's sad that in Git we often forgot to warn and notify
> > users of options that are only functional with certain other
> > arguments; it makes it harder for users to figure out, and has in the
> > past even made it harder for other developers to figure out what was
> > meant and how things are to be used.  I think I've seen multiple Git
> > devs be confused over ls-files --directory and --no-empty-directory
> > options, assuming they'd do something sensible for tracked files, when
> > in fact those arguments are simply ignored because they are only
> > modifiers for how untracked files are treated.)
>
> There's a much simpler way to do what you're trying to do here which is
> to only parse --write-tree, and as soon as you have that pass off two
> one function or the other, and have those functions call
> parse_options().

But that makes --write-tree a mandatory argument when trying to use
that mode, right?  If so, that is not a simpler way to do what I'm
trying to do at all; it breaks my intended usage.

--write-tree is a documentation-only construct that users should never
have to pass.

Also, what happens if we remove the --trivial-merge flag and its whole
mode after a sufficient deprecation period?  Would the --write-tree
parameter remain required in your model to select the only existing
mode, simply due to us having gone through a transition period?

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-03 10:42               ` Johannes Altmanninger
@ 2022-02-03 16:54                 ` Elijah Newren
  2022-02-21  9:06                   ` Johannes Schindelin
  2022-02-03 20:05                 ` Junio C Hamano
  1 sibling, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-03 16:54 UTC (permalink / raw)
  To: Johannes Altmanninger
  Cc: Junio C Hamano, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Ramsay Jones, Johannes Schindelin,
	Christian Couder, René Scharfe,
	Ævar Arnfjörð Bjarmason, Johannes Sixt

Hi,

On Thu, Feb 3, 2022 at 2:42 AM Johannes Altmanninger <aclopte@gmail.com> wrote:
>
> On Wed, Feb 02, 2022 at 04:18:39PM -0800, Elijah Newren wrote:
> > On Wed, Feb 2, 2022 at 2:01 PM Junio C Hamano <gitster@pobox.com> wrote:
> > >
> > > Elijah Newren <newren@gmail.com> writes:
> > >
> > > > Yes, you are reading right.  I think the cherry-pick/rebase
> > > > replacement actually deserves a separate command from what merges
> > > > should use; replaying a sequence of commits just has a number of UI
> > > > differences and abilities that I think pull it in a different
> > > > direction.
> > >
> > > I completely disagree.  Each individual step in a sequence of
> > > replaying commits in order (or in reverse order) should be
> > > scriptable as a single merge-tree that takes "apply the change to go
> > > from A^ to A on X".  Sequencing and placing UI around it is a job
> > > for the script that drives merge-tree.
> >
> > Adding such an ability to merge-tree would be trivial -- it basically
> > involves just two things: (1) accepting one extra argument, and (2)
> > calling merge_incore_nonrecursive() instead of
> > merge_incore_recursive().
> >
> > However, I think forking a subprocess for every merge of a series of
> > commits is a completely unreasonable overhead, so even if we provide
> > such an option to merge-tree, I still want a separate plumbing-ish
> > tool that does non-worktree/non-index replaying of commits which is
> > not written as a driver of merge-tree.  That other tool should just
> > call merge_incore_nonrecursive() directly.  And such a tool, since it
> > should handle an arbitrary number of commits, should certainly be able
> > to handle just one commit.  From that angle, it feels like adding
> > another mode to merge-tree would just be a partial duplication of the
> > other tool.
>
> I wonder how the UI of a tool that does non-worktree/non-index cherry-picks
> will look like.  I'd expect it to produce the same output as merge-tree,
> except cherry-pick should probably output a commit OID, not a tree.
>
> Maybe we want a unified command that produces commits from any sequence of
> merge/cherry-pick/revert/reword steps. The obvious UI would use something
> like the rebase-todo list as input.  For example:
>
>         $ echo '
>         pick commit1
>         reword commit2  # edit commit message in $GIT_EDITOR
>         merge commit3 -m "log message"
>         ' | git create-commit commit0
>         <OID of final commit>
>
> we start from commit0 and apply steps one-by-one. Obviously, one unsolved
> problem is how to pass parameters like commit messages if no editor should
> be invoked (my sketch uses -m).
> If any of the steps fails when merging merge, then we get the tree with
> conflicts
>
>         $ echo '
>         pick commit1
>         pick commit2
>         pick commit-that-does-not-apply
>         ' | git create-commit commit0
>         <OID of commit after step 2>
>         <OID of toplevel tree after failed merge>
>         <Conflicted file info>
>         <Informational messages>
>
> Replaying a series of commits might look like this:
>
>         $ echo 'pick commit1 ^commit0' | git create-commit new-base
>
> I'm concluding that this is a difficult UI problem

I agree.  I've got a lot of thoughts on it, and some work in progress
towards it (https://github.com/newren/git/tree/replay -- _very_ hacky,
not even close to alpha quality, lots of fixup commits, todo comments,
random brain dump files added to the tree, based on a previous round
of this patch series, not updated for weeks, etc., etc.)

> and having a merge-tree
> command that accepts a "common ancestor" parameter could make it easier
> to experiment.  Of course that depends on who is experimenting.

I think that would result in experiments and eventually full-blown
scripts designed around forking subprocesses for every merge, and
pushes us back into the world of having a scripted-rebase again.  Yes,
I know people can transliterate shell back to C; it seems to always be
done as a half-way measure with the forking just being done from C or
have other UI-warts guided by the shell design.  In fact, *that* was
the primary reason for me not providing a merge-tree option based on
merge_incore_nonrecursive(), despite how trivial it'd be to provide
it.  If someone wanted a merge_incore_nonrecursive() mode for
merge-tree for reasons other than attempting to build a
rebase/cherry-pick replacement based on it, then I'd be much happier
to provide it.

If someone wants to experiment with what a plumbing-ish
rebase/cherry-pick would look like, the _right_ way to do it would be
making using of merge_incore_nonrecursive() directly.  If they want
example code, I already provided some a year and a half ago and got it
merged into git.git in the form of t/helper/test-fast-rebase.c.  My
"replay" branch is based on that code, but (a) moves it from t/helper
to a real builtin, (b) removes the hardcoded very strict input, (c)
removes the line of code doing the index & working tree updates, and
(d) modifies the output to be a more plumbing-ish style.

We'll certainly have discussions on what that should look like.  But a
plumbing-ish replacement for merge was much simpler, and made sense to
do first.  I would prefer to concentrate on getting that hammered down
first.  Then I'll start discussions on a plumbing-ish
rebase/cherry-pick.  And if that doesn't fulfill all the needs that
folks think they want out of merge-tree, then we can add a
merge_incore_nonrecursive()-based mode to merge-tree.  It's all
coming, but having fought transliterations-of-scripts in
merge-recursive.c, sequencer.c, stash.c, rebase.c, etc. for years I
really, really don't want any more of that.  Let's end that insanity.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 08/15] merge-ort: allow update messages to be written to different file stream
  2022-02-03 16:19               ` Ævar Arnfjörð Bjarmason
@ 2022-02-03 17:00                 ` Elijah Newren
  2022-02-21  9:13                   ` Johannes Schindelin
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-03 17:00 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Thu, Feb 3, 2022 at 8:24 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
> > On Thu, Feb 3, 2022 at 2:26 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> >>
> >> On Thu, Feb 03 2022, Elijah Newren wrote:
> >>
> >> > On Wed, Feb 2, 2022 at 6:01 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> > [...]
> >> >> I would get it if the point was to actually use the full usage.c
> >> >> machinery, but we're just either calling warning(), or printing a
> >> >> formatted string to a file FILE *. There's no need to go through usage.c
> >> >> for that, and adding an API to it that behaves like this new
> >> >> warning_fp() is really confusing.
> >> >
> >> > Because the formatted string being printed to the file won't have the
> >> > same "warning: " prefix that is normally added to stuff in usage?
> >>
> >> But the pre-image doesn't add that either. We're just calling
> >> vfprintf(), not our own vreportf().
> >
> > Right, I'm saying that I thought you were reporting the original patch
> > as buggy because it doesn't produce the same message when given a
> > different stream; it'll omit the "warning: " prefix.  And I was
> > agreeing that it was buggy for those reasons.
> >
> > Or was there a different reason you didn't like that function being in usage.c?
>
> Maybe it was accidentally a bug report :) But no, I was just observing
> that it was odd that it was in usage.c when it seemingly had almost
> nothing to do with what that API accomplishes.
>
> But maybe the underlying issue is that the "warning: " part is missing
> here. But I didn't mean to report that/missed it.
>
> >> > That's a fair point; that does have a bit of a consistency problem.
> >> > And I'd rather the messages were consistent regardless of where they
> >> > are printed.
> >>
> >> I think that makes sense, that's why I added die_message() recently. If
> >> you meant to print a "warning: " prefix I think it would also be fine in
> >> this case to just do it inline. See prior art at:
> >>
> >>     git grep '"(fatal|error|warning):' -- '*.c'
> >
> > So, making diff_warn_rename_limit() stop using warning(), and just
> > always directly writing out and including "warning:" in its message?
> >
> > I'm wondering if that might cause problems if there are any existing
> > callers of diff_warn_rename_limit() that might also be using
> > set_warn_routine() (e.g. perhaps apply.c?).  Of course, those callers
> > probably couldn't handle anything other than the default stream.
> > Hmm...
>
> Using set_warn_routine() is the "right" way to do it currently, and with
> or without a "warning: " prefix the current API use is "wrong" if the
> purpose is to have it behave nicely with the pluggable usage.c API.
>
> But of course that may not be the goal at all, i.e. I think here we've
> probably stopped caring about usage.c's formatting, logging
> etc. entirely, and are just emitting a string.
>
> Just like serve.c emits "E <msg>" or whatever (and not with error()).
>
> >> >> diff --git a/diff.c b/diff.c
> >> >> index 28368110147..4cf67e93dea 100644
> >> >> --- a/diff.c
> >> >> +++ b/diff.c
> >> >> @@ -6377,14 +6377,21 @@ static const char rename_limit_advice[] =
> >> >>  N_("you may want to set your %s variable to at least "
> >> >>     "%d and retry the command.");
> >> >>
> >> >> +#define warning_fp(out, fmt, ...) do { \
> >> >> +       if (out == stderr) \
> >> >> +               warning(fmt, __VA_ARGS__); \
> >> >> +       else \
> >> >> +               fprintf(out, fmt, __VA_ARGS__); \
> >> >> +} while (0)
> >> >> +
> >> >>  void diff_warn_rename_limit(const char *varname, int needed, int degraded_cc,
> >> >>                             FILE *out)
> >> >>  {
> >> >>         fflush(stdout);
> >> >>         if (degraded_cc)
> >> >> -               warning_fp(out, _(degrade_cc_to_c_warning));
> >> >> +               warning_fp(out, _(degrade_cc_to_c_warning), NULL);
> >> >>         else if (needed)
> >> >> -               warning_fp(out, _(rename_limit_warning));
> >> >> +               warning_fp(out, _(rename_limit_warning), NULL);
> >> >
> >> > Why do the only callers have a NULL parameter here?  Is this one of
> >> > those va_list/va_args things I never bothered to properly learn?
> >>
> >> That's wrong (I blame tiredness last night),an actual working version is
> >> produced below. Clang accepted my broken code, but gcc rightly yells
> >> about it:
> >
> > Well, seeing the new code makes me feel better as it makes more sense
> > to me now.  ;-)
> >
> >> Note that both your pre-image, my macro version and Johannes's
> >> linked-to-above are technically buggy in that they treat a
> >> non-formatting format as a formatting format. I.e. we should use
> >> warning("%s", msg) in that case, not warning(msg).
> >>
> >> See 927dc330705 (advice.h: add missing __attribute__((format)) & fix
> >> usage, 2021-07-13) for a similar bug/fix.
> >
> > Good point.
> >
> > Man, what a can of worms this all is.  Maybe I really should just drop
> > patches 5, 6, and 8 for now...
>
> Yeah, I really think it's worth it to just sprinkle a tiny bit of
> if/else (or a macro) here and print to stderr inline or not. We can make
> some use of some usage.c when there's good reason to do so, but this bit
> just seems like a needless digression.
>
> I hope all of this has helped somewhat ...

Absolutely; thanks for reviewing!  These parts may just end up in me
dropping some patches for now (since they're not actually being used
anyway), but I think it's all good feedback.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-03 16:20               ` Elijah Newren
@ 2022-02-03 17:15                 ` Ævar Arnfjörð Bjarmason
  2022-02-03 18:18                   ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-02-03 17:15 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt


On Thu, Feb 03 2022, Elijah Newren wrote:

> On Thu, Feb 3, 2022 at 1:52 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>
>> On Thu, Feb 03 2022, Elijah Newren wrote:
>>
> [...]
>> > ...also, even if we did require the `--write-tree` flag, we'd still
>> > have to look at argc.  Since the option parsing handles both modes,
>> > someone could leave off --write-tree, but include a bunch of other
>> > options that only make sense with --write-tree.  Individually checking
>> > the setting of every extra flag along with write_tree is a royal pain
>> > and I don't want to repeat that for each new option added.  Simply
>> > checking argc allows you to provide an error message if the user does
>> > that.
>> >
>> > (And I think it's sad that in Git we often forgot to warn and notify
>> > users of options that are only functional with certain other
>> > arguments; it makes it harder for users to figure out, and has in the
>> > past even made it harder for other developers to figure out what was
>> > meant and how things are to be used.  I think I've seen multiple Git
>> > devs be confused over ls-files --directory and --no-empty-directory
>> > options, assuming they'd do something sensible for tracked files, when
>> > in fact those arguments are simply ignored because they are only
>> > modifiers for how untracked files are treated.)
>>
>> There's a much simpler way to do what you're trying to do here which is
>> to only parse --write-tree, and as soon as you have that pass off two
>> one function or the other, and have those functions call
>> parse_options().
>
> But that makes --write-tree a mandatory argument when trying to use
> that mode, right?  If so, that is not a simpler way to do what I'm
> trying to do at all; it breaks my intended usage.
>
> --write-tree is a documentation-only construct that users should never
> have to pass.
>
> Also, what happens if we remove the --trivial-merge flag and its whole
> mode after a sufficient deprecation period?  Would the --write-tree
> parameter remain required in your model to select the only existing
> mode, simply due to us having gone through a transition period?

You can have your cake and eat it too by running parse_optionss() N
number of times. Although perhaps in this case the end result isn't
worth it.

I was hoping this could be a simpler case of a subcommand dispatch, and
perhaps it can still be generalized to that.

If the "trivial" mode never takes options and always 3 argv elements, we
could just run parse_options() for it with no options, after checking
that we have 3 arguments, and none start with '-'.

But the below is a generalization of this I tried out just now, it
passes all your tests, and means that whenever you add new options you
don't need to keep saying "no, not with the trivial mode" for each one.

Basically we run parse_options() once with the full set of options, and
save away argc/argv (note the lack of strvec_clear() there, that's a
TODO memory leak).

Then we've got a o.mode, which along with argc is the *only* thing we
pay attention to at that point.

Then we dispatch to the "trivial" or "write" functions, which do
parse_options() again, this time with only their options.

It means that e.g. this now works as expected:
    
    ./git merge-tree --trivial-merge -z origin/{master,next,seen}
    error: unknown switch `z'
    usage: git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>
    
        --trivial-merge       do a trivial merge only

I.e. we error out, with your verison we'll just ignore the -z.

Your "--trivial-merge is incompatible with all other options" doesn't
work as you expect, and is buggy whether you want to go this route or
not, as the added tests show.

Basically it does nothing at all. Because if you add --foo we'll die
before we get there, parse_options() will die for us.

But if you do --trivial-merge -z your argc/argv will be trimmed, because
-z is a known option. So your check is doing nothing. Your tests also
pass with this removal of the only option compatibily check on top:
	
	diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
	index 58c0ddc5a32..08f18d43334 100644
	--- a/builtin/merge-tree.c
	+++ b/builtin/merge-tree.c
	@@ -476,7 +476,6 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
	 {
	 	struct merge_tree_options o = { .show_messages = -1 };
	 	int expected_remaining_argc;
	-	int original_argc;
	 
	 	const char * const merge_tree_usage[] = {
	 		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
	@@ -505,7 +504,6 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
	 	};
	 
	 	/* Parse arguments */
	-	original_argc = argc;
	 	argc = parse_options(argc, argv, prefix, mt_options,
	 			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
	 	if (o.mode) {
	@@ -517,8 +515,6 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
	 			usage_with_options(merge_tree_usage, mt_options);
	 		o.mode = (argc == 2 ? 'w' : 't');
	 	}
	-	if (o.mode == 't' && original_argc < argc)
	-		die(_("--trivial-merge is incompatible with all other options"));
	 
	 	/* Do the relevant type of merge */
	 	if (o.mode == 'w')

So, here it is in all its glory :) A bit nasty for sure, but IMO
preferrable to an ever expanding list of "X isn't compatible with A".

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 58c0ddc5a32..1d47912816d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -12,6 +12,7 @@
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 #include "quote.h"
+#include "strvec.h"
 
 static int line_termination = '\n';
 
@@ -371,13 +372,66 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(const char *base,
-			 const char *branch1,
-			 const char *branch2)
+struct merge_tree_options {
+	int mode;
+	int allow_unrelated_histories;
+	int show_messages;
+	int exclude_modes_oids_stages;
+};
+
+#define BUILTIN_MERGE_TREE_USAGE_WRITE \
+		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>")
+#define BUILTIN_MERGE_TREE_USAGE_TRIVIAL \
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>")
+
+#define BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL \
+		OPT_CMDMODE(0, "trivial-merge", &o.mode, \
+			    N_("do a trivial merge only"), 't')
+
+#define BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE \
+		OPT_CMDMODE(0, "write-tree", &o.mode, \
+			    N_("do a real merge instead of a trivial merge"), \
+			    'w')
+
+#define BUILTIN_MERGE_TREE_OPT_WRITE \
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE, \
+		OPT_BOOL(0, "messages", &o.show_messages, \
+			 N_("also show informational/conflict messages")), \
+		OPT_SET_INT('z', NULL, &line_termination, \
+			    N_("separate paths with the NUL character"), '\0'), \
+		OPT_BOOL_F('l', "exclude-modes-oids-stages", \
+			   &o.exclude_modes_oids_stages, \
+			   N_("list conflicted files without modes/oids/stages"), \
+			   PARSE_OPT_NONEG), \
+		OPT_BOOL_F(0, "allow-unrelated-histories", \
+			   &o.allow_unrelated_histories, \
+			   N_("allow merging unrelated histories"), \
+			   PARSE_OPT_NONEG)
+
+static int trivial_merge(int argc, const char **argv, const char *prefix)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
+	struct merge_tree_options o = { 0 };
+	const char * const usage[] = {
+		BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
+		NULL,
+	};
+	struct option options[] = {
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
+		OPT_END()
+	};
+	const char *base, *branch1, *branch2;
+
+	argc = parse_options(argc, argv, prefix, options, usage,
+			     PARSE_OPT_STOP_AT_NON_OPTION);
+	if (argc != 3)
+		BUG("should have ensured remaining argc == 3 already! Got %d", argc);
+
+	base = argv[0];
+	branch1 = argv[1];
+	branch2 = argv[2];;
 
 	buf1 = get_tree_descriptor(r, t+0, base);
 	buf2 = get_tree_descriptor(r, t+1, branch1);
@@ -391,24 +445,34 @@ static int trivial_merge(const char *base,
 	return 0;
 }
 
-struct merge_tree_options {
-	int mode;
-	int allow_unrelated_histories;
-	int show_messages;
-	int exclude_modes_oids_stages;
-};
-
-static int real_merge(struct merge_tree_options *o,
-		      const char *branch1, const char *branch2,
-		      const char *prefix)
+static int real_merge(int argc, const char **argv, const char *prefix)
 {
 	struct commit *parent1, *parent2;
 	struct commit_list *common;
 	struct commit_list *merge_bases = NULL;
 	struct commit_list *j;
+	struct merge_tree_options o = { .show_messages = 1 };
 	struct merge_options opt;
 	struct merge_result result = { 0 };
 
+	const char * const usage[] = {
+		BUILTIN_MERGE_TREE_USAGE_WRITE,
+		NULL,
+	};
+	struct option options[] = {
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
+		BUILTIN_MERGE_TREE_OPT_WRITE,
+		OPT_END()
+	};
+	const char *branch1, *branch2;
+
+	argc = parse_options(argc, argv, prefix, options, usage,
+			     PARSE_OPT_STOP_AT_NON_OPTION);
+	if (argc != 2)
+		BUG("should have ensured remaining argc == 2 already! Got %d", argc);
+	branch1 = argv[0];
+	branch2 = argv[1];
+
 	parent1 = get_merge_parent(branch1);
 	if (!parent1)
 		help_unknown_ref(branch1, "merge-tree",
@@ -431,7 +495,7 @@ static int real_merge(struct merge_tree_options *o,
 	 * merge_incore_recursive in merge-ort.h
 	 */
 	common = get_merge_bases(parent1, parent2);
-	if (!common && !o->allow_unrelated_histories)
+	if (!common && !o.allow_unrelated_histories)
 		die(_("refusing to merge unrelated histories"));
 	for (j = common; j; j = j->next)
 		commit_list_insert(j->item, &merge_bases);
@@ -440,8 +504,8 @@ static int real_merge(struct merge_tree_options *o,
 	if (result.clean < 0)
 		die(_("failure to merge"));
 
-	if (o->show_messages == -1)
-		o->show_messages = !result.clean;
+	if (o.show_messages == -1)
+		o.show_messages = !result.clean;
 
 	puts(oid_to_hex(&result.tree->object.oid));
 	if (!result.clean) {
@@ -453,7 +517,7 @@ static int real_merge(struct merge_tree_options *o,
 		for (i = 0; i < conflicted_files.nr; i++) {
 			const char *name = conflicted_files.items[i].string;
 			struct stage_info *c = conflicted_files.items[i].util;
-			if (!o->exclude_modes_oids_stages)
+			if (!o.exclude_modes_oids_stages)
 				printf("%06o %s %d\t",
 				       c->mode, oid_to_hex(&c->oid), c->stage);
 			else if (last && !strcmp(last, name))
@@ -464,7 +528,7 @@ static int real_merge(struct merge_tree_options *o,
 		}
 		string_list_clear(&conflicted_files, 1);
 	}
-	if (o->show_messages) {
+	if (o.show_messages) {
 		putchar(line_termination);
 		merge_display_update_messages(&opt, &result, stdout);
 	}
@@ -474,40 +538,32 @@ static int real_merge(struct merge_tree_options *o,
 
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	struct merge_tree_options o = { .show_messages = -1 };
+	struct merge_tree_options o;
 	int expected_remaining_argc;
-	int original_argc;
-
+	int original_argc = argc;
+	struct strvec original_args = STRVEC_INIT;
 	const char * const merge_tree_usage[] = {
-		N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
-		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		BUILTIN_MERGE_TREE_USAGE_WRITE,
+		BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
 		NULL
 	};
 	struct option mt_options[] = {
-		OPT_CMDMODE(0, "write-tree", &o.mode,
-			    N_("do a real merge instead of a trivial merge"),
-			    'w'),
-		OPT_CMDMODE(0, "trivial-merge", &o.mode,
-			    N_("do a trivial merge only"), 't'),
-		OPT_BOOL(0, "messages", &o.show_messages,
-			 N_("also show informational/conflict messages")),
-		OPT_SET_INT('z', NULL, &line_termination,
-			    N_("separate paths with the NUL character"), '\0'),
-		OPT_BOOL_F('l', "exclude-modes-oids-stages",
-			   &o.exclude_modes_oids_stages,
-			   N_("list conflicted files without modes/oids/stages"),
-			   PARSE_OPT_NONEG),
-		OPT_BOOL_F(0, "allow-unrelated-histories",
-			   &o.allow_unrelated_histories,
-			   N_("allow merging unrelated histories"),
-			   PARSE_OPT_NONEG),
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
+		BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
+		BUILTIN_MERGE_TREE_OPT_WRITE,
 		OPT_END()
 	};
 
+	/* We only care about deciding "o.mode" here */
+	o.mode = 0;
+	/*
+	 * We need our original argv, and
+	 * PARSE_OPT_KEEP_{ARGV0,UNKNOWN} would do the wrong thing
+	 */
+	strvec_pushv(&original_args, argv);
 	/* Parse arguments */
-	original_argc = argc;
 	argc = parse_options(argc, argv, prefix, mt_options,
-			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+			     merge_tree_usage, 0);
 	if (o.mode) {
 		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
 		if (argc != expected_remaining_argc)
@@ -517,12 +573,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 			usage_with_options(merge_tree_usage, mt_options);
 		o.mode = (argc == 2 ? 'w' : 't');
 	}
-	if (o.mode == 't' && original_argc < argc)
-		die(_("--trivial-merge is incompatible with all other options"));
 
 	/* Do the relevant type of merge */
 	if (o.mode == 'w')
-		return real_merge(&o, argv[0], argv[1], prefix);
+		return real_merge(original_argc, original_args.v, prefix);
 	else
-		return trivial_merge(argv[0], argv[1], argv[2]);
+		return trivial_merge(original_argc, original_args.v, prefix);
 }
diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
index 4de089d976d..749bdb6862d 100755
--- a/t/t4301-merge-tree-write-tree.sh
+++ b/t/t4301-merge-tree-write-tree.sh
@@ -92,6 +92,18 @@ test_expect_success 'Barf on too many arguments' '
 	grep "^usage: git merge-tree" expect
 '
 
+for opt in $(git merge-tree --git-completion-helper-all)
+do
+	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
+	then
+		continue
+	fi
+
+	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
+		test_expect_code 129 git merge-tree --trivial-merge $opt side1 side2 side3
+	'
+done
+
 test_expect_success 'test conflict notices and such' '
 	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-03 17:15                 ` Ævar Arnfjörð Bjarmason
@ 2022-02-03 18:18                   ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-02-03 18:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Johannes Sixt

On Thu, Feb 3, 2022 at 9:38 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> On Thu, Feb 03 2022, Elijah Newren wrote:
>
[...]
> > But that makes --write-tree a mandatory argument when trying to use
> > that mode, right?  If so, that is not a simpler way to do what I'm
> > trying to do at all; it breaks my intended usage.
> >
> > --write-tree is a documentation-only construct that users should never
> > have to pass.
> >
> > Also, what happens if we remove the --trivial-merge flag and its whole
> > mode after a sufficient deprecation period?  Would the --write-tree
> > parameter remain required in your model to select the only existing
> > mode, simply due to us having gone through a transition period?
>
> You can have your cake and eat it too by running parse_optionss() N
> number of times. Although perhaps in this case the end result isn't
> worth it.
>
> I was hoping this could be a simpler case of a subcommand dispatch, and
> perhaps it can still be generalized to that.
>
> If the "trivial" mode never takes options and always 3 argv elements, we
> could just run parse_options() for it with no options, after checking
> that we have 3 arguments, and none start with '-'.
>
> But the below is a generalization of this I tried out just now, it
> passes all your tests, and means that whenever you add new options you
> don't need to keep saying "no, not with the trivial mode" for each one.
>
> Basically we run parse_options() once with the full set of options, and
> save away argc/argv (note the lack of strvec_clear() there, that's a
> TODO memory leak).
>
> Then we've got a o.mode, which along with argc is the *only* thing we
> pay attention to at that point.
>
> Then we dispatch to the "trivial" or "write" functions, which do
> parse_options() again, this time with only their options.
>
> It means that e.g. this now works as expected:
>
>     ./git merge-tree --trivial-merge -z origin/{master,next,seen}
>     error: unknown switch `z'
>     usage: git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>
>
>         --trivial-merge       do a trivial merge only
>
> I.e. we error out, with your verison we'll just ignore the -z.
>
> Your "--trivial-merge is incompatible with all other options" doesn't
> work as you expect, and is buggy whether you want to go this route or
> not, as the added tests show.
>
> Basically it does nothing at all. Because if you add --foo we'll die
> before we get there, parse_options() will die for us.
>
> But if you do --trivial-merge -z your argc/argv will be trimmed, because
> -z is a known option. So your check is doing nothing.

Gah, good catch.  How did I get the check reversed??  I know I tested
it at one point.  Anyway, this fixes it:

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 58c0ddc5a3..fb25e3d10d 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -505,19 +505,22 @@ int cmd_merge_tree(int argc, const char **argv, const cha
r *prefix)
        };

        /* Parse arguments */
-       original_argc = argc;
+       original_argc = argc - 1; /* ignoring program name, i.e. argv[0] */
        argc = parse_options(argc, argv, prefix, mt_options,
                             merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
        if (o.mode) {
                expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
                if (argc != expected_remaining_argc)
                        usage_with_options(merge_tree_usage, mt_options);
+               if (o.mode == 't')
+                       /* Removal of `--trivial-merge` is expected */
+                       original_argc_options--;
        } else {
                if (argc < 2 || argc > 3)
                        usage_with_options(merge_tree_usage, mt_options);
                o.mode = (argc == 2 ? 'w' : 't');
        }
-       if (o.mode == 't' && original_argc < argc)
+       if (o.mode == 't' && argc < original_argc)
                die(_("--trivial-merge is incompatible with all other
options"));

        /* Do the relevant type of merge */


and results in this error being shown when any other option is listed
with --trivial-merge.

> So, here it is in all its glory :) A bit nasty for sure, but IMO
> preferrable to an ever expanding list of "X isn't compatible with A".

Agreed...on both counts.  :-)

> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 58c0ddc5a32..1d47912816d 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -12,6 +12,7 @@
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>  #include "quote.h"
> +#include "strvec.h"
>
>  static int line_termination = '\n';
>
> @@ -371,13 +372,66 @@ static void *get_tree_descriptor(struct repository *r,
>         return buf;
>  }
>
> -static int trivial_merge(const char *base,
> -                        const char *branch1,
> -                        const char *branch2)
> +struct merge_tree_options {
> +       int mode;
> +       int allow_unrelated_histories;
> +       int show_messages;
> +       int exclude_modes_oids_stages;
> +};
> +
> +#define BUILTIN_MERGE_TREE_USAGE_WRITE \
> +               N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>")
> +#define BUILTIN_MERGE_TREE_USAGE_TRIVIAL \
> +               N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>")
> +
> +#define BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL \
> +               OPT_CMDMODE(0, "trivial-merge", &o.mode, \
> +                           N_("do a trivial merge only"), 't')
> +
> +#define BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE \
> +               OPT_CMDMODE(0, "write-tree", &o.mode, \
> +                           N_("do a real merge instead of a trivial merge"), \
> +                           'w')
> +
> +#define BUILTIN_MERGE_TREE_OPT_WRITE \
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE, \
> +               OPT_BOOL(0, "messages", &o.show_messages, \
> +                        N_("also show informational/conflict messages")), \
> +               OPT_SET_INT('z', NULL, &line_termination, \
> +                           N_("separate paths with the NUL character"), '\0'), \
> +               OPT_BOOL_F('l', "exclude-modes-oids-stages", \
> +                          &o.exclude_modes_oids_stages, \
> +                          N_("list conflicted files without modes/oids/stages"), \
> +                          PARSE_OPT_NONEG), \
> +               OPT_BOOL_F(0, "allow-unrelated-histories", \
> +                          &o.allow_unrelated_histories, \
> +                          N_("allow merging unrelated histories"), \
> +                          PARSE_OPT_NONEG)
> +
> +static int trivial_merge(int argc, const char **argv, const char *prefix)
>  {
>         struct repository *r = the_repository;
>         struct tree_desc t[3];
>         void *buf1, *buf2, *buf3;
> +       struct merge_tree_options o = { 0 };
> +       const char * const usage[] = {
> +               BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
> +               NULL,
> +       };
> +       struct option options[] = {
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
> +               OPT_END()
> +       };
> +       const char *base, *branch1, *branch2;
> +
> +       argc = parse_options(argc, argv, prefix, options, usage,
> +                            PARSE_OPT_STOP_AT_NON_OPTION);
> +       if (argc != 3)
> +               BUG("should have ensured remaining argc == 3 already! Got %d", argc);
> +
> +       base = argv[0];
> +       branch1 = argv[1];
> +       branch2 = argv[2];;
>
>         buf1 = get_tree_descriptor(r, t+0, base);
>         buf2 = get_tree_descriptor(r, t+1, branch1);
> @@ -391,24 +445,34 @@ static int trivial_merge(const char *base,
>         return 0;
>  }
>
> -struct merge_tree_options {
> -       int mode;
> -       int allow_unrelated_histories;
> -       int show_messages;
> -       int exclude_modes_oids_stages;
> -};
> -
> -static int real_merge(struct merge_tree_options *o,
> -                     const char *branch1, const char *branch2,
> -                     const char *prefix)
> +static int real_merge(int argc, const char **argv, const char *prefix)
>  {
>         struct commit *parent1, *parent2;
>         struct commit_list *common;
>         struct commit_list *merge_bases = NULL;
>         struct commit_list *j;
> +       struct merge_tree_options o = { .show_messages = 1 };
>         struct merge_options opt;
>         struct merge_result result = { 0 };
>
> +       const char * const usage[] = {
> +               BUILTIN_MERGE_TREE_USAGE_WRITE,
> +               NULL,
> +       };
> +       struct option options[] = {
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
> +               BUILTIN_MERGE_TREE_OPT_WRITE,
> +               OPT_END()
> +       };
> +       const char *branch1, *branch2;
> +
> +       argc = parse_options(argc, argv, prefix, options, usage,
> +                            PARSE_OPT_STOP_AT_NON_OPTION);
> +       if (argc != 2)
> +               BUG("should have ensured remaining argc == 2 already! Got %d", argc);
> +       branch1 = argv[0];
> +       branch2 = argv[1];
> +
>         parent1 = get_merge_parent(branch1);
>         if (!parent1)
>                 help_unknown_ref(branch1, "merge-tree",
> @@ -431,7 +495,7 @@ static int real_merge(struct merge_tree_options *o,
>          * merge_incore_recursive in merge-ort.h
>          */
>         common = get_merge_bases(parent1, parent2);
> -       if (!common && !o->allow_unrelated_histories)
> +       if (!common && !o.allow_unrelated_histories)
>                 die(_("refusing to merge unrelated histories"));
>         for (j = common; j; j = j->next)
>                 commit_list_insert(j->item, &merge_bases);
> @@ -440,8 +504,8 @@ static int real_merge(struct merge_tree_options *o,
>         if (result.clean < 0)
>                 die(_("failure to merge"));
>
> -       if (o->show_messages == -1)
> -               o->show_messages = !result.clean;
> +       if (o.show_messages == -1)
> +               o.show_messages = !result.clean;
>
>         puts(oid_to_hex(&result.tree->object.oid));
>         if (!result.clean) {
> @@ -453,7 +517,7 @@ static int real_merge(struct merge_tree_options *o,
>                 for (i = 0; i < conflicted_files.nr; i++) {
>                         const char *name = conflicted_files.items[i].string;
>                         struct stage_info *c = conflicted_files.items[i].util;
> -                       if (!o->exclude_modes_oids_stages)
> +                       if (!o.exclude_modes_oids_stages)
>                                 printf("%06o %s %d\t",
>                                        c->mode, oid_to_hex(&c->oid), c->stage);
>                         else if (last && !strcmp(last, name))
> @@ -464,7 +528,7 @@ static int real_merge(struct merge_tree_options *o,
>                 }
>                 string_list_clear(&conflicted_files, 1);
>         }
> -       if (o->show_messages) {
> +       if (o.show_messages) {
>                 putchar(line_termination);
>                 merge_display_update_messages(&opt, &result, stdout);
>         }
> @@ -474,40 +538,32 @@ static int real_merge(struct merge_tree_options *o,
>
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -       struct merge_tree_options o = { .show_messages = -1 };
> +       struct merge_tree_options o;
>         int expected_remaining_argc;
> -       int original_argc;
> -
> +       int original_argc = argc;
> +       struct strvec original_args = STRVEC_INIT;
>         const char * const merge_tree_usage[] = {
> -               N_("git merge-tree [--write-tree] [<options>] <branch1> <branch2>"),
> -               N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +               BUILTIN_MERGE_TREE_USAGE_WRITE,
> +               BUILTIN_MERGE_TREE_USAGE_TRIVIAL,
>                 NULL
>         };
>         struct option mt_options[] = {
> -               OPT_CMDMODE(0, "write-tree", &o.mode,
> -                           N_("do a real merge instead of a trivial merge"),
> -                           'w'),
> -               OPT_CMDMODE(0, "trivial-merge", &o.mode,
> -                           N_("do a trivial merge only"), 't'),
> -               OPT_BOOL(0, "messages", &o.show_messages,
> -                        N_("also show informational/conflict messages")),
> -               OPT_SET_INT('z', NULL, &line_termination,
> -                           N_("separate paths with the NUL character"), '\0'),
> -               OPT_BOOL_F('l', "exclude-modes-oids-stages",
> -                          &o.exclude_modes_oids_stages,
> -                          N_("list conflicted files without modes/oids/stages"),
> -                          PARSE_OPT_NONEG),
> -               OPT_BOOL_F(0, "allow-unrelated-histories",
> -                          &o.allow_unrelated_histories,
> -                          N_("allow merging unrelated histories"),
> -                          PARSE_OPT_NONEG),
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_TRIVIAL,
> +               BUILTIN_MERGE_TREE_OPT_CMDMODE_WRITE,
> +               BUILTIN_MERGE_TREE_OPT_WRITE,
>                 OPT_END()
>         };
>
> +       /* We only care about deciding "o.mode" here */
> +       o.mode = 0;
> +       /*
> +        * We need our original argv, and
> +        * PARSE_OPT_KEEP_{ARGV0,UNKNOWN} would do the wrong thing
> +        */
> +       strvec_pushv(&original_args, argv);
>         /* Parse arguments */
> -       original_argc = argc;
>         argc = parse_options(argc, argv, prefix, mt_options,
> -                            merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +                            merge_tree_usage, 0);
>         if (o.mode) {
>                 expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
>                 if (argc != expected_remaining_argc)
> @@ -517,12 +573,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>                         usage_with_options(merge_tree_usage, mt_options);
>                 o.mode = (argc == 2 ? 'w' : 't');
>         }
> -       if (o.mode == 't' && original_argc < argc)
> -               die(_("--trivial-merge is incompatible with all other options"));
>
>         /* Do the relevant type of merge */
>         if (o.mode == 'w')
> -               return real_merge(&o, argv[0], argv[1], prefix);
> +               return real_merge(original_argc, original_args.v, prefix);
>         else
> -               return trivial_merge(argv[0], argv[1], argv[2]);
> +               return trivial_merge(original_argc, original_args.v, prefix);
>  }


Yeah, that's waaay more complex than my code which basically just
needs 6 lines to check the incompatibility; the basic patch for that
part (going back to before this patch we're commenting on) was just:

 +    original_argc = argc - 1;  /* ignore program name, i.e. argv[0] */
...
 +        if (o.mode == 't')
 +            /* Removal of `--trivial-merge` is expected */
 +            original_argc_options--;
...
 +    if (o.mode == 't' && argc < original_argc)
 +        die(_("--trivial-merge is incompatible with all other options"));


> diff --git a/t/t4301-merge-tree-write-tree.sh b/t/t4301-merge-tree-write-tree.sh
> index 4de089d976d..749bdb6862d 100755
> --- a/t/t4301-merge-tree-write-tree.sh
> +++ b/t/t4301-merge-tree-write-tree.sh
> @@ -92,6 +92,18 @@ test_expect_success 'Barf on too many arguments' '
>         grep "^usage: git merge-tree" expect
>  '
>
> +for opt in $(git merge-tree --git-completion-helper-all)
> +do
> +       if test $opt = "--trivial-merge" || test $opt = "--write-tree"
> +       then
> +               continue
> +       fi
> +
> +       test_expect_success "usage: --trivial-merge is incompatible with $opt" '
> +               test_expect_code 129 git merge-tree --trivial-merge $opt side1 side2 side3
> +       '
> +done

Yep, I was clearly missing an important testcase.  Thanks for
providing one and for pointing out the error in my patch!  Very cool.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-03 10:42               ` Johannes Altmanninger
  2022-02-03 16:54                 ` Elijah Newren
@ 2022-02-03 20:05                 ` Junio C Hamano
  1 sibling, 0 replies; 215+ messages in thread
From: Junio C Hamano @ 2022-02-03 20:05 UTC (permalink / raw)
  To: Johannes Altmanninger
  Cc: Elijah Newren, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Ramsay Jones, Johannes Schindelin,
	Christian Couder, René Scharfe,
	Ævar Arnfjörð Bjarmason, Johannes Sixt

Johannes Altmanninger <aclopte@gmail.com> writes:

> I wonder how the UI of a tool that does non-worktree/non-index cherry-picks
> will look like.  I'd expect it to produce the same output as merge-tree,
> except cherry-pick should probably output a commit OID, not a tree.

A tool that creates a "merge", "cherry-pick" and "revert" should all
output a resulting commit object name, not a tree object.

And "merge-tree" that creates a three-way merge to apply a change to
go from tree A to tree B on top of tree X should take three tree-ish
object names and produce a resulting one tree object name.

Using the "merge-tree" that merges two trees using the third tree as
a common ancestor, these three higher-level tools can perform
"merge", "cherry-pick", and "revert".  They would all take two
commit objects and produce one commit object.



   

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts
  2022-01-29 18:07   ` [PATCH v2 10/13] merge-tree: provide a list of which files have conflicts Elijah Newren via GitGitGadget
  2022-02-02 21:32     ` Junio C Hamano
  2022-02-02 21:32     ` Junio C Hamano
@ 2022-02-03 23:55     ` Junio C Hamano
  2 siblings, 0 replies; 215+ messages in thread
From: Junio C Hamano @ 2022-02-03 23:55 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +Conflicted file list
> +~~~~~~~~~~~~~~~~~~~~
> +
> +This is a sequence of lines containing a filename on each line, quoted
> +as explained for the configuration variable `core.quotePath` (see
> +linkgit:git-config[1]).

Makes sense.  Ideally things like this should be discoverable by
inspecting the tree object shown as the result of the (conflicted)
merge, but since the design of the output is to show only a single
tree, there is nowhere to store such an extra piece of information
per path (grepping for markers in blobs of course does not count).

I guess an alternative to show four trees when conflicted instead of
one (i.e. the primary tree may either contain only the cleanly
merged paths _or_ also blobs with conflict markers for conflicted
paths; the three other trees record three stages that would be in
the index, if we were performing the same merge using the index),
but a machine-parseable list of paths is fine.

> +		merge_get_conflicted_files(&result, &conflicted_files);
> +		for (i = 0; i < conflicted_files.nr; i++) {
> +			const char *name = conflicted_files.items[i].string;
> +			if (last && !strcmp(last, name))
> +				continue;
> +			write_name_quoted_relative(
> +				name, prefix, stdout, line_termination);
> +			last = name;

OK.  The iteration used here makes casual readers wonder why the
helper doesn't make paths unique, but the string list item holds
in its util pointer a pointer to a structure with <stage, mode, oid>
tuple, so it is natural to make the consumer, who wants uniquified
list, responsible for deduping, like this loop.

> +		}
> +		string_list_clear(&conflicted_files, 1);

And the stage-info structure associated with these paths are
deallocated with this call.  Good.

> +	}


^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-02  7:34     ` [PATCH v3 04/15] merge-tree: implement real merges Elijah Newren via GitGitGadget
  2022-02-02 21:22       ` Junio C Hamano
@ 2022-02-04  4:48       ` Josh Steadmon
  2022-02-04  6:08         ` Elijah Newren
  1 sibling, 1 reply; 215+ messages in thread
From: Josh Steadmon @ 2022-02-04  4:48 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

On 2022.02.02 07:34, Elijah Newren via GitGitGadget wrote:
> diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> index 58731c19422..569485815a0 100644
> --- a/Documentation/git-merge-tree.txt
> +++ b/Documentation/git-merge-tree.txt
> @@ -3,26 +3,73 @@ git-merge-tree(1)
>  
>  NAME
>  ----
> -git-merge-tree - Show three-way merge without touching index
> +git-merge-tree - Perform merge without touching index or working tree
>  
>  
>  SYNOPSIS
>  --------
>  [verse]
> -'git merge-tree' <base-tree> <branch1> <branch2>
> +'git merge-tree' [--write-tree] <branch1> <branch2>
> +'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
>  
>  DESCRIPTION
>  -----------
> -Reads three tree-ish, and output trivial merge results and
> -conflicting stages to the standard output.  This is similar to
> -what three-way 'git read-tree -m' does, but instead of storing the
> -results in the index, the command outputs the entries to the
> -standard output.
> -
> -This is meant to be used by higher level scripts to compute
> -merge results outside of the index, and stuff the results back into the
> -index.  For this reason, the output from the command omits
> -entries that match the <branch1> tree.
> +
> +Performs a merge, but does not make any new commits and does not read
> +from or write to either the working tree or index.
> +
> +The second form is deprecated and supported only for backward
> +compatibility.  It will likely be removed in the future, and will not
> +be discussed further in this manual.
> +
> +The first form will merge the two branches, doing a real merge.  A real
> +merge is distinguished from a trivial merge in that it includes:
> +
> +  * three way content merges of individual files
> +  * rename detection
> +  * proper directory/file conflict handling
> +  * recursive ancestor consolidation (i.e. when there is more than one
> +    merge base, creating a virtual merge base by merging the merge bases)
> +  * etc.
> +
> +After the merge completes, it will create a new toplevel tree object.
> +See `OUTPUT` below for details.
> +
> +OUTPUT
> +------
> +
> +For either a successful or conflicted merge, the output from
> +git-merge-tree is simply one line:
> +
> +	<OID of toplevel tree>
> +
> +The printed tree object corresponds to what would be checked out in
> +the working tree at the end of `git merge`, and thus may have files
> +with conflict markers in them.
> +
> +EXIT STATUS
> +-----------
> +
> +For a successful, non-conflicted merge, the exit status is 0.  When the
> +merge has conflicts, the exit status is 1.  If the merge is not able to
> +complete (or start) due to some kind of error, the exit status is
> +something other than 0 or 1.
> +
> +USAGE NOTES
> +-----------
> +
> +git-merge-tree was written to be low-level plumbing, similar to
> +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
> +be used as a part of a series of steps such as
> +
> +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> +       test $? -eq 0 || die "There were conflicts..."
> +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> +       git update-ref $BRANCH1 $NEWCOMMIT
> +
> +However, it does not quite fit into the same category of low-level
> +plumbing commands since the possibility of merge conflicts give it a
> +much higher chance of the command not succeeding.

I found this final paragraph confusing. It seems to be hinting at some
conclusion it expects readers to make, but I haven't been able to figure
out what. Could this be made more explicit, or perhaps dropped
altogether?

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 04/15] merge-tree: implement real merges
  2022-02-04  4:48       ` Josh Steadmon
@ 2022-02-04  6:08         ` Elijah Newren
  0 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren @ 2022-02-04  6:08 UTC (permalink / raw)
  To: Josh Steadmon, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

On Thu, Feb 3, 2022 at 8:48 PM Josh Steadmon <steadmon@google.com> wrote:
>
> On 2022.02.02 07:34, Elijah Newren via GitGitGadget wrote:
[...]
> > +USAGE NOTES
> > +-----------
> > +
> > +git-merge-tree was written to be low-level plumbing, similar to
> > +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
> > +be used as a part of a series of steps such as
> > +
> > +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> > +       test $? -eq 0 || die "There were conflicts..."
> > +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> > +       git update-ref $BRANCH1 $NEWCOMMIT
> > +
> > +However, it does not quite fit into the same category of low-level
> > +plumbing commands since the possibility of merge conflicts give it a
> > +much higher chance of the command not succeeding.
>
> I found this final paragraph confusing. It seems to be hinting at some
> conclusion it expects readers to make, but I haven't been able to figure
> out what. Could this be made more explicit, or perhaps dropped
> altogether?

Yep, I'll drop it.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-01-29 16:47         ` Elijah Newren
@ 2022-02-04 23:10           ` Johannes Schindelin
  2022-02-05  0:54             ` Elijah Newren
  0 siblings, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-02-04 23:10 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Johannes Sixt, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

Hi Elijah,

On Sat, 29 Jan 2022, Elijah Newren wrote:

> On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
> >
> > Just a heckling from the peanut gallery...
> >
> > Am 29.01.22 um 07:08 schrieb Elijah Newren:
> > > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> > > <Johannes.Schindelin@gmx.de> wrote:
> > >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> > >> missing from the second conflict, in the output we would see stages 1, 2,
> > >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> > >> different conflicts.
> > >
> > > I don't understand why you're fixating on the stage here.  Why would
> > > you want to group all the stage 2s together, count them up, and then
> > > determine there are N conflicting files because there are N stage 2's?
> >
> > Looks like you are misunderstanding Dscho's point: When you have two
> > conflicts, the first with stages 1 and 2, the second with stages 2 and
> > 3, then the 2s occur lumped together when the 4 lines are printed in a
> > row, and that is the cue to the parser where the new conflict begins.
> > Dscho did not mean that all N 2s of should be listed together.
>
> Ah, so...I didn't understand his misunderstanding?  Using stages as a
> cue to the parser where the new conflict begins is broken; you should
> instead check for when the filename listed on a line does not match
> the filename on the previous line.

But that would break down in case of rename/rename conflicts, right?

> In particular, if one conflict has stages 1 and 2, and the next conflict
> has only stage 3, then looking at stages only might cause you to
> accidentally lump unrelated conflicts together.

Precisely. That's why I would love to have a way to deviate from the
output of `ls-files -u`'s format, and have a reliable way to indicate
stages that belong to the same merge conflict.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 09/12] merge-tree: provide a list of which files have conflicts
  2022-01-29  6:21     ` Elijah Newren
@ 2022-02-04 23:12       ` Johannes Schindelin
       [not found]         ` <CABPp-BFyaakDSjHULpBRPQqq_jz2keyufHo1MjNS6dHQNR+JLQ@mail.gmail.com>
  0 siblings, 1 reply; 215+ messages in thread
From: Johannes Schindelin @ 2022-02-04 23:12 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

Hi Elijah,

On Fri, 28 Jan 2022, Elijah Newren wrote:

> On Fri, Jan 28, 2022 at 8:57 AM Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > Hi Elijah,
> >
> > On Sat, 22 Jan 2022, Elijah Newren via GitGitGadget wrote:
> >
> > > From: Elijah Newren <newren@gmail.com>
> > >
> [...]
> > > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > > index fd7a867de60..041a4ac2785 100644
> > > --- a/Documentation/git-merge-tree.txt
> > > +++ b/Documentation/git-merge-tree.txt
> > > @@ -58,6 +58,7 @@ simply one line:
> > >  Whereas for a conflicted merge, the output is by default of the form:
> > >
> > >       <OID of toplevel tree>
> > > +     <Conflicted file list>
> > >       <Informational messages>
> >
> > To distinguish between the list of conflicted files and the informational
> > messages, I think it would be good to insert an empty line, as a
> > separator, like.
>
> Yes, I agree; that's why I did so.  :-)

My concern was that I did not see this empty line reflected in the quoted
diff. I would have expected an empty line between the `<Conflicted [...]>`
and the `<Informational [...]>` line.

Thanks,
Dscho

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH 08/12] merge-ort: provide a merge_get_conflicted_files() helper function
  2022-02-04 23:10           ` Johannes Schindelin
@ 2022-02-05  0:54             ` Elijah Newren
  2022-02-21 10:46               ` Johannes Schindelin
  0 siblings, 1 reply; 215+ messages in thread
From: Elijah Newren @ 2022-02-05  0:54 UTC (permalink / raw)
  To: Johannes Schindelin
  Cc: Johannes Sixt, Elijah Newren via GitGitGadget, Git Mailing List,
	Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Christian Couder, René Scharfe

On Fri, Feb 4, 2022 at 3:10 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> On Sat, 29 Jan 2022, Elijah Newren wrote:
>
> > On Sat, Jan 29, 2022 at 12:23 AM Johannes Sixt <j6t@kdbg.org> wrote:
> > >
> > > Just a heckling from the peanut gallery...
> > >
> > > Am 29.01.22 um 07:08 schrieb Elijah Newren:
> > > > On Fri, Jan 28, 2022 at 8:55 AM Johannes Schindelin
> > > > <Johannes.Schindelin@gmx.de> wrote:
> > > >> Meaning: Even if stage 3 is missing from the first conflict and stage 1 is
> > > >> missing from the second conflict, in the output we would see stages 1, 2,
> > > >> 2, 3, i.e. a duplicate stage 2, signifying that we're talking about two
> > > >> different conflicts.
> > > >
> > > > I don't understand why you're fixating on the stage here.  Why would
> > > > you want to group all the stage 2s together, count them up, and then
> > > > determine there are N conflicting files because there are N stage 2's?
> > >
> > > Looks like you are misunderstanding Dscho's point: When you have two
> > > conflicts, the first with stages 1 and 2, the second with stages 2 and
> > > 3, then the 2s occur lumped together when the 4 lines are printed in a
> > > row, and that is the cue to the parser where the new conflict begins.
> > > Dscho did not mean that all N 2s of should be listed together.
> >
> > Ah, so...I didn't understand his misunderstanding?  Using stages as a
> > cue to the parser where the new conflict begins is broken; you should
> > instead check for when the filename listed on a line does not match
> > the filename on the previous line.
>
> But that would break down in case of rename/rename conflicts, right?
>
> > In particular, if one conflict has stages 1 and 2, and the next conflict
> > has only stage 3, then looking at stages only might cause you to
> > accidentally lump unrelated conflicts together.
>
> Precisely. That's why I would love to have a way to deviate from the
> output of `ls-files -u`'s format, and have a reliable way to indicate
> stages that belong to the same merge conflict.

Ah, attempting to somehow identify and present logical separate
conflicts?  That could be awesome, but I'm not sure it's technically
possible.  It certainly isn't with today's merge-ort.

Let me ask some questions first...

If I understand you correctly then in the event of a rename/rename,
i.e. foo->bar & foo->baz, then you want foo's, bar's, & baz's stages
all listed together.  Right?  And in some way that you can identify
them as related?

If we do so, how do we mark the beginning and the end of what you call
"the same merge conflict"?  If you say it's always 3 stages (with the
possibility of all-zero modes/oids), then what about the rename/rename
case above modified so that the side that did foo->baz also added a
different 'bar'?  That'd be 4 non-zero modes/oids, all of them
relevant.  Or what if the side that did foo->bar also renamed
something else to 'baz', giving us even more non-zero stages for these
three paths?  Perhaps you consider these different conflicts and want
them listed separately -- if so, where does one conflict begin and
another start and which stages are parts of which conflict?

If you are attempting to somehow present the stuff that "belongs to
the same merge conflict" are you also trying to identify what kind of
merge conflict it is?  If so, do you want each type of merge conflict
listed?  For example, let's switch from the example above of logically
disjoint paths coming together to result in more than 3 stages, and
instead pick an example with a single logical path with less than
three stages.  And now let's say that path has multiple conflicts
associated with it; let's use an example with 3: rename/delete +
modify/delete + directory/file (one side renames foo->bar while
modifying the contents, the other side deletes foo and adds the
directory 'bar/').  In this case, there is a target file 'bar' that
has two non-zero modes/oids in the ls-files-u output.  If all three
types of conflicts need to be listed, does each need to be listed with
the two non-zero modes/oids (and perhaps one zero mode/oid), resulting
in six listings for 'bar'?  Or would the duplication be confusing
enough that we instead decide to list some merge conflicts with no
stages associated with them?

Thinking about both sets of questions in the last two paragraphs from
a higher level -- should we focus on and group the higher order stages
by the individual conflicts that happen, or should we group them by
the paths that they happen to (which is what `ls-files -u` happens to
do), or should we not bother grouping them and instead duplicate the
higher order stages for each logical conflict it is part of?

As an alternative to duplicating higher order stages, do we sometimes
decide to "lump" separate conflicts together and treat them as one
conflict?  If so, what are the rules on how we decide to lump
conflicts and when not to?  Is there a bright line boundary?  And can
it be done without sucking in arbitrarily more stages for a single
conflict?


Some testcases that might be useful while considering the above
questions: take a look at the "rad", "rrdd", and "mod6" tests of
t6422.  How many "same merge conflicts" are there for each of those,
and what's the boundary between them?  And can you give the answer in
the form of rules that generically handle all cases, rather than just
answering these three specific cases?


I've thought about this problem long and hard before (in part because
of some conversations I had with Edward Thompson about libgit2 and
merging at Git Merge 2020).  It wasn't at all clear to me that libgit2
had considered anything beyond simple rename cases.  The only rules I
ever figured out that made sense to me was "group the stages by target
filename rather than by logical conflict" (so we get `ls -files -u`
populated) and print a meant-for-human message for each logical
conflict (found in the <Informational Messages> section for
merge-tree), and make NO attempt to connect stages by conflict type.

I'm sure that's not what you wanted to hear, and maybe doesn't even
play nicely with your design.  But short of ignoring the edge and
corner cases, I don't see how to solve that problem.  If you do just
want to ignore edge and corner cases, then just ignore the
rename/rename case you brought up in the first place and just use
`ls-files -u`-type output as-is within your design.  If you don't want
to ignore edge cases and want something that works with a specific
design that somehow groups conflicted file stages by conflict type,
then we're going to have to dig into all these questions above and do
some big replumbing within merge-ort.

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-02  7:34     ` [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
  2022-02-03  2:05       ` Ævar Arnfjörð Bjarmason
@ 2022-02-07 22:41       ` Emily Shaffer
  2022-02-07 23:36         ` Junio C Hamano
  1 sibling, 1 reply; 215+ messages in thread
From: Emily Shaffer @ 2022-02-07 22:41 UTC (permalink / raw)
  To: Elijah Newren via GitGitGadget
  Cc: git, Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt

On Wed, Feb 02, 2022 at 07:34:29AM +0000, Elijah Newren via GitGitGadget wrote:
> 
> 
> Let merge-tree accept a `--write-tree` parameter for choosing real
> merges instead of trivial merges, and accept an optional
> `--trivial-merge` option to get the traditional behavior.  Note that
> these accept different numbers of arguments, though, so these names
> need not actually be used.
> 
> Note that real merges differ from trivial merges in that they handle:
>   - three way content merges
>   - recursive ancestor consolidation
>   - renames
>   - proper directory/file conflict handling
>   - etc.
> Basically all the stuff you'd expect from `git merge`, just without
> updating the index and working tree.  The initial shell added here does
> nothing more than die with "real merges are not yet implemented", but
> that will be fixed in subsequent commits.
> 
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  builtin/merge-tree.c | 61 +++++++++++++++++++++++++++++++++++++-------
>  git.c                |  2 +-
>  2 files changed, 53 insertions(+), 10 deletions(-)
> 
> diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
> index 914ec960b7e..e98ec8a9f1d 100644
> --- a/builtin/merge-tree.c
> +++ b/builtin/merge-tree.c
> @@ -3,13 +3,12 @@
>  #include "tree-walk.h"
>  #include "xdiff-interface.h"
>  #include "object-store.h"
> +#include "parse-options.h"
>  #include "repository.h"
>  #include "blob.h"
>  #include "exec-cmd.h"
>  #include "merge-blobs.h"
>  
> -static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
> -
>  struct merge_list {
>  	struct merge_list *next;
>  	struct merge_list *link;	/* other stages for this object */
> @@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
>  	return buf;
>  }
>  
> -static int trivial_merge(int argc, const char **argv)
> +static int trivial_merge(const char *base,
> +			 const char *branch1,
> +			 const char *branch2)
>  {
>  	struct repository *r = the_repository;
>  	struct tree_desc t[3];
>  	void *buf1, *buf2, *buf3;
>  
> -	buf1 = get_tree_descriptor(r, t+0, argv[1]);
> -	buf2 = get_tree_descriptor(r, t+1, argv[2]);
> -	buf3 = get_tree_descriptor(r, t+2, argv[3]);
> +	buf1 = get_tree_descriptor(r, t+0, base);
> +	buf2 = get_tree_descriptor(r, t+1, branch1);
> +	buf3 = get_tree_descriptor(r, t+2, branch2);
>  	trivial_merge_trees(t, "");
>  	free(buf1);
>  	free(buf2);
> @@ -384,9 +385,51 @@ static int trivial_merge(int argc, const char **argv)
>  	return 0;
>  }
>  
> +struct merge_tree_options {
> +	int mode;
> +};
> +
> +static int real_merge(struct merge_tree_options *o,
> +		      const char *branch1, const char *branch2)
> +{
> +	die(_("real merges are not yet implemented"));
> +}
> +
>  int cmd_merge_tree(int argc, const char **argv, const char *prefix)
>  {
> -	if (argc != 4)
> -		usage(merge_tree_usage);
> -	return trivial_merge(argc, argv);
> +	struct merge_tree_options o = { 0 };
> +	int expected_remaining_argc;
> +
> +	const char * const merge_tree_usage[] = {
> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
> +		NULL
> +	};
> +	struct option mt_options[] = {
> +		OPT_CMDMODE(0, "write-tree", &o.mode,
> +			    N_("do a real merge instead of a trivial merge"),
> +			    'w'),
> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
> +			    N_("do a trivial merge only"), 't'),
> +		OPT_END()
> +	};

In the review club last week I had mentioned I thought OPT_CMDMODE
worked well with enums. I found some a reasonably nice example in
builtin/replace.c:cmd_replace(), although I have some Opinions about the
enum declaration placement there. Regardless, I think using an enum
instead of a single character would make this more readable - otherwise
I need to remember what 'w' means when I'm reasoning about how many args
to expect below.


> +
> +	/* Parse arguments */
> +	argc = parse_options(argc, argv, prefix, mt_options,
> +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
> +	if (o.mode) {
> +		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
> +		if (argc != expected_remaining_argc)
> +			usage_with_options(merge_tree_usage, mt_options);
> +	} else {
> +		if (argc < 2 || argc > 3)
> +			usage_with_options(merge_tree_usage, mt_options);
> +		o.mode = (argc == 2 ? 'w' : 't');
> +	}
> +
> +	/* Do the relevant type of merge */
> +	if (o.mode == 'w')
> +		return real_merge(&o, argv[0], argv[1]);
> +	else
> +		return trivial_merge(argv[0], argv[1], argv[2]);
>  }

Sorry for the slow reply.

 - Emily

^ permalink raw reply	[flat|nested] 215+ messages in thread

* Re: [PATCH v3 03/15] merge-tree: add option parsing and initial shell for real merge function
  2022-02-07 22:41       ` Emily Shaffer
@ 2022-02-07 23:36         ` Junio C Hamano
  0 siblings, 0 replies; 215+ messages in thread
From: Junio C Hamano @ 2022-02-07 23:36 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: Elijah Newren via GitGitGadget, git, Christian Couder,
	Taylor Blau, Johannes Altmanninger, Ramsay Jones,
	Johannes Schindelin, Christian Couder, René Scharfe,
	Ævar Arnfjörð Bjarmason, Elijah Newren,
	Johannes Sixt

Emily Shaffer <emilyshaffer@google.com> writes:

>> +	const char * const merge_tree_usage[] = {
>> +		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
>> +		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
>> +		NULL
>> +	};
>> +	struct option mt_options[] = {
>> +		OPT_CMDMODE(0, "write-tree", &o.mode,
>> +			    N_("do a real merge instead of a trivial merge"),
>> +			    'w'),
>> +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
>> +			    N_("do a trivial merge only"), 't'),
>> +		OPT_END()
>> +	};
>
> In the review club last week I had mentioned I thought OPT_CMDMODE
> worked well with enums. I found some a reasonably nice example in
> builtin/replace.c:cmd_replace(), although I have some Opinions about the
> enum declaration placement there. Regardless, I think using an enum
> instead of a single character would make this more readable - otherwise
> I need to remember what 'w' means when I'm reasoning about how many args
> to expect below.

I am reasonably sure whoever did the above use of OPT_CMDMODE()
feature mimicked an existing one that use it to make options with a
single-letter shorthand mutually exclusive.  If the options are with
short-hand, you wouldn't be complaining that you do not know what
'w' stands for.  When using it with options without short-hand, like
this case, I'd agree it would make it easier to read to use symbolic
constants of some kind.


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v4 00/12] In-core git merge-tree ("Server side merges")
  2022-02-02  7:34   ` [PATCH v3 00/15] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                       ` (14 preceding siblings ...)
  2022-02-02  7:34     ` [PATCH v3 15/15] git-merge-tree.txt: add a section on potentional usage mistakes Elijah Newren via GitGitGadget
@ 2022-02-12 20:34     ` Elijah Newren via GitGitGadget
  2022-02-12 20:34       ` [PATCH v4 01/12] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
                         ` (12 more replies)
  15 siblings, 13 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-12 20:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Josh Steadmon, Emily Shaffer,
	Elijah Newren

Note: Depends on en/remerge-diff (to avoid a small textual conflict)

== Basic Summary ==

This series introduces a new mode to git merge-tree allowing it to perform
real merges (three-way text content merges, recursive ancestor
consolidation, rename detection, proper directory/file conflict handling,
etc.) and write the result as a toplevel tree. It doesn't touch the working
tree or index, and doesn't create any commits or update any refs. It could
be used to do merges when in a bare repository (thus potentially making it
of interest to Git hosting sites, i.e. "Server side merges"), or for doing a
merge of branches that aren't checked out.

It does not handle similar functionality for cherry-picks, rebases, or
reverts; that is also of interest, but is being deferred for a future
series.

== Quick Overview ==

 * Patches 1-2: preparatory cleanups
 * Patches 3-4: implement basic real merges
 * Patches 5-6: include informational messages ("CONFLICT" messages and
   such) in output
 * Patches 7-10: add ability to include ls-files -u style of info in the
   output
 * Patch 11: support --allow-unrelated-histories
 * Patch 12: augment the manual with potential usage mistakes

== Updates Log ==

Many thanks to the many reviewers who provided good feedback on the most
recent round -- Junio, Ævar, Josh, Emily, and perhaps some others I've
forgotten from review club.

Updates since v3 (or v5, if you include the rounds at
https://lore.kernel.org/git/pull.1114.v2.git.git.1641403655.gitgitgadget@gmail.com/):

 * Dropped previous patches 5, 6, and 8 of the old series; they weren't
   being used and opened a can of worms[1]
 * [Patch 3] Restructured argument checking, including using an enum
 * [Patch 4] Restored the extended paragraph about the deprecated form of
   git-merge-tree, mentioned write-tree in plumbing commands, and a few
   other small fixups to the documentation
 * [Patch 4] Also provide an example of a clean merge rather than just a
   conflicted one
 * [Patch 6] Fix the incompatible arguments check and add some tests for it
 * [Patch 6] Introduce an anonymize_hash() shell function to make tests
   easier to read (less repeated sed)
 * [Patch 9] Rename --exclude-modes-oids-stages to --name-only; no short
   option for now
 * [Patch 10] When -z passed, the tree in the first section should have a
   trailing NUL rather than trailing newline [1]
   https://lore.kernel.org/git/CABPp-BEKuXHELVx4=5JJTj5HVOKZ=Y-4G4BK47BCZYYRSrkFsQ@mail.gmail.com/

Stuff NOT included that reviewers brought up in earlier rounds:

 * Very generic (mode, oid, stage, filename) printing formatting[2]
 * Always printing 3 stages for each filename with conflicts[3]
 * Attempting to group conflict stages by logical conflict rather than by
   affected target filepath[4]
 * Providing similar functionality for doing cherry-picks/rebases/reverts,
   i.e. a scheme for three-way merges with a specified merge-base[5]. That's
   being deferred to a future series. [2]
   https://lore.kernel.org/git/CABPp-BGnOes7J_piDyBUeuLVm274w4-9G3k0vR-0it3z7TPn_w@mail.gmail.com/
   [3]
   https://lore.kernel.org/git/CABPp-BG2rMEYBLuBW=0wtpJe4aUFGCFa8D0NTSKz9Sm+CkXPxw@mail.gmail.com/
   [4]
   https://lore.kernel.org/git/CABPp-BGCL0onSmpgKuO1k2spYCkx=v27ed9TSSxFib=OdDcLbw@mail.gmail.com/
   [5]
   https://lore.kernel.org/git/CABPp-BEaemkGGm0cSofP0gau7YN-y6HFoi0yJbHA8+iGjxsYSA@mail.gmail.com/

Updates since v2:

 * Improved patches from Dscho for the diff_warn_rename_limit() handling
 * Add a -z option for NUL-terminated conflict info lines (so that filenames
   do not have to be quoted)

Updates since v1 (or v3 depending on how you count; thanks to René, Ævar,
Christian, Dscho for very helpful feedback):

 * New patch from Dscho allowing diff_warn_rename_limit() to print somewhere
   other than stdout (I hope he's okay with me including his Signed-off-by)
 * Now prints filenames relative to prefix, much like ls-files
 * Renamed --exclude-oids-and-modes to --exclude-modes-oids-stages and gave
   it a -l shorthand; I'm wondering if I should just drop this option,
   though.
 * And numerous cleanups, in lots of areas:
   * Multiple parse-options cleanups
   * Lots of commit message cleanups
   * Wording tweaks to the "Description" section of the manual
   * Several small code cleanups
 * I dropped the RFC label

Updates since original submission v2 (thanks to Christian, Dscho, Ramsay,
and René for suggestions and comments):

 * Significant changes to output format:
   * Flags no longer take a filename for additional output; they write to
     stdout instead.
   * More information included by default when there are conflicts (no need
     to request it with additional flags, instead flags can be used to
     suppress it).
   * Provide (mode, oid, stage, file) tuples -- i.e. ls-files -u style of
     information -- when there are conflicts. Add a flag to only list
     conflicted files if that's preferred.
 * Much more thorough manual for git-merge-tree.txt
 * Renamed option from --real to --write-tree
 * Accept an optional --trivial-merge option to get old style merge-tree
   behavior
 * Allow both --write-tree and --trivial-merge to be omitted since we can
   deduce which from number of arguments
 * Document exit code when the merge cannot be run (so we can distinguish
   other error cases from conflicts)
 * testcase cleanups: test_tick, early skip of test when using recursive
   backend, variable renames, etc.
 * various minor code cleanups
 * Add a new --allow-unrelated-histories option (with same meaning as the
   one used in git merge)
 * Rebased on top of en/remerge-diff to avoid a small conflict

Updates since original submission v1 (thanks to Johannes Altmanninger and
Fabian for suggestions):

 * Fixed a bad patch splitting, and a style issue pointed out by Johannes
   Altimanninger
 * Fixed misleading commit messages in new test cases
 * Fixed my comments about how commit-tree could be used to correctly use
   two -p flags

Elijah Newren (12):
  merge-tree: rename merge_trees() to trivial_merge_trees()
  merge-tree: move logic for existing merge into new function
  merge-tree: add option parsing and initial shell for real merge
    function
  merge-tree: implement real merges
  merge-ort: split out a separate display_update_messages() function
  merge-tree: support including merge messages in output
  merge-ort: provide a merge_get_conflicted_files() helper function
  merge-tree: provide a list of which files have conflicts
  merge-tree: provide easy access to `ls-files -u` style info
  merge-tree: allow `ls-files -u` style info to be NUL terminated
  merge-tree: add a --allow-unrelated-histories flag
  git-merge-tree.txt: add a section on potentional usage mistakes

 Documentation/git-merge-tree.txt | 204 ++++++++++++++++++++++++--
 builtin/merge-tree.c             | 189 ++++++++++++++++++++++--
 git.c                            |   2 +-
 merge-ort.c                      | 109 +++++++++-----
 merge-ort.h                      |  29 ++++
 t/t4301-merge-tree-write-tree.sh | 238 +++++++++++++++++++++++++++++++
 6 files changed, 709 insertions(+), 62 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh


base-commit: ea5df61cf358d3c831189e2f04863abc2157e3e1
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1122%2Fnewren%2Fin-core-merge-tree-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1122/newren/in-core-merge-tree-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1122

Range-diff vs v3:

  1:  4a7cd5542bb =  1:  4a7cd5542bb merge-tree: rename merge_trees() to trivial_merge_trees()
  2:  4780ff6784d =  2:  4780ff6784d merge-tree: move logic for existing merge into new function
  3:  63f42df21ae !  3:  60253745f5c merge-tree: add option parsing and initial shell for real merge function
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
       	return 0;
       }
       
     ++enum mode {
     ++	MODE_UNKNOWN,
     ++	MODE_TRIVIAL,
     ++	MODE_REAL,
     ++};
     ++
      +struct merge_tree_options {
      +	int mode;
      +};
     @@ builtin/merge-tree.c: static int trivial_merge(int argc, const char **argv)
      +	struct option mt_options[] = {
      +		OPT_CMDMODE(0, "write-tree", &o.mode,
      +			    N_("do a real merge instead of a trivial merge"),
     -+			    'w'),
     ++			    MODE_REAL),
      +		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     -+			    N_("do a trivial merge only"), 't'),
     ++			    N_("do a trivial merge only"), MODE_TRIVIAL),
      +		OPT_END()
      +	};
      +
      +	/* Parse arguments */
      +	argc = parse_options(argc, argv, prefix, mt_options,
      +			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     -+	if (o.mode) {
     -+		expected_remaining_argc = (o.mode == 'w' ? 2 : 3);
     -+		if (argc != expected_remaining_argc)
     ++	switch (o.mode) {
     ++	default:
     ++		BUG("unexpected command mode %d", o.mode);
     ++	case MODE_UNKNOWN:
     ++		switch (argc) {
     ++		default:
      +			usage_with_options(merge_tree_usage, mt_options);
     -+	} else {
     -+		if (argc < 2 || argc > 3)
     -+			usage_with_options(merge_tree_usage, mt_options);
     -+		o.mode = (argc == 2 ? 'w' : 't');
     ++		case 2:
     ++			o.mode = MODE_REAL;
     ++			break;
     ++		case 3:
     ++			o.mode = MODE_TRIVIAL;
     ++			break;
     ++		}
     ++		expected_remaining_argc = argc;
     ++		break;
     ++	case MODE_REAL:
     ++		expected_remaining_argc = 2;
     ++		break;
     ++	case MODE_TRIVIAL:
     ++		expected_remaining_argc = 3;
     ++		break;
      +	}
      +
     ++	if (argc != expected_remaining_argc)
     ++		usage_with_options(merge_tree_usage, mt_options);
     ++
      +	/* Do the relevant type of merge */
     -+	if (o.mode == 'w')
     ++	if (o.mode == MODE_REAL)
      +		return real_merge(&o, argv[0], argv[1]);
      +	else
      +		return trivial_merge(argv[0], argv[1], argv[2]);
  4:  02c29f920d0 !  4:  d7b51da94e6 merge-tree: implement real merges
     @@ Commit message
          conflict/warning messages normally output during a merge, or have quick
          access to a list of files with conflicts.  That is not available in this
          preliminary implementation, but subsequent commits will add that
     -    ability.
     +    ability (meaning that NEWTREE would be a lot more than a tree in the
     +    case of conflicts).
      
          This also marks the traditional trivial merge of merge-tree as
          deprecated.  The trivial merge not only had limited applicability, the
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +Performs a merge, but does not make any new commits and does not read
      +from or write to either the working tree or index.
      +
     -+The second form is deprecated and supported only for backward
     -+compatibility.  It will likely be removed in the future, and will not
     -+be discussed further in this manual.
     -+
      +The first form will merge the two branches, doing a real merge.  A real
      +merge is distinguished from a trivial merge in that it includes:
      +
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +    merge base, creating a virtual merge base by merging the merge bases)
      +  * etc.
      +
     -+After the merge completes, it will create a new toplevel tree object.
     -+See `OUTPUT` below for details.
     ++After the merge completes, the first form will create a new toplevel
     ++tree object.  See `OUTPUT` below for details.
     ++
     ++The second form is deprecated; it is kept for backward compatibility
     ++reasons but may be deleted in the future.  Other than the optional
     ++`--trivial-merge`, it accepts no options.  It can only do a trivial
     ++merge.  It reads three tree-ish, and outputs trivial merge results and
     ++conflicting stages to the standard output in a semi-diff format.
     ++Since this was designed for higher level scripts to consume and merge
     ++the results back into the index, it omits entries that match
     ++<branch1>.  The result of this second form is is similar to what
     ++three-way 'git read-tree -m' does, but instead of storing the results
     ++in the index, the command outputs the entries to the standard output.
     ++This form not only has limited applicability, the output format is
     ++also difficult to work with, and it will generally be less performant
     ++than the first form even on successful merges (especially if working
     ++in large repositories).  The remainder of this manual will only
     ++discuss the first form.
      +
      +OUTPUT
      +------
     @@ Documentation/git-merge-tree.txt: git-merge-tree(1)
      +For a successful, non-conflicted merge, the exit status is 0.  When the
      +merge has conflicts, the exit status is 1.  If the merge is not able to
      +complete (or start) due to some kind of error, the exit status is
     -+something other than 0 or 1.
     ++something other than 0 or 1 (and the output is unspecified).
      +
      +USAGE NOTES
      +-----------
      +
      +git-merge-tree was written to be low-level plumbing, similar to
     -+hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
     -+be used as a part of a series of steps such as
     ++hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
     ++Thus, it could be used as a part of a series of steps such as
      +
      +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
      +       test $? -eq 0 || die "There were conflicts..."
      +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
      +       git update-ref $BRANCH1 $NEWCOMMIT
     -+
     -+However, it does not quite fit into the same category of low-level
     -+plumbing commands since the possibility of merge conflicts give it a
     -+much higher chance of the command not succeeding.
       
       GIT
       ---
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +
      +	git branch side1 &&
      +	git branch side2 &&
     ++	git branch side3 &&
      +
      +	git checkout side1 &&
      +	test_write_lines 1 2 3 4 5 6 >numbers &&
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +	>whatever/empty &&
      +	git add numbers greeting whatever/empty &&
      +	test_tick &&
     -+	git commit -m other-modifications
     ++	git commit -m other-modifications &&
     ++
     ++	git checkout side3 &&
     ++	git mv numbers sequence &&
     ++	test_tick &&
     ++	git commit -m rename-numbers
     ++'
     ++
     ++test_expect_success 'Clean merge' '
     ++	git merge-tree --write-tree side1 side3 >RESULT &&
     ++	q_to_tab <<-EOF >expect &&
     ++	100644 blob $(git rev-parse side1:greeting)Qgreeting
     ++	100644 blob $(git rev-parse side1:numbers)Qsequence
     ++	100644 blob $(git rev-parse side1:whatever)Qwhatever
     ++	EOF
     ++
     ++	git ls-tree $(cat RESULT) >actual &&
     ++	test_cmp expect actual
      +'
      +
      +test_expect_success 'Content merge and a few conflicts' '
     @@ t/t4301-merge-tree-write-tree.sh (new)
      +'
      +
      +test_expect_success 'Barf on too many arguments' '
     -+	test_expect_code 129 git merge-tree --write-tree side1 side2 side3 2>expect &&
     ++	test_expect_code 129 git merge-tree --write-tree side1 side2 invalid 2>expect &&
      +
      +	grep "^usage: git merge-tree" expect
      +'
  5:  290b42846b5 <  -:  ----------- Introduce a variant of the `warning()` function that takes a `FILE *`
  6:  2083fbe9b2e <  -:  ----------- diff: allow diff_warn_rename_limit to write somewhere besides stderr
  7:  1be858e6aa6 !  5:  58a5594aeb6 merge-ort: split out a separate display_update_messages() function
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
      +
      +	/* Also include needed rename limit adjustment now */
      +	diff_warn_rename_limit("merge.renamelimit",
     -+			       opti->renames.needed_limit, 0, stderr);
     ++			       opti->renames.needed_limit, 0);
      +
      +	trace2_region_leave("merge", "display messages", opt->repo);
      +}
     @@ merge-ort.c: static int record_conflicted_index_entries(struct merge_options *op
       			    struct tree *head,
       			    struct merge_result *result,
      @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
     + 		fclose(fp);
       		trace2_region_leave("merge", "write_auto_merge", opt->repo);
       	}
     - 
     +-
      -	if (display_update_msgs) {
      -		struct merge_options_internal *opti = result->priv;
      -		struct hashmap_iter iter;
     @@ merge-ort.c: void merge_switch_to_result(struct merge_options *opt,
      -
      -		/* Also include needed rename limit adjustment now */
      -		diff_warn_rename_limit("merge.renamelimit",
     --				       opti->renames.needed_limit, 0, stderr);
     +-				       opti->renames.needed_limit, 0);
      -
      -		trace2_region_leave("merge", "display messages", opt->repo);
      -	}
  8:  04c3bdc44d2 <  -:  ----------- merge-ort: allow update messages to be written to different file stream
  9:  c8ed002408d !  6:  fa55cb4d644 merge-tree: support including merge messages in output
     @@ Documentation/git-merge-tree.txt: git-merge-tree - Perform merge without touchin
       'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
       
       DESCRIPTION
     -@@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in that it includes:
     - After the merge completes, it will create a new toplevel tree object.
     - See `OUTPUT` below for details.
     +@@ Documentation/git-merge-tree.txt: than the first form even on successful merges (especially if working
     + in large repositories).  The remainder of this manual will only
     + discuss the first form.
       
      +OPTIONS
      +-------
     @@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in
      +	<Informational messages>
      +
      +These are discussed individually below.
     -+
     + 
     +-The printed tree object corresponds to what would be checked out in
     +-the working tree at the end of `git merge`, and thus may have files
     +-with conflict markers in them.
      +OID of toplevel tree
      +~~~~~~~~~~~~~~~~~~~~
      +
     @@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in
      +
      +This always starts with a blank line to separate it from the previous
      +section, and then has free-form messages about the merge, such as:
     - 
     --The printed tree object corresponds to what would be checked out in
     --the working tree at the end of `git merge`, and thus may have files
     --with conflict markers in them.
     ++
      +  * "Auto-merging <file>"
      +  * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
      +  * "Failed to merge submodule <submodule> (<reason>)"
     @@ Documentation/git-merge-tree.txt: merge is distinguished from a trivial merge in
       
       EXIT STATUS
       -----------
     -@@ Documentation/git-merge-tree.txt: be used as a part of a series of steps such as
     - 
     - However, it does not quite fit into the same category of low-level
     - plumbing commands since the possibility of merge conflicts give it a
     --much higher chance of the command not succeeding.
     -+much higher chance of the command not succeeding (and NEWTREE containing
     -+a bunch of stuff other than just a toplevel tree).
     +@@ Documentation/git-merge-tree.txt: Thus, it could be used as a part of a series of steps such as
     +        NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
     +        git update-ref $BRANCH1 $NEWCOMMIT
       
     ++Note that when the exit status is non-zero, NEWTREE in this sequence
     ++will contain a lot more output than just a tree.
     ++
       GIT
       ---
     + Part of the linkgit:git[1] suite
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     +@@ builtin/merge-tree.c: enum mode {
       
       struct merge_tree_options {
       	int mode;
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	puts(oid_to_hex(&result.tree->object.oid));
      +	if (o->show_messages) {
      +		printf("\n");
     -+		merge_display_update_messages(&opt, &result, stdout);
     ++		merge_display_update_messages(&opt, &result);
      +	}
       	merge_finalize(&opt, &result);
       	return !result.clean; /* result.clean < 0 handled above */
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		NULL
       	};
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			    'w'),
     + 			    MODE_REAL),
       		OPT_CMDMODE(0, "trivial-merge", &o.mode,
     - 			    N_("do a trivial merge only"), 't'),
     + 			    N_("do a trivial merge only"), MODE_TRIVIAL),
      +		OPT_BOOL(0, "messages", &o.show_messages,
      +			 N_("also show informational/conflict messages")),
       		OPT_END()
       	};
       
       	/* Parse arguments */
     -+	original_argc = argc;
     ++	original_argc = argc - 1; /* ignoring argv[0] */
       	argc = parse_options(argc, argv, prefix, mt_options,
       			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
     - 	if (o.mode) {
     + 	switch (o.mode) {
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			usage_with_options(merge_tree_usage, mt_options);
     - 		o.mode = (argc == 2 ? 'w' : 't');
     + 		break;
     + 	case MODE_TRIVIAL:
     + 		expected_remaining_argc = 3;
     ++		/* Removal of `--trivial-merge` is expected */
     ++		original_argc--;
     + 		break;
       	}
     -+	if (o.mode == 't' && original_argc < argc)
     ++	if (o.mode == MODE_TRIVIAL && argc < original_argc)
      +		die(_("--trivial-merge is incompatible with all other options"));
       
     - 	/* Do the relevant type of merge */
     - 	if (o.mode == 'w')
     + 	if (argc != expected_remaining_argc)
     + 		usage_with_options(merge_tree_usage, mt_options);
      
       ## t/t4301-merge-tree-write-tree.sh ##
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
       	grep "^usage: git merge-tree" expect
       '
       
     ++anonymize_hash() {
     ++	sed -e "s/[0-9a-f]\{40,\}/HASH/g" "$@"
     ++}
     ++
      +test_expect_success 'test conflict notices and such' '
      +	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	# Expected results:
      +	#   "greeting" should merge with conflicts
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many argument
      +
      +	test_cmp expect actual
      +'
     ++
     ++for opt in $(git merge-tree --git-completion-helper-all)
     ++do
     ++	if test $opt = "--trivial-merge" || test $opt = "--write-tree"
     ++	then
     ++		continue
     ++	fi
     ++
     ++	test_expect_success "usage: --trivial-merge is incompatible with $opt" '
     ++		test_expect_code 128 git merge-tree --trivial-merge $opt side1 side2 side3
     ++	'
     ++done
      +
       test_done
 10:  1c2a3f5ef63 !  7:  f3ad7add515 merge-ort: provide a merge_get_conflicted_files() helper function
     @@ merge-ort.h
       
       struct commit;
       struct tree;
     -@@ merge-ort.h: void merge_display_update_messages(struct merge_options *opt,
     - 				   struct merge_result *result,
     - 				   FILE *stream);
     +@@ merge-ort.h: void merge_switch_to_result(struct merge_options *opt,
     + void merge_display_update_messages(struct merge_options *opt,
     + 				   struct merge_result *result);
       
      +struct stage_info {
      +	struct object_id oid;
 11:  9c2389eef0e !  8:  6058190d1b1 merge-tree: provide a list of which files have conflicts
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
      +	}
       	if (o->show_messages) {
       		printf("\n");
     - 		merge_display_update_messages(&opt, &result, stdout);
     + 		merge_display_update_messages(&opt, &result);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
       
       	/* Do the relevant type of merge */
     - 	if (o.mode == 'w')
     + 	if (o.mode == MODE_REAL)
      -		return real_merge(&o, argv[0], argv[1]);
      +		return real_merge(&o, argv[0], argv[1], prefix);
       	else
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and
       
       	Auto-merging greeting
       	CONFLICT (content): Merge conflict in greeting
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
     - 	test_cmp expect actual
     - '
     +@@ t/t4301-merge-tree-write-tree.sh: do
     + 	'
     + done
       
      +test_expect_success 'Just the conflicted files without the messages' '
      +	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	test_write_lines HASH greeting whatever~side1 >expect &&
      +
 12:  2188a8ca1e7 !  9:  435f66ea699 merge-tree: provide easy access to `ls-files -u` style info
     @@ Commit message
          Much like `git merge` updates the index with information of the form
              (mode, oid, stage, name)
          provide this output for conflicted files for merge-tree as well.
     -    Provide an --exclude-modes-oids-stages/-l option for users to exclude
     -    the mode, oid, and stage and only get the list of conflicted filenames.
     +    Provide a --name-only option for users to exclude the mode, oid, and
     +    stage and only get the list of conflicted filenames.
      
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
     +@@ Documentation/git-merge-tree.txt: discuss the first form.
       OPTIONS
       -------
       
     -+--exclude-oids-and-modes::
     -+	Instead of writing a list of (mode, oid, stage, path) tuples
     -+	to output for conflicted files, just provide a list of
     -+	filenames with conflicts.
     ++--name-only::
     ++	In the Conflicted file info section, instead of writing a list
     ++	of (mode, oid, stage, path) tuples to output for conflicted
     ++	files, just provide a list of filenames with conflicts (and
     ++	do not list filenames multiple times if they have multiple
     ++	conflicting stages).
      +
       --[no-]messages::
       	Write any informational messages such as "Auto-merging <path>"
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
      +
      +The filename will be quoted as explained for the configuration
      +variable `core.quotePath` (see linkgit:git-config[1]).  However, if
     -+the `--exclude-oids-and-modes` option is passed, the mode, object, and
     -+stage will be omitted.
     ++the `--name-only` option is passed, the mode, object, and stage will
     ++be omitted.
       
       Informational messages
       ~~~~~~~~~~~~~~~~~~~~~~
     @@ Documentation/git-merge-tree.txt: This is a tree object that represents what wou
       
         * "Auto-merging <file>"
         * "CONFLICT (rename/delete): <oldfile> renamed...but deleted in..."
     -@@ Documentation/git-merge-tree.txt: plumbing commands since the possibility of merge conflicts give it a
     - much higher chance of the command not succeeding (and NEWTREE containing
     - a bunch of stuff other than just a toplevel tree).
     +@@ Documentation/git-merge-tree.txt: Thus, it could be used as a part of a series of steps such as
     + Note that when the exit status is non-zero, NEWTREE in this sequence
     + will contain a lot more output than just a tree.
       
      +git-merge-tree was written to provide users with the same information
      +that they'd have access to if using `git merge`:
     @@ Documentation/git-merge-tree.txt: plumbing commands since the possibility of mer
       Part of the linkgit:git[1] suite
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     +@@ builtin/merge-tree.c: enum mode {
       struct merge_tree_options {
       	int mode;
       	int show_messages;
     -+	int exclude_modes_oids_stages;
     ++	int name_only;
       };
       
       static int real_merge(struct merge_tree_options *o,
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			const char *name = conflicted_files.items[i].string;
      -			if (last && !strcmp(last, name))
      +			struct stage_info *c = conflicted_files.items[i].util;
     -+			if (!o->exclude_modes_oids_stages)
     ++			if (!o->name_only)
      +				printf("%06o %s %d\t",
      +				       c->mode, oid_to_hex(&c->oid), c->stage);
      +			else if (last && !strcmp(last, name))
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       			write_name_quoted_relative(
       				name, prefix, stdout, line_termination);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			    N_("do a trivial merge only"), 't'),
     + 			    N_("do a trivial merge only"), MODE_TRIVIAL),
       		OPT_BOOL(0, "messages", &o.show_messages,
       			 N_("also show informational/conflict messages")),
     -+		OPT_BOOL_F('l', "exclude-modes-oids-stages",
     -+			   &o.exclude_modes_oids_stages,
     -+			   N_("list conflicted files without modes/oids/stages"),
     ++		OPT_BOOL_F(0, "name-only",
     ++			   &o.name_only,
     ++			   N_("list filenames without modes/oids/stages"),
      +			   PARSE_OPT_NONEG),
       		OPT_END()
       	};
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Content merge and a few c
       	test_when_finished "git reset --hard" &&
       
       	test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Barf on too many arguments' '
     - '
     +@@ t/t4301-merge-tree-write-tree.sh: anonymize_hash() {
     + }
       
       test_expect_success 'test conflict notices and such' '
      -	test_expect_code 1 git merge-tree --write-tree side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --exclude-modes-oids-stages side1 side2 >out &&
     - 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	test_expect_code 1 git merge-tree --write-tree --name-only side1 side2 >out &&
     + 	anonymize_hash out >actual &&
       
       	# Expected results:
     -@@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'test conflict notices and such' '
     - '
     +@@ t/t4301-merge-tree-write-tree.sh: do
     + done
       
       test_expect_success 'Just the conflicted files without the messages' '
      -	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	test_expect_code 1 git merge-tree --write-tree --no-messages --exclude-modes-oids-stages side1 side2 >out &&
     - 	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	test_expect_code 1 git merge-tree --write-tree --no-messages --name-only side1 side2 >out &&
     + 	anonymize_hash out >actual &&
       
       	test_write_lines HASH greeting whatever~side1 >expect &&
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Just the conflicted files without the messages' '
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Just the conflicted files
       
      +test_expect_success 'Check conflicted oids and modes without messages' '
      +	test_expect_code 1 git merge-tree --write-tree --no-messages side1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	# Compare the basic output format
      +	q_to_tab >expect <<-\EOF &&
 13:  52339b396fa ! 10:  5f253e298b3 merge-tree: allow `ls-files -u` style info to be NUL terminated
     @@ Commit message
          Signed-off-by: Elijah Newren <newren@gmail.com>
      
       ## Documentation/git-merge-tree.txt ##
     -@@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
     +@@ Documentation/git-merge-tree.txt: discuss the first form.
       OPTIONS
       -------
       
     @@ Documentation/git-merge-tree.txt: See `OUTPUT` below for details.
      +	newline.  Also begin the messages section with a NUL character
      +	instead of a newline.  See OUTPUT below for more information.
      +
     - --exclude-oids-and-modes::
     - 	Instead of writing a list of (mode, oid, stage, path) tuples
     - 	to output for conflicted files, just provide a list of
     + --name-only::
     + 	In the Conflicted file info section, instead of writing a list
     + 	of (mode, oid, stage, path) tuples to output for conflicted
      @@ Documentation/git-merge-tree.txt: OID of toplevel tree
       
       This is a tree object that represents what would be checked out in the
       working tree at the end of `git merge`.  If there were conflicts, then
      -files within this tree may have embedded conflict markers.
      +files within this tree may have embedded conflict markers.  This section
     -+is always followed by a newline.
     ++is always followed by a newline (or NUL if `-z` is passed).
       
       Conflicted file info
       ~~~~~~~~~~~~~~~~~~~~
      @@ Documentation/git-merge-tree.txt: This is a sequence of lines with the format
       The filename will be quoted as explained for the configuration
       variable `core.quotePath` (see linkgit:git-config[1]).  However, if
     - the `--exclude-oids-and-modes` option is passed, the mode, object, and
     --stage will be omitted.
     -+stage will be omitted.  If `-z` is passed, the "lines" are terminated
     -+by a NUL character instead of a newline character.
     + the `--name-only` option is passed, the mode, object, and stage will
     +-be omitted.
     ++be omitted.  If `-z` is passed, the "lines" are terminated by a NUL
     ++character instead of a newline character.
       
       Informational messages
       ~~~~~~~~~~~~~~~~~~~~~~
     @@ Documentation/git-merge-tree.txt: This is a sequence of lines with the format
       
      
       ## builtin/merge-tree.c ##
     +@@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
     + 	if (o->show_messages == -1)
     + 		o->show_messages = !result.clean;
     + 
     +-	puts(oid_to_hex(&result.tree->object.oid));
     ++	printf("%s%c", oid_to_hex(&result.tree->object.oid), line_termination);
     + 	if (!result.clean) {
     + 		struct string_list conflicted_files = STRING_LIST_INIT_NODUP;
     + 		const char *last = NULL;
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       		string_list_clear(&conflicted_files, 1);
       	}
       	if (o->show_messages) {
      -		printf("\n");
      +		putchar(line_termination);
     - 		merge_display_update_messages(&opt, &result, stdout);
     + 		merge_display_update_messages(&opt, &result);
       	}
       	merge_finalize(&opt, &result);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			    N_("do a trivial merge only"), 't'),
     + 			    N_("do a trivial merge only"), MODE_TRIVIAL),
       		OPT_BOOL(0, "messages", &o.show_messages,
       			 N_("also show informational/conflict messages")),
      +		OPT_SET_INT('z', NULL, &line_termination,
      +			    N_("separate paths with the NUL character"), '\0'),
     - 		OPT_BOOL_F('l', "exclude-modes-oids-stages",
     - 			   &o.exclude_modes_oids_stages,
     - 			   N_("list conflicted files without modes/oids/stages"),
     + 		OPT_BOOL_F(0, "name-only",
     + 			   &o.name_only,
     + 			   N_("list filenames without modes/oids/stages"),
      
       ## t/t4301-merge-tree-write-tree.sh ##
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and modes without messages' '
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'Check conflicted oids and
      +	git commit -m "Renamed numbers" &&
      +
      +	test_expect_code 1 git merge-tree --write-tree -z tweak1 side2 >out &&
     -+	sed -e "s/[0-9a-f]\{40,\}/HASH/g" out >actual &&
     ++	anonymize_hash out >actual &&
      +
      +	# Expected results:
      +	#   "greeting" should merge with conflicts
      +	#   "whatever" has *both* a modify/delete and a file/directory conflict
      +	#   "Αυτά μου φαίνονται κινέζικα" should have a conflict
     -+	echo HASH >expect &&
     ++	echo HASH | lf_to_nul >expect &&
      +
      +	q_to_tab <<-EOF | lf_to_nul >>expect &&
      +	100644 HASH 1Qgreeting
 14:  c854ecb5f4a ! 11:  e706cf31c6e merge-tree: add a --allow-unrelated-histories flag
     @@ Documentation/git-merge-tree.txt: OPTIONS
       
      
       ## builtin/merge-tree.c ##
     -@@ builtin/merge-tree.c: static int trivial_merge(const char *base,
     +@@ builtin/merge-tree.c: enum mode {
       
       struct merge_tree_options {
       	int mode;
      +	int allow_unrelated_histories;
       	int show_messages;
     - 	int exclude_modes_oids_stages;
     + 	int name_only;
       };
      @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	 * merge_incore_recursive in merge-ort.h
     @@ builtin/merge-tree.c: static int real_merge(struct merge_tree_options *o,
       	for (j = common; j; j = j->next)
       		commit_list_insert(j->item, &merge_bases);
      @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char *prefix)
     - 			   &o.exclude_modes_oids_stages,
     - 			   N_("list conflicted files without modes/oids/stages"),
     + 			   &o.name_only,
     + 			   N_("list filenames without modes/oids/stages"),
       			   PARSE_OPT_NONEG),
      +		OPT_BOOL_F(0, "allow-unrelated-histories",
      +			   &o.allow_unrelated_histories,
     @@ builtin/merge-tree.c: int cmd_merge_tree(int argc, const char **argv, const char
      
       ## t/t4301-merge-tree-write-tree.sh ##
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
     - 	>whatever/empty &&
     - 	git add numbers greeting whatever/empty &&
     + 	git checkout side3 &&
     + 	git mv numbers sequence &&
       	test_tick &&
     --	git commit -m other-modifications
     -+	git commit -m other-modifications &&
     +-	git commit -m rename-numbers
     ++	git commit -m rename-numbers &&
      +
      +	git switch --orphan unrelated &&
      +	>something-else &&
     @@ t/t4301-merge-tree-write-tree.sh: test_expect_success setup '
      +	git commit -m first-commit
       '
       
     - test_expect_success 'Content merge and a few conflicts' '
     + test_expect_success 'Clean merge' '
      @@ t/t4301-merge-tree-write-tree.sh: test_expect_success 'NUL terminated conflicted file "lines"' '
       	test_cmp expect actual
       '
 15:  bc8591bbb63 = 12:  c279236ab65 git-merge-tree.txt: add a section on potentional usage mistakes

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v4 01/12] merge-tree: rename merge_trees() to trivial_merge_trees()
  2022-02-12 20:34     ` [PATCH v4 00/12] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
@ 2022-02-12 20:34       ` Elijah Newren via GitGitGadget
  2022-02-12 20:34       ` [PATCH v4 02/12] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
                         ` (11 subsequent siblings)
  12 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-12 20:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Josh Steadmon, Emily Shaffer,
	Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

merge-recursive.h defined its own merge_trees() function, different than
the one found in builtin/merge-tree.c.  That was okay in the past, but
we want merge-tree to be able to use the merge-ort functions, which will
end up including merge-recursive.h.  Rename the function found in
builtin/merge-tree.c to avoid the conflict.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 5dc94d6f880..06f9eee9f78 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -28,7 +28,7 @@ static void add_merge_entry(struct merge_list *entry)
 	merge_result_end = &entry->next;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base);
+static void trivial_merge_trees(struct tree_desc t[3], const char *base);
 
 static const char *explanation(struct merge_list *entry)
 {
@@ -225,7 +225,7 @@ static void unresolved_directory(const struct traverse_info *info,
 	buf2 = fill_tree_descriptor(r, t + 2, ENTRY_OID(n + 2));
 #undef ENTRY_OID
 
-	merge_trees(t, newbase);
+	trivial_merge_trees(t, newbase);
 
 	free(buf0);
 	free(buf1);
@@ -342,7 +342,7 @@ static int threeway_callback(int n, unsigned long mask, unsigned long dirmask, s
 	return mask;
 }
 
-static void merge_trees(struct tree_desc t[3], const char *base)
+static void trivial_merge_trees(struct tree_desc t[3], const char *base)
 {
 	struct traverse_info info;
 
@@ -378,7 +378,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
-	merge_trees(t, "");
+	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
 	free(buf3);
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v4 02/12] merge-tree: move logic for existing merge into new function
  2022-02-12 20:34     ` [PATCH v4 00/12] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  2022-02-12 20:34       ` [PATCH v4 01/12] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
@ 2022-02-12 20:34       ` Elijah Newren via GitGitGadget
  2022-02-12 20:34       ` [PATCH v4 03/12] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
                         ` (10 subsequent siblings)
  12 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-12 20:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Josh Steadmon, Emily Shaffer,
	Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

In preparation for adding a non-trivial merge capability to merge-tree,
move the existing merge logic for trivial merges into a new function.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 06f9eee9f78..914ec960b7e 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -366,15 +366,12 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+static int trivial_merge(int argc, const char **argv)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	if (argc != 4)
-		usage(merge_tree_usage);
-
 	buf1 = get_tree_descriptor(r, t+0, argv[1]);
 	buf2 = get_tree_descriptor(r, t+1, argv[2]);
 	buf3 = get_tree_descriptor(r, t+2, argv[3]);
@@ -386,3 +383,10 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 	show_result();
 	return 0;
 }
+
+int cmd_merge_tree(int argc, const char **argv, const char *prefix)
+{
+	if (argc != 4)
+		usage(merge_tree_usage);
+	return trivial_merge(argc, argv);
+}
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v4 03/12] merge-tree: add option parsing and initial shell for real merge function
  2022-02-12 20:34     ` [PATCH v4 00/12] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
  2022-02-12 20:34       ` [PATCH v4 01/12] merge-tree: rename merge_trees() to trivial_merge_trees() Elijah Newren via GitGitGadget
  2022-02-12 20:34       ` [PATCH v4 02/12] merge-tree: move logic for existing merge into new function Elijah Newren via GitGitGadget
@ 2022-02-12 20:34       ` Elijah Newren via GitGitGadget
  2022-02-12 20:34       ` [PATCH v4 04/12] merge-tree: implement real merges Elijah Newren via GitGitGadget
                         ` (9 subsequent siblings)
  12 siblings, 0 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-12 20:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Josh Steadmon, Emily Shaffer,
	Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

Let merge-tree accept a `--write-tree` parameter for choosing real
merges instead of trivial merges, and accept an optional
`--trivial-merge` option to get the traditional behavior.  Note that
these accept different numbers of arguments, though, so these names
need not actually be used.

Note that real merges differ from trivial merges in that they handle:
  - three way content merges
  - recursive ancestor consolidation
  - renames
  - proper directory/file conflict handling
  - etc.
Basically all the stuff you'd expect from `git merge`, just without
updating the index and working tree.  The initial shell added here does
nothing more than die with "real merges are not yet implemented", but
that will be fixed in subsequent commits.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 builtin/merge-tree.c | 84 +++++++++++++++++++++++++++++++++++++++-----
 git.c                |  2 +-
 2 files changed, 76 insertions(+), 10 deletions(-)

diff --git a/builtin/merge-tree.c b/builtin/merge-tree.c
index 914ec960b7e..0f9d928e862 100644
--- a/builtin/merge-tree.c
+++ b/builtin/merge-tree.c
@@ -3,13 +3,12 @@
 #include "tree-walk.h"
 #include "xdiff-interface.h"
 #include "object-store.h"
+#include "parse-options.h"
 #include "repository.h"
 #include "blob.h"
 #include "exec-cmd.h"
 #include "merge-blobs.h"
 
-static const char merge_tree_usage[] = "git merge-tree <base-tree> <branch1> <branch2>";
-
 struct merge_list {
 	struct merge_list *next;
 	struct merge_list *link;	/* other stages for this object */
@@ -366,15 +365,17 @@ static void *get_tree_descriptor(struct repository *r,
 	return buf;
 }
 
-static int trivial_merge(int argc, const char **argv)
+static int trivial_merge(const char *base,
+			 const char *branch1,
+			 const char *branch2)
 {
 	struct repository *r = the_repository;
 	struct tree_desc t[3];
 	void *buf1, *buf2, *buf3;
 
-	buf1 = get_tree_descriptor(r, t+0, argv[1]);
-	buf2 = get_tree_descriptor(r, t+1, argv[2]);
-	buf3 = get_tree_descriptor(r, t+2, argv[3]);
+	buf1 = get_tree_descriptor(r, t+0, base);
+	buf2 = get_tree_descriptor(r, t+1, branch1);
+	buf3 = get_tree_descriptor(r, t+2, branch2);
 	trivial_merge_trees(t, "");
 	free(buf1);
 	free(buf2);
@@ -384,9 +385,74 @@ static int trivial_merge(int argc, const char **argv)
 	return 0;
 }
 
+enum mode {
+	MODE_UNKNOWN,
+	MODE_TRIVIAL,
+	MODE_REAL,
+};
+
+struct merge_tree_options {
+	int mode;
+};
+
+static int real_merge(struct merge_tree_options *o,
+		      const char *branch1, const char *branch2)
+{
+	die(_("real merges are not yet implemented"));
+}
+
 int cmd_merge_tree(int argc, const char **argv, const char *prefix)
 {
-	if (argc != 4)
-		usage(merge_tree_usage);
-	return trivial_merge(argc, argv);
+	struct merge_tree_options o = { 0 };
+	int expected_remaining_argc;
+
+	const char * const merge_tree_usage[] = {
+		N_("git merge-tree [--write-tree] <branch1> <branch2>"),
+		N_("git merge-tree [--trivial-merge] <base-tree> <branch1> <branch2>"),
+		NULL
+	};
+	struct option mt_options[] = {
+		OPT_CMDMODE(0, "write-tree", &o.mode,
+			    N_("do a real merge instead of a trivial merge"),
+			    MODE_REAL),
+		OPT_CMDMODE(0, "trivial-merge", &o.mode,
+			    N_("do a trivial merge only"), MODE_TRIVIAL),
+		OPT_END()
+	};
+
+	/* Parse arguments */
+	argc = parse_options(argc, argv, prefix, mt_options,
+			     merge_tree_usage, PARSE_OPT_STOP_AT_NON_OPTION);
+	switch (o.mode) {
+	default:
+		BUG("unexpected command mode %d", o.mode);
+	case MODE_UNKNOWN:
+		switch (argc) {
+		default:
+			usage_with_options(merge_tree_usage, mt_options);
+		case 2:
+			o.mode = MODE_REAL;
+			break;
+		case 3:
+			o.mode = MODE_TRIVIAL;
+			break;
+		}
+		expected_remaining_argc = argc;
+		break;
+	case MODE_REAL:
+		expected_remaining_argc = 2;
+		break;
+	case MODE_TRIVIAL:
+		expected_remaining_argc = 3;
+		break;
+	}
+
+	if (argc != expected_remaining_argc)
+		usage_with_options(merge_tree_usage, mt_options);
+
+	/* Do the relevant type of merge */
+	if (o.mode == MODE_REAL)
+		return real_merge(&o, argv[0], argv[1]);
+	else
+		return trivial_merge(argv[0], argv[1], argv[2]);
 }
diff --git a/git.c b/git.c
index 5ff21be21f3..6090a1289db 100644
--- a/git.c
+++ b/git.c
@@ -558,7 +558,7 @@ static struct cmd_struct commands[] = {
 	{ "merge-recursive-ours", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-recursive-theirs", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
 	{ "merge-subtree", cmd_merge_recursive, RUN_SETUP | NEED_WORK_TREE | NO_PARSEOPT },
-	{ "merge-tree", cmd_merge_tree, RUN_SETUP | NO_PARSEOPT },
+	{ "merge-tree", cmd_merge_tree, RUN_SETUP },
 	{ "mktag", cmd_mktag, RUN_SETUP | NO_PARSEOPT },
 	{ "mktree", cmd_mktree, RUN_SETUP },
 	{ "multi-pack-index", cmd_multi_pack_index, RUN_SETUP },
-- 
gitgitgadget


^ permalink raw reply	[flat|nested] 215+ messages in thread

* [PATCH v4 04/12] merge-tree: implement real merges
  2022-02-12 20:34     ` [PATCH v4 00/12] In-core git merge-tree ("Server side merges") Elijah Newren via GitGitGadget
                         ` (2 preceding siblings ...)
  2022-02-12 20:34       ` [PATCH v4 03/12] merge-tree: add option parsing and initial shell for real merge function Elijah Newren via GitGitGadget
@ 2022-02-12 20:34       ` Elijah Newren via GitGitGadget
  2022-02-14 17:51         ` Junio C Hamano
  2022-02-15  8:46         ` Ævar Arnfjörð Bjarmason
  2022-02-12 20:34       ` [PATCH v4 05/12] merge-ort: split out a separate display_update_messages() function Elijah Newren via GitGitGadget
                         ` (8 subsequent siblings)
  12 siblings, 2 replies; 215+ messages in thread
From: Elijah Newren via GitGitGadget @ 2022-02-12 20:34 UTC (permalink / raw)
  To: git
  Cc: Christian Couder, Taylor Blau, Johannes Altmanninger,
	Ramsay Jones, Johannes Schindelin, Christian Couder,
	René Scharfe, Ævar Arnfjörð Bjarmason,
	Elijah Newren, Johannes Sixt, Josh Steadmon, Emily Shaffer,
	Elijah Newren, Elijah Newren

From: Elijah Newren <newren@gmail.com>

This adds the ability to perform real merges rather than just trivial
merges (meaning handling three way content merges, recursive ancestor
consolidation, renames, proper directory/file conflict handling, and so
forth).  However, unlike `git merge`, the working tree and index are
left alone and no branch is updated.

The only output is:
  - the toplevel resulting tree printed on stdout
  - exit status of 0 (clean), 1 (conflicts present), anything else
    (merge could not be performed; unknown if clean or conflicted)

This output is meant to be used by some higher level script, perhaps in
a sequence of steps like this:

   NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
   test $? -eq 0 || die "There were conflicts..."
   NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
   git update-ref $BRANCH1 $NEWCOMMIT

Note that higher level scripts may also want to access the
conflict/warning messages normally output during a merge, or have quick
access to a list of files with conflicts.  That is not available in this
preliminary implementation, but subsequent commits will add that
ability (meaning that NEWTREE would be a lot more than a tree in the
case of conflicts).

This also marks the traditional trivial merge of merge-tree as
deprecated.  The trivial merge not only had limited applicability, the
output format was also difficult to work with (and its format
undocumented), and will generally be less performant than real merges.

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 Documentation/git-merge-tree.txt |  79 +++++++++++++++++++----
 builtin/merge-tree.c             |  44 ++++++++++++-
 t/t4301-merge-tree-write-tree.sh | 106 +++++++++++++++++++++++++++++++
 3 files changed, 216 insertions(+), 13 deletions(-)
 create mode 100755 t/t4301-merge-tree-write-tree.sh

diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
index 58731c19422..586733ea564 100644
--- a/Documentation/git-merge-tree.txt
+++ b/Documentation/git-merge-tree.txt
@@ -3,26 +3,81 @@ git-merge-tree(1)
 
 NAME
 ----
-git-merge-tree - Show three-way merge without touching index
+git-merge-tree - Perform merge without touching index or working tree
 
 
 SYNOPSIS
 --------
 [verse]
-'git merge-tree' <base-tree> <branch1> <branch2>
+'git merge-tree' [--write-tree] <branch1> <branch2>
+'git merge-tree' [--trivial-merge] <base-tree> <branch1> <branch2> (deprecated)
 
 DESCRIPTION
 -----------
-Reads three tree-ish, and output trivial merge results and
-conflicting stages to the standard output.  This is similar to
-what three-way 'git read-tree -m' does, but instead of storing the
-results in the index, the command outputs the entries to the
-standard output.
-
-This is meant to be used by higher level scripts to compute
-merge results outside of the index, and stuff the results back into the
-index.  For this reason, the output from the command omits
-entries that match the <branch1> tree.
+
+Performs a merge, but does not make any new commits and does not read
+from or write to either the working tree or index.
+
+The first form will merge the two branches, doing a real merge.  A real
+merge is distinguished from a trivial merge in that it includes:
+
+  * three way content merges of individual files
+  * rename detection
+  * proper directory/file conflict handling
+  * recursive ancestor consolidation (i.e. when there is more than one
+    merge base, creating a virtual merge base by merging the merge bases)
+  * etc.
+
+After the merge completes, the first form will create a new toplevel
+tree object.  See `OUTPUT` below for details.
+
+The second form is deprecated; it is kept for backward compatibility
+reasons but may be deleted in the future.  Other than the optional
+`--trivial-merge`, it accepts no options.  It can only do a trivial
+merge.  It reads three tree-ish, and outputs trivial merge results and
+conflicting stages to the standard output in a semi-diff format.
+Since this was designed for higher level scripts to consume and merge
+the results back into the index, it omits entries that match
+<branch1>.  The result of this second form is is similar to what
+three-way 'git read-tree -m' does, but instead of storing the results
+in the index, the command outputs the entries to the standard output.
+This form not only has limited applicability, the output format is
+also difficult to work with, and it will generally be less performant
+than the first form even on successful merges (especially if working
+in large repositories).  The remainder of this manual will only
+discuss the first form.
+
+OUTPUT
+------
+
+For either a successful or conflicted merge, the output from
+git-merge-tree is simply one line:
+
+	<OID of toplevel tree>
+
+The printed tree object corresponds to what would be checked out in
+the working tree at the end of `git merge`, and thus may have files
+with conflict markers in them.
+
+EXIT STATUS
+-----------
+
+For a successful, non-conflicted merge, the exit status is 0.  When the
+merge has conflicts, the exit status is 1.  If the merge is not able to
+complete (or start) due to some kind of error, the exit status is
+something other than 0 or 1 (and the output is unspecified).
+
+USAGE NOTES
+-----------
+
+git-merge-tree was written to be low-level plumbing, similar to
+hash-object, mktree, commit-tree, write-tree, update-ref, and mktag.
+Thus, it could be used as a part of a series of steps such as
+
+       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
+       test $? -eq 0 || die "There were conflicts..."
+       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1