Re: [PATCH v2 04/13] merge-tree: implement real merges

[Elijah Newren <newren@gmail.com>]

On Wed, Feb 2, 2022 at 1:30 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> "Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
> > From: Elijah Newren <newren@gmail.com>
> >
> > This adds the ability to perform real merges rather than just trivial
> > merges (meaning handling three way content merges, recursive ancestor
> > consolidation, renames, proper directory/file conflict handling, and so
> > forth).  However, unlike `git merge`, the working tree and index are
> > left alone and no branch is updated.
> >
> > The only output is:
> >   - the toplevel resulting tree printed on stdout
> >   - exit status of 0 (clean), 1 (conflicts present), anything else
> >     (merge could not be performed; unknown if clean or conflicted)
> >
> > This output is meant to be used by some higher level script, perhaps in
> > a sequence of steps like this:
> >
> >    NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> >    test $? -eq 0 || die "There were conflicts..."
> >    NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> >    git update-ref $BRANCH1 $NEWCOMMIT
>
> It is unclear what NEWTREE has, if anything meaningful, when the
> command exited with non-zero status.  Let's read on.
>
> > diff --git a/Documentation/git-merge-tree.txt b/Documentation/git-merge-tree.txt
> > index 58731c19422..569485815a0 100644
> > --- a/Documentation/git-merge-tree.txt
> > +++ b/Documentation/git-merge-tree.txt
> > @@ -3,26 +3,73 @@ git-merge-tree(1)
> >
> >  NAME
> >  ----
> > -git-merge-tree - Show three-way merge without touching index
> > +git-merge-tree - Perform merge without touching index or working tree
>
> OK.  It is interesting that both apply equally to either command
> mode ;-)
>
> > +Performs a merge, but does not make any new commits and does not read
> > +from or write to either the working tree or index.
> > +
> > +The second form is deprecated and supported only for backward
> > +compatibility.  It will likely be removed in the future, and will not
> > +be discussed further in this manual.
>
> This, especially the deletion of the original description on what
> trivial merge does, may be premature, especially if it is still
> "supported for backward compatibility".

I actually extended it, but Dscho suggested removing it entirely --
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2201251804250.2121@tvgsbejvaqbjf.bet/.
I can restore it; does that paragraph look good to you (you can see
the full thing even if it's split by Dscho's commentary).

> > +The first form will merge the two branches, doing a real merge.  A real
> > +merge is distinguished from a trivial merge in that it includes:
>
> I think this is worth keeping, simply because I do not think you
> should entirely omit description on the trivial one.
>
> But if we were to remove the description on the trivial one, then it
> is totally meaningless to label the following as "... distinguished
> from a trivial merge in that".  The list of things the real merge
> command mode does (below) is of course worth having.
>
> > +  * three way content merges of individual files
> > +  * rename detection
> > +  * proper directory/file conflict handling
> > +  * recursive ancestor consolidation (i.e. when there is more than one
> > +    merge base, creating a virtual merge base by merging the merge bases)
> > +  * etc.
> > +
> > +After the merge completes, it will create a new toplevel tree object.
> > +See `OUTPUT` below for details.
> > +
> > +OUTPUT
> > +------
> > +
> > +For either a successful or conflicted merge, the output from
> > +git-merge-tree is simply one line:
> > +
> > +     <OID of toplevel tree>
> > +
> > +The printed tree object corresponds to what would be checked out in
> > +the working tree at the end of `git merge`, and thus may have files
> > +with conflict markers in them.
>
> So we would leave cruft in the object store, but that is very much
> on purpose, and the expectation is that the user would commit-tree
> the tree object and reference it with a ref soon enough before
> garbage collection prunes them, just like how write-tree is meant to
> be used.  OK.
>
> > +EXIT STATUS
> > +-----------
> > +
> > +For a successful, non-conflicted merge, the exit status is 0.  When the
> > +merge has conflicts, the exit status is 1.  If the merge is not able to
> > +complete (or start) due to some kind of error, the exit status is
> > +something other than 0 or 1.
>
> And the output given to the standard output stream, when the command
> exits with status higher than 1, is...?  "unspecified" is of course
> an acceptable answer, of course, but I am wondering if it is worth
> spelling out in the doc.

Yeah, "unspecified" -- whatever random error the user triggered
(couldn't parse arguments, the refs you gave don't exist, the repo you
are running from isn't a repo, disk is full, etc.)

> > +USAGE NOTES
> > +-----------
> > +
> > +git-merge-tree was written to be low-level plumbing, similar to
> > +hash-object, mktree, commit-tree, update-ref, and mktag.  Thus, it could
>
> A notable omission in the above list is 'write-tree'.

Ooh, yeah, I should add that one.

> > +be used as a part of a series of steps such as
> > +
> > +       NEWTREE=$(git merge-tree --write-tree $BRANCH1 $BRANCH2)
> > +       test $? -eq 0 || die "There were conflicts..."
> > +       NEWCOMMIT=$(git commit-tree $NEWTREE -p $BRANCH1 -p $BRANCH2)
> > +       git update-ref $BRANCH1 $NEWCOMMIT
> > +
> > +However, it does not quite fit into the same category of low-level
> > +plumbing commands since the possibility of merge conflicts give it a
> > +much higher chance of the command not succeeding.
>
> I am not sure if that is a fair categorization.  It is a fine
> building block at the lowest level of the tool hierarchy.  The
> primary thing that differentiates plumbing from Porcelain is that
> the former is a better fit for scripting: doing one thing and one
> thing well with minimum and stable UI.  The complexity of that one
> thing it does has much less to do with the categorization, and the
> rate at which users may use the command in a failure-inducing
> situation has nothing to do with it.  The write-tree plumbing
> command will reliably fail with 100% chance when the index used as
> its input is unmerged.
>
> > @@ -392,7 +395,46 @@ struct merge_tree_options {
> >  static int real_merge(struct merge_tree_options *o,
> >                     const char *branch1, const char *branch2)
> >  {
> > -     die(_("real merges are not yet implemented"));
> > +     struct commit *parent1, *parent2;
> > +     struct commit_list *common;
> > +     struct commit_list *merge_bases = NULL;
> > +     struct commit_list *j;
> > +     struct merge_options opt;
> > +     struct merge_result result = { 0 };
> > +
> > +     parent1 = get_merge_parent(branch1);
> > +     if (!parent1)
> > +             help_unknown_ref(branch1, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     parent2 = get_merge_parent(branch2);
> > +     if (!parent2)
> > +             help_unknown_ref(branch2, "merge-tree",
> > +                              _("not something we can merge"));
> > +
> > +     init_merge_options(&opt, the_repository);
> > +
> > +     opt.show_rename_progress = 0;
> > +
> > +     opt.branch1 = branch1;
> > +     opt.branch2 = branch2;
> > +
> > +     /*
> > +      * Get the merge bases, in reverse order; see comment above
> > +      * merge_incore_recursive in merge-ort.h
> > +      */
> > +     common = get_merge_bases(parent1, parent2);
> > +     if (!common)
> > +             die(_("refusing to merge unrelated histories"));
> > +     for (j = common; j; j = j->next)
> > +             commit_list_insert(j->item, &merge_bases);
> > +
> > +     merge_incore_recursive(&opt, merge_bases, parent1, parent2, &result);
> > +     if (result.clean < 0)
> > +             die(_("failure to merge"));
> > +     puts(oid_to_hex(&result.tree->object.oid));
> > +     merge_finalize(&opt, &result);
> > +     return !result.clean; /* result.clean < 0 handled above */
> >  }
>
> The implementation is rather straight-forward, if you know how to
> drive the merge_incore_recursive() helper ;-)

:-)

>
> > +test_expect_success setup '
> > +     test_write_lines 1 2 3 4 5 >numbers &&
> > +     echo hello >greeting &&
> > +     echo foo >whatever &&
> > +     git add numbers greeting whatever &&
> > +     test_tick &&
> > +     git commit -m initial &&
> > +
> > +     git branch side1 &&
> > +     git branch side2 &&
> > +
> > +     git checkout side1 &&
> > +     test_write_lines 1 2 3 4 5 6 >numbers &&
> > +     echo hi >greeting &&
> > +     echo bar >whatever &&
> > +     git add numbers greeting whatever &&
> > +     test_tick &&
> > +     git commit -m modify-stuff &&
> > +
> > +     git checkout side2 &&
> > +     test_write_lines 0 1 2 3 4 5 >numbers &&
> > +     echo yo >greeting &&
> > +     git rm whatever &&
> > +     mkdir whatever &&
> > +     >whatever/empty &&
> > +     git add numbers greeting whatever/empty &&
> > +     test_tick &&
> > +     git commit -m other-modifications
> > +'
> > +
> > +test_expect_success 'Content merge and a few conflicts' '
> > +     git checkout side1^0 &&
> > +     test_must_fail git merge side2 &&
> > +     expected_tree=$(cat .git/AUTO_MERGE) &&
> > +
> > +     # We will redo the merge, while we are still in a conflicted state!
> > +     test_when_finished "git reset --hard" &&
> > +
> > +     test_expect_code 1 git merge-tree --write-tree side1 side2 >RESULT &&
> > +     actual_tree=$(head -n 1 RESULT) &&
> > +
> > +     # Due to differences of e.g. "HEAD" vs "side1", the results will not
> > +     # exactly match.  Dig into individual files.
> > +
> > +     # Numbers should have three-way merged cleanly
> > +     test_write_lines 0 1 2 3 4 5 6 >expect &&
> > +     git show ${actual_tree}:numbers >actual &&
> > +     test_cmp expect actual &&
> > +
> > +     # whatever and whatever~<branch> should have same HASHES
> > +     git rev-parse ${expected_tree}:whatever ${expected_tree}:whatever~HEAD >expect &&
> > +     git rev-parse ${actual_tree}:whatever ${actual_tree}:whatever~side1 >actual &&
> > +     test_cmp expect actual &&
> > +
> > +     # greeting should have a merge conflict
> > +     git show ${expected_tree}:greeting >tmp &&
> > +     cat tmp | sed -e s/HEAD/side1/ >expect &&
> > +     git show ${actual_tree}:greeting >actual &&
> > +     test_cmp expect actual
> > +'
>
> It is somewhat sad that we need to reivent merge test cases over and
> over, instead of easily reuse an existing one by replacing
>
>         git checkout one &&
>         git merge two
>
> with
>
>         git checkout one &&
>         T=$(git merge-tree HEAD two) &&
>         C=$(git commit-tree $T -p HEAD -p two) &&
>         git reset --hard $C
>
> ;-)

Sorry...I'm afraid I'm not following.